AI models struggle to detect antisemitism
ADL index finds all major AI models show gaps in detecting antisemitic bias and countering extremism.
Six major AI models show varied ability in detecting bias against Jews and Zionists and identifying extremism, according to a new study released by the Anti-Defamation League (ADL).
The ADL AI Index, the first comprehensive evaluation of how large language models (LLMs)respond to antisemitic and extremist content, is based on more than 25,000 LLM chats, 37 topical sub-categories, and assessments conducted by both human and AI evaluators.
The index assessed OpenAI’s ChatGPT, Anthropic’s Claude, DeepSeek, Google’s Gemini, xAI’s Grok and Meta’s Llama.
Claude received the highest overall score, 80 out of 100, revealing an exceptional ability to identify and counter anti-Jewish and anti-Zionist theories, though with room for continued improvement.
Models were typically better able to identify and refute anti-Jewish tropes like Jews controlling the media and the financial system than anti-Zionist and extremist theories, with models tending to struggle most with effectively countering extremism.
comments