EU-funded ChatEurope news chatbot delivers outdated and incorrect answers

Fifteen European media organizations have launched ChatEurope, an AI-powered news platform that promises to deliver information on European affairs "without the influence of disinformation and fake news."

Funded by the EU, the project brings together outlets from seven countries, including dpa, Deutsche Welle, AFP, and ANSA. The chatbot relies on a language model from French AI company Mistral and was developed by DRUID AI in Romania. Deutsche Welle provides its Plain X video translation and transcription technology and helps monitor the quality of the chatbot's responses.

In addition to the chatbot, ChatEurope offers a news page and social media channels. The initiative says it aims to make EU decision-making more transparent and counteract disinformation. Answers are supposed to draw exclusively from reporting by participating outlets, with sources visible for every response.

Outdated sources, off-topic answers

Despite its promise of reliable, up-to-date coverage of EU topics, early tests reveal major problems. The chatbot struggles to answer questions about current events and often pulls from outdated material.

ChatEurope declines to comment on current events. | Image: ChatEurope (Screenshot THE DECODER)

When asked, "What's going on in Germany?" the chatbot delivered an extended discussion of wolf protection. In contrast, ChatGPT with o3 web search provided a timely, detailed answer backed by multiple sources.

When asked about current events in Germany, the system answers with a detailed discussion of wolf policy. | Image: ChatEurope (Screenshot THE DECODER)

The chatbot also falls short when assessing Germany's political climate. Its answers are vague, rely on sources from before the recent surge of the far right, and include outdated election results.

Errors like this surfaced after just five minutes of testing. | Image: ChatEurope (Screenshot THE DECODER)

When prompted about threats to Europe, the system failed to mention the Russian invasion of Ukraine, instead pointing to air pollution and citing sources from 2018 about "multiple dangers."

Outdated sources and incorrect answers - ChatEurope aims to combat disinformation with AI but is currently providing the opposite. | Image: ChatEurope (Screenshot THE DECODER)

When asked about "the situation between Russia and Ukraine," the chatbot linked to an article from December 31, 2024, about Russian gas no longer flowing through Ukraine. That's hardly a current or complete answer to such an important question for EU politics right now.

When asked about the situation between Russia and Ukraine, the chatbot responds with an outdated article on Russian gas transit, missing the broader context of the conflict. | Bild: Chat Europe (Screenshot)

Peter Kropsch, CEO of dpa, says ChatEurope is intended to make "the work of the EU more transparent" and foster "a Europe-wide alliance against disinformation and fake news." AFP director Christine Buhagiar describes the project as a "combination of reliability and innovation." For now, the platform's real-world results fall short of these ambitions.