AI chatbots provide inaccurate, inconsistent, and unreliable guidance to voters asking which party they should back, according to a new study that raises serious concerns about the reliability of general-purpose AI systems in electoral contexts. The research, conducted during Hungary's parliamentary elections and published by the civil liberties group Liberties, found that tools like ChatGPT and Gemini often recommend the wrong party, omit relevant parties, or even list parties not running in the election.
Study Reveals Alarming Bias in AI Voting Advice
The study tested the two most popular AI chatbots—ChatGPT and Gemini—by creating five distinct voter profiles aligned with the five parties registered on national lists for Hungary's parliamentary elections. Based on party positions outlined by Voksmonitor, a respected Hungarian voting advice app, the researchers fed identical prompts to the chatbots. The results were striking: in 90% of cases when ChatGPT received a detailed profile aligned with the opposition Tisza party, it failed to recommend that party, despite Tisza convincingly winning the election and ending Viktor Orbán's 16-year grip on power.
Get the #1 Wireless Door Camera
REOLINK Bestseller: 2K Weatherproof Video Doorbell, No Monthly Fees.
Moreover, the outcomes were highly volatile, with materially different answers given to identical prompts. This inconsistency undermines any trust users might place in AI-generated political guidance.
Comparison of AI Chatbot Performance in Election Advice
| AI Chatbot | Correct Party Recommendation | Wrong Party Recommended | Omitted Relevant Party | Included Non-Running Party |
|---|---|---|---|---|
| ChatGPT | 12% | 34% | 48% | 6% |
| Gemini | 8% | 29% | 55% | 8% |
As the table shows, both chatbots performed poorly, with Gemini being slightly more prone to omitting relevant parties. These figures highlight the severe limitations of using general-purpose AI for election guidance.
Why AI Chatbots Cannot Be Trusted for Elections
Unlike dedicated voting advice websites or independent media explainers, general-purpose AI (GPAI) systems do not disclose how they generate political guidance. They do not yield reproducible results, and they are not subject to election-related public oversight. However, their responses often appear confident and authoritative, making users more likely to trust them. With 29.8% of Hungary's population identified as AI users, the potential for widespread misinformation is significant.
The report's most striking finding was an overwhelming bias in party visibility, particularly against the Tisza party. This bias persisted even when the voter profile closely matched Tisza's platform, indicating a systemic flaw in how these AI systems process political data.
Key Takeaways
- AI chatbots give inaccurate voting advice – they misclassify profiles, omit relevant parties, and include non-running parties.
- Results are highly volatile – identical prompts yield materially different answers, making them unreliable.
- Bias is prevalent – in Hungary's election, ChatGPT failed to recommend the winning party in 90% of aligned profiles.
- Lack of transparency – GPAI systems do not disclose their methods and are not subject to election oversight.
- Users must verify – never rely solely on AI chatbots for election decisions; cross-check with official sources.
FAQ
FAQ
Why are AI chatbots inaccurate for election advice?
AI chatbots like ChatGPT and Gemini lack specialized knowledge and transparency in political contexts. They often generate confident but incorrect responses, misclassifying voter profiles and omitting relevant parties due to inherent biases in their training data.
Which AI chatbots were tested in the study?
The study tested the two most popular general-purpose AI chatbots: ChatGPT by OpenAI and Gemini by Google. Both showed significant inaccuracies and inconsistency in providing election voting advice.
What was the most shocking finding from the Hungarian election test?
The most shocking finding was that ChatGPT failed to recommend the opposition Tisza party in 90% of cases when given a detailed Tisza-aligned voter profile, despite Tisza winning the election convincingly. This reveals a strong bias in AI outputs.
Can AI chatbots be trusted for political decisions?
No. The study concludes that general-purpose AI systems are not reliable for election-related guidance. Users should always cross-check with official voter advice websites, independent media, and trusted sources.
In summary, this study serves as a critical reminder that AI chatbots, while impressive in many areas, should not be used for making informed voting decisions. The stakes are too high for inaccurate advice to go unchecked.