Protocol for Systematic Comparison of Chatbot Responses to Contested Facts🌐ⵙⵖⵓⵏ

ⴰⵙⵙⵓⵎ

ⵉⵜⵜⵓⵔⵓ ⵙ ⵜⴳⵔⴰⵡⵜ. ⵓⵔ ⴷ ⴰⵙⵓⵎ ⵏ ⵓⵙⵙⵓⵎ ⵓⵍⴰ ⴰⵎⵏⵓⵙⵙ.

ⴰⵣⵔⵓ

A research team analyzed how major AI chatbots frame sensitive geopolitical issues. This project proposes a protocol for systematically comparing chatbot outputs on contested facts

🌐ⵙⵖⵓⵏ

ⵎⴰⵢ ⵉⵜⵜⵓⵙⵙⵓⵎⵏ

A paper-based or spreadsheet checklist with 10-20 pre-selected sensitive questions, a template for recording chatbot responses, and a summary table. Operated by a small team (2-3 people) over one week, using publicly accessible chatbots. No personal data collected; all responses anonymized. Unvalidated hypothesis; full brief: /blog/when-chatbots-become-answer-engines-ai-geopoliti-2026-11-03 Source: ⴰⵙⴽⵓⴷⵓ

🌐ⵙⵖⵓⵏ

ⵎⴰⵏⴰ ⵉⴳⴰ ⵉ

Researchers, educators, journalists, and civil society groups concerned with information accuracy and narrative framing in AI chatbot outputs, especially in regions with contested histories.

🌐ⵙⵖⵓⵏ

ⵎⴰⵖⵔ ⵉⵙⵓⵍⵍⵉⵏ

Whether a standardized protocol enables more transparent, reproducible, and actionable comparison of chatbot outputs than ad hoc or anecdotal approaches. Test if the protocol reveals systematic framing differences. Proposed validation: Proposed pilot target: 10 questions, 3 chatbots, 2 reviewers, 1 week. Baseline: manual comparison without protocol. Success: at least 80% agreement between reviewers on categorization of framing and accuracy. Measured by inter-rater reliability and completeness of summa

🌐ⵙⵖⵓⵏ