News Ninja[1] is a game designed to educate players on detecting linguistic bias in news articles and gather data for improving automated bias detection systems.
- Key Concepts:
- Linguistic bias means using language to influence people’s views on events, groups, or individuals, often unknowingly.
- News Ninja helps players learn to detect these biases while collecting annotations (player-generated labels) to train automated systems.
- How News Ninja Works:
- Players go through an interactive tutorial on bias detection and then play games to annotate sentences and words as biased or not biased.
- The game uses game mechanics like feedback, rewards, and progress to motivate players and improve their skills over time.
- Five game modes provide variety and progression to keep the gameplay engaging and educational.
- Data Collection & Benefits:
- We compared game-collected annotations to expert labels.
- Results show that player-generated labels have a high inter-annotator agreement (10.28% higher than the baseline dataset), indicating good data quality.
- News Ninja educated players about bias while gathering high-quality data for bias detection systems.
- News Ninja can adapt datasets over time, updating them as biases evolve, helping to keep it current and reliable.
- Future Directions:
- News Ninja could be expanded to other types of bias and be integrated into educational settings to improve media literacy.
- Further studies are planned to analyze the long-term learning effects and ensure the game’s effectiveness in improving bias detection skills.
News Ninja approach offers a scalable, cost-effective solution to increase bias awareness and data collection for future automated bias detection systems to mitigate negative effects of media bias.
References
- (2024): News Ninja: Gamified Annotation of Linguistic Bias in Online News. In: Proc. ACM Hum.-Comput. Interact., vol. 8, no. CHI PLAY, 2024, (Place: New York, NY, USA Publisher: Association for Computing Machinery tex.articleno: 327).