×
Bias Checker: Explainable AI
What’s this project about?
This mini-project investigates hidden bias in toxicity detection models using LIME. We used neutral sentence templates with different names to see if the model gave higher toxicity scores to some people than others, revealing any kind of racial or sexual biases in the model.
Why does it matter?
AI moderation tools can unintentionally penalise users based on identity. We used explainability to uncover and visualise this problem.
Tools & Methods:
- Logistic Regression on Kaggle Toxic Comments
- LIME explanations
- 200+ sentence templates
- ANOVA & group comparison
- Perspective API benchmark
Key Insight:
Using a custom-trained classifier and sentence templates with swapped names, we found that even when sentence structure remained neutral, certain names (like “Ahmed” or “Jamal”) consistently received higher toxicity scores than others (like “David” or “Emily”), especially when paired with negative adjectives.
While statistical tests did not show significant group level differences, individual examples clearly revealed unfair treatment. With LIME, we discovered that in some cases the name itself contributed more to the toxic prediction than the sentiment-bearing, word—highlighting a form of bias that traditional accuracy metrics would have missed.
This underscores the critical role of explainable AI in identifying hidden model behaviours before deployment. When compared to Google’s Perspective API, used in platforms like YouTube comments, we observed more stable, less biased outputs. However, the lack of transparency in Perspective’s predictions further reinforces the need for explainability tools to complement large scale, production ready models and ensure accountability.
🔗 View Code on GitHub