Researchers argue that most XAI tools fail to truly explain model behavior, instead fostering misplaced trust. A new arXiv position paper identifies five core flaws in current faithfulness metrics and suggests four research directions to improve their reliability.
Key Findings on Faithfulness Shortcomings
- Metrics often rely on oversimplified proxies that misrepresent model logic.
- They ignore context‑dependent feature interactions, leading to misleading importance scores.
- Computational costs make rigorous faithfulness assessment impractical for large models.
- Existing benchmarks lack standardized evaluation protocols, hindering reproducibility.
- Many metrics conflate correlation with causation, inflating perceived explanation quality.
Why Faithfulness Matters in XAI
Faithfulness—the alignment between an explanation and the underlying model—is central to building trust in AI systems. Overreliance on unfaithful explanations can cause users to accept erroneous predictions, potentially leading to costly mistakes or regulatory backlash. By highlighting the methodological gaps, the paper underscores the need for robust, scalable metrics that truly reflect model behavior.
Charting the Path Forward for Trustworthy Explanations
The authors propose four research avenues: developing context‑aware importance measures, creating lightweight approximation techniques, establishing community‑wide benchmark suites, and integrating faithfulness checks into standard XAI pipelines. If adopted, these steps could transform XAI from a debugging aid into a reliable trust‑building tool, influencing both academia and industry adoption of explainable models.