⚡ Quick Summary
Published by the National Institute of Standards and Technology, this report introduces a human-centred framework for explainable artificial intelligence (AI). It was informed by a NIST public workshop and public comment period, and places explainability among several characteristics associated with trustworthy AI, while focusing solely on explainable AI systems.
The framework sets out four principles. Explanation requires accompanying evidence or reasons for an output or process. Meaningful requires that intended consumers understand the explanation. Explanation Accuracy requires that the explanation faithfully reflect the reason for an output or the system process, distinguishing this from whether the decision itself is correct. Knowledge Limits requires operation only within intended conditions and at sufficient confidence.
The report characterises explanation by purpose and style, reviews risks, literature, algorithms and evaluation approaches, and considers humans as a comparison group. It presents the principles as a basis for future measurement and evaluation, rather than as universal metrics or a deployment procedure.
🧩 What’s Covered
- Definitions and principles: Defines explanation, output and process, then explains how the four principles apply to systems that produce explanations across the full range of AI techniques. It emphasises that an explanation may concern either a system outcome or its underlying process.
- Meaningfulness and accuracy trade-offs: Discusses how audience expertise, purpose and context shape meaningfulness. It explains that detailed accounts may better reflect processing while becoming less accessible, whereas concise explanations can be understandable without fully characterising a system.
- Purposes and styles: Distinguishes why an explanation is requested from how it is delivered. Style is organised by level of detail, degree of human-machine interaction—declarative, one-way and two-way—and format, including graphical, verbal, auditory and visual alerts.
- Risk management: Examines both benefits and risks of explanations. These include support for system understanding and regulatory compliance, alongside exposure of proprietary information, system vulnerabilities, misleading explanations and model use beyond knowledge limits.
- Literature and algorithm overview: Relates the principles to existing explainability and interpretability literature. It groups methods into self-interpretable models and post-hoc explanations, including local and global methods such as LIME, SHAP, counterfactuals, saliency methods, PDPs and TCAV.
- Adversarial and evaluation issues: Describes attacks that can manipulate post-hoc explanations or conceal unfairness. It reviews measures of meaningfulness, such as human simulatability, task accuracy, response time and subjective scales, and fidelity-oriented approaches to explanation accuracy.
- Humans as a comparison group: Reviews evidence on whether human explanations meet the same principles. It considers verbal interference with judgement, variation in audience understanding, unreliable introspection, and limited but sometimes useful confidence judgements about knowledge limits.
- Conclusions: Argues that the four principles are strongest when applied together and that explanation needs depend on practical context. It identifies human-machine interaction as an important area for further work on explainable systems.
💡 Why it matters?
The report gives AI developers, evaluators and governance teams a common vocabulary for separating four issues that are often conflated: whether an explanation exists, whether its audience can understand it, whether it is faithful to the system, and whether the system can recognise its operating limits. This distinction supports more focused design and assessment of explanations.
It also connects explainability to practical settings including quality control, customer relations, regulator and legal requirements, and system risk management. Its discussion of adversarial manipulation and intellectual-property exposure makes clear that providing explanations can create governance trade-offs as well as benefits.
❓ What’s Missing
The report deliberately does not prescribe deployment practices: it states that the principles concern humans consuming explanations and do not pertain to a system's usage during deployment. It offers a framework, literature review and examples rather than a checklist, control catalogue or implementation method for a particular sector. It also says that evaluation metrics for explanation accuracy remain under development and that there is limited work on developing and evaluating knowledge limits. The principles are presented as initial guidance expected to benefit from further refinement and community input, so readers seeking fixed thresholds or universal measures will need material beyond this report.
👥 Best For
This report is best suited to AI governance leads, product designers, developers and evaluators defining explanation requirements for a particular audience. It is also useful to risk, compliance and audit practitioners assessing whether system explanations are accessible, faithful to system behaviour and appropriately bounded by confidence or operating conditions.
📄 Source Details
Four Principles of Explainable Artificial Intelligence is NISTIR 8312, a 43-page English National Institute of Standards and Technology Interagency or Internal Report published in September 2021. Its authors are P. Jonathon Phillips, Carina A. Hahn, Peter C. Fontana, Amy N. Yates, Kristen Greene, David A. Broniatowski and Mark A. Przybocki. https://doi.org/10.6028/NIST.IR.8312
