> ## Content Index
> Fetch the complete content index at: https://www.aigl.blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# Response to the NIST RFI on Auditing, Evaluating, and Red-Teaming AI Systems
- URL: https://www.aigl.blog/response-to-the-nist-rfi-on-auditing-evaluating-and-red-teaming-ai-systems/
- Published: 2026-09-18T18:57:33.000Z
- Updated: 2026-09-18T18:57:33.000Z
- Description: This response to a NIST request for information sets out seven recommendations for AI safety, security, evaluation, auditing, and red-teaming of dual-use foundation models. It links proposed practices to the NIST AI Risk Management Framework and NIST AI 100-1.
- Author: Jakub Szarmach
- Tags: Briefing, NIST AI RMF, Model Evaluation, AI Security, AI Safety, AI Governance, #aigl-library

## ⚡ Quick Summary

Published by the Institute for AI Policy and Strategy, this response to a National Institute of Standards and Technology (NIST) request for information proposes guidelines, standards, and practices for auditing, evaluating, and red-teaming AI systems. Dated 2 February 2024, it focuses on Section 1 of the RFI, concerning AI safety and security, and explicitly does not address the RFI’s sections on synthetic content or responsible global technical standards.

The submission centres on dual-use foundation models (DUFMs), using the definition in Executive Order 14110, and also addresses generative AI in relation to NIST AI 100-1, a companion resource to the AI Risk Management Framework (AI RMF). Its seven recommendations call for capable and independent external scrutiny; incident response plans and thresholds for dangerous capabilities; earlier safety and security work through a “shift left” approach; independent layers of defence; clearer red-teaming terminology; threat modelling; and adversary simulation. The proposed measures connect repeatedly to AI RMF categories including Govern, Map, Measure, and Manage, with particular attention to CBRN risks, cyber risks, model autonomy, and catastrophic misuse.

## 🧩 What’s Covered

The document presents a summary followed by seven detailed recommendations in this order:

- **External scrutiny ecosystem:** Recommends secure and sufficiently capable access for independent auditors, evaluators, and red teams, including research APIs and structured access tools. It specifies access, independence, broad domain expertise, and resources such as time, funding, and compute, and links these points to Measure 1.3, Govern 5.1, Govern 3.1, and Manage 2.1.
- **Outcome-focused evaluations:** Argues that standards for dangerous-capability evaluations should identify desired outcomes, such as capability benchmarks related to CBRN weapons, rather than mandate technical methods that may become obsolete.
- **Incident response plans:** Recommends that DUFM developers maintain plans and response thresholds for dangerous capabilities alongside existing cybersecurity and adversarial-machine-learning incident arrangements. It proposes rehearsals, “deployment corrections” up to model decommissioning, and sharing plans with relevant US agencies; the discussion maps these ideas to Manage 2.3, 2.4, 4.1, and 4.3.
- **Shift left:** Adapts the DevSecOps principle of addressing safety and security early in development. Suggested stages are plan and design, data collection and processing, and model training and alignment; examples include behavioural requirements, dataset curation, and research on more robust techniques than post-training RLHF alone.
- **Defense-in-depth:** Calls for multiple measures with independent failure mechanisms in high-priority AI RMF activities, rather than merely overlapping controls. It highlights Govern 2.1, Map 5.1, and Measure 1.1, and illustrates diverse risk identification through scenario analysis, taxonomies, fishbone methods, independent teams, and adversarial analysis.
- **Red-teaming archetypes:** Distinguishes in-house stress testing, third-party stress testing, capture-the-flag events, and adversary simulation. For each, it describes the purpose and provides examples, while noting that the list is not comprehensive and that “red-teaming” can also refer to automated stress testing.
- **Threat modelling and adversary simulation:** Defines threat models as structured causal accounts linking catastrophic harms to capabilities, actors, and vulnerabilities. It recommends continual updates, causal mapping or probabilistic risk analysis, relevant specialist expertise, control groups, extended engagements, and responsible disclosure of results for severe misuse risks.

## 💡 Why it matters?

The response gives AI developers, evaluators, and public bodies a structured set of proposals for turning high-level AI safety concerns into governance and assurance activities. It treats external access and evaluator independence as conditions for testing developer claims, while connecting incident handling, risk identification, and post-deployment actions to named AI RMF subcategories.

Its distinction between red-teaming approaches is particularly relevant for teams deciding what a given exercise can establish. Threat modelling is presented as a basis for prioritising evaluations, and adversary simulation as a way to test whether model behaviours provide practical advantages in severe misuse scenarios rather than merely identifying isolated unwanted outputs.

## ❓ What’s Missing

The submission is deliberately limited to Section 1 of the NIST RFI and does not address the RFI’s sections on reducing synthetic-content risks or advancing responsible global technical standards. It is a set of recommendations to NIST, not a final NIST guidance document: it does not establish mandatory requirements, provide operational templates, or specify final resourcing thresholds and capability benchmarks. The paper also says its red-teaming archetypes are not comprehensive. For adversary simulation, it identifies an unresolved limitation: whether such exercises are cost-effective for smaller-scale risks or unintentional harms is unclear. The focus on DUFMs and severe misuse means other AI-system contexts receive less detailed treatment.

## 👥 Best For

Best suited to NIST-facing policy teams, frontier-model developers, independent evaluators, and AI assurance leads working on dangerous-capability assessment. It is especially useful for teams mapping evaluation, incident-response, and risk-management practices to the AI RMF, or designing red-teaming that involves external experts, structured threat models, and severe-misuse scenarios.

## 📄 Source Details

*Response to the NIST RFI on Auditing, Evaluating, and Red-Teaming AI Systems* was published by the Institute for AI Policy and Strategy on 2 February 2024\. The listed authors are Shaun Ee, Jam Kraprayoon, Joe O’Brien, Bill Anderson-Samways, and Zoe Williams. The complete 15-page English PDF was available for analysis; no edition or version number is printed.