⚡ Quick Summary
The AI Vendor Assessment Guide (v2.0) by Dennis Ah King offers a conversation-driven methodology designed to replace traditional, passive procurement questionnaires. Built to address the non-deterministic behavior, data exposure, and regulatory liabilities specific to artificial intelligence, the guide structures vendor evaluations into 12 core risk domains across three cumulative tiers: Tier 1 Basic (12 questions), Tier 2 Intermediate (22 questions), and Tier 3 Comprehensive (33 questions).
Aligned with ISO/IEC 42001, the NIST AI Risk Management Framework (AI RMF), and the EU AI Act, the toolkit includes a five-question upfront risk screener, four-part question guidance rubrics (Why It Matters, Look For, Watch Out For, Red Flags), and a clear four-tier post-assessment decision framework to qualify or disqualify third-party AI systems.
🧩 What's Covered
The guide establishes a complete operational workflow spanning pre-meeting alignment, real-time live scoring, and qualification decision-making:
- Strategic Foundations & Risk Tiering: A 5-point screener (evaluating decision impact, regulatory exposure, public visibility, board-level risk, and agentic workflows) that routes systems into Tier 1 (0–2 points), Tier 2 (3 points), or Tier 3 (4–5 points).
- 12 Assessment Domains: Structured conversational probes covering AI Governance & Oversight, Regulatory Compliance & Standards, AI Agent Scope & Autonomy, Model Performance & Explainability, Bias & Ethical AI, Data Management Practices, Security & Robustness, Privacy Compliance, Model Cards & Documentation, Incident Response, Commercials & SLAs, and Legal Liability & Insurance.
- Dynamic Dialogue & Evaluation Rubrics: Every question features an evaluator guide detailing underlying regulatory rationale, positive indicators, subtle warning signs, and critical red flags to differentiate substantive controls from vendor marketing.
- Live Scoring & Decision Matrix: A standardized scoring mechanism (Satisfactory, Follow Up, Concern) tied to an Assessment Tracking Summary and actionable next steps ranging from contractual hardening to engagement escalation or discontinuation.
- Companion Tool Integration: Outlines workflow integration with the browser-based AI Vendor Intelligence Scorecard and references the upcoming full AI Vendor Question Repository for deep-dive post-selection audits.
💡 Why it matters?
Standard procurement questionnaires routinely generate false assurance because vendors can script polished, evasive replies. By shifting third-party risk management into an active, live conversational interrogation with explicit red flags, organizations can expose superficial governance before signing contracts. Furthermore, explicit coverage of agentic autonomy, model fine-tuning deletion pipelines, and AI-specific liability caps ensures deployers do not inherit hidden regulatory exposure under the EU AI Act or GDPR.
❓ What's Missing
The guide primarily serves as an initial qualification tool rather than an exhaustive audit. It explicitly defers full technical security assessments, architecture reviews, and deep evidence verification to subsequent pipeline stages. Additionally, the comprehensive question repository referenced as a companion resource is noted as upcoming rather than embedded within this release.
👥 Best For
Procurement teams, AI governance officers, CISOs, privacy leaders, and enterprise buyers tasked with vetting external AI vendors, foundation model integrations, and agentic tools.
📄 Source Details
Title: AI Vendor Assessment Guide (v2.0)
Author: Dennis Ah King (AI Governance Blueprint)
Published: 2026
License: CC-BY-NC-SA-4.0
Framework Alignment: ISO/IEC 42001, NIST AI RMF, EU AI Act, GDPR
📝 Thanks to
Dennis Ah King for developing and releasing this structured practitioner blueprint under an open Creative Commons framework.
