⚡ Quick Summary
This guide, credited to Dennis Ah-king, is a question set for evaluating AI vendors before contracts are signed. It is designed to help organisations gather information about a vendor's security, safety, privacy and ethical practices and to prepare for vendor meetings. Version 1.01 is dated April 2025, and the work is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International.
The guide begins with internal grounding steps: define the business need and expected benefits, confirm that simpler alternatives cannot meet it, focus the deepest scrutiny on high-risk systems, tailor the questions to the organisation's risk tolerance and industry regulation, and treat the exercise as information gathering that must be followed by a separate formal threat and risk assessment.
It then moves through question areas: impact assessment, data sensitivity and criticality for the internal business client; AI governance and oversight; regulatory compliance and standards, naming SOC 2, ISO 27001, NIST CSF, ISO/IEC 42001, NIST AI RMF, the EU AI Act and state-level AI laws; bias mitigation and dataset provenance; third-party validation; data handling, residency, retention, deletion and ownership; robustness against prompt injection and data poisoning; content safety; vulnerability management and hallucination mitigation; explainability and model documentation; audit logs, monitoring and model drift; operational security; nth-party supply chain risk; and human oversight. Final notes recommend embedding the questions in an existing third-party risk management framework.
🧩 What’s Covered
- Before meeting a vendor: define the business need and expected benefits, confirm AI is the right solution, prioritise the deepest assessment on high-risk systems, tailor questions to the organisation's risk tolerance and industry regulation, and follow up with a separate formal threat and risk assessment.
- Internal business client questions: impact assessment (effects on rights, safety, health, finances, access to essential services), data sensitivity (PII, PHI, financial and confidential data) and criticality to core operations.
- Governance, compliance and ethics: AI governance bodies, ethical guidelines and responsible AI training; general frameworks (SOC 2, ISO 27001, NIST CSF) and AI-focused standards (ISO/IEC 42001, NIST AI RMF); tracking of the EU AI Act, state-level AI laws and other global regulation; bias and toxicity testing and dataset provenance.
- Data lifecycle: handling of inputs, outputs and feedback; segregation of client data; training use and opt-out; prompt review controls; data residency and geographic restrictions; encryption, bring-your-own-key and controls against inappropriate PII uploads; retention, deletion and termination procedures; ownership of generated outputs.
- Robustness and safety: defences against prompt injection, jailbreaking and adversarial inputs; protection of the training pipeline from data poisoning; adversarial training and red-teaming; input sensitivity; content filters and guardrails; model vulnerability testing and remediation; misuse prevention and hallucination mitigation.
- Transparency and audit: explainability and interpretability; model documentation, scorecards and communication of updates; audit logs and retention; performance monitoring; model drift; customer reporting; notice to end-users that they are interacting with AI.
- Operational security and supply chain: least-privilege access control, SSO, MFA and role-based access; API controls; session monitoring and exfiltration prevention; nth-party components and AI supply chain risk; incident response, business continuity and RTO/RPO.
- Implementation and final notes: human intervention, review and override in high-risk applications; disabling AI for users or the whole organisation; configurable guardrails; and final notes recommending integration into an existing third-party risk management framework with documented responses, contractual obligations and continuous monitoring.
💡 Why it matters?
Vendor due diligence for AI systems often stalls because buyers lack concrete questions to ask. This guide supplies a structured question set that procurement, security and privacy teams can put to a vendor, and it names the artefacts worth requesting: SOC 2 reports, penetration test summaries, model scorecards or factsheets, audit log retention periods and RTO/RPO targets. It also directs attention to AI-specific risks that generic software questionnaires miss, such as prompt injection, data poisoning, hallucination and model drift. Its own framing ties the questions to existing third-party risk management practice and to a formal threat and risk assessment of the solution before production.
❓ What’s Missing
The guide is a question bank rather than an assessment method: it offers no scoring, risk-rating or acceptance thresholds, no sample answers, and no mapping of individual questions to specific legal obligations. It states no jurisdiction and no scope beyond the buyer's own context, and it assumes the reader already has a third-party risk management framework and the authority to request evidence. Contractual, verification and monitoring steps are recommended in the final notes but not described. The regulatory examples, notably the EU AI Act and state-level AI laws, are dated to April 2025 and will need checking against current law. Data residency examples are illustrative only, and cost, commercial terms and post-contract performance review are not covered.
👥 Best For
Best for procurement, third-party risk and vendor management teams preparing AI vendor due diligence; security, privacy and compliance leads who need a structured question set for vendor meetings; and AI governance committees standardising how AI suppliers are assessed before contracts are signed and before systems enter production.
📄 Source Details
The document is titled AI Vendor Security and Safety Assessment Guide, credited to Dennis Ah-king, Version 1.01, April 2025. It runs to 22 pages in English and states no publisher, ISBN or reference number; page 1 prints the author's LinkedIn profile address, www.linkedin.com/in/dennisahking. The work is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International, © 2025 Dennis Ah-king (page 22). The input was the text extraction of all 22 pages, from the cover through the question sections to the licence page.
