⚡ Quick Summary
Published by NCSC, this guidance explains the benefits and cyber security risks of using AI tools for managers, board members and senior executives without a technical background. Its stated aim is to help those leaders understand enough about AI risks to discuss them with key staff. It introduces AI, generative AI, machine learning and large language models (LLMs), including how LLMs are trained on large volumes of internet-derived text.
The guidance argues that cyber security is a precondition for the safety, resilience, privacy, fairness, efficacy and reliability of AI systems, and that security must be a core requirement throughout an AI system’s lifecycle. It identifies AI hallucination, bias, toxic-content generation, prompt injection and data poisoning as important flaws or vulnerabilities. The central organisational mechanism is a secure-by-design approach supported by leadership, culture, process and communication, rather than technical measures alone. Its practical deliverable is a set of questions leaders can use with technical and security staff on accountability, governance, assets, incident response, supply chains and skills gaps.
🧩 What’s Covered
The guidance moves from foundational concepts to security risks, leadership responsibilities and a discussion checklist.
- AI and generative AI: Defines AI as computer systems performing tasks that usually require human intelligence, and describes generative AI as tools producing text, images, video and multimodal content. It uses ChatGPT and DALL-E as examples of tools oriented to particular tasks or domains.
- Machine learning and LLM training: Explains machine learning as systems finding patterns in data or solving problems without explicit human programming. It describes LLMs as trained on large amounts of text-based data, potentially including web pages, research, books and social-media posts, and notes that filtering all training material is not possible.
- Wider adoption and security rationale: Links increased interest to AI integrations in internal and customer-facing products and services since ChatGPT’s release. It states that AI systems face novel vulnerabilities alongside standard cyber security threats and that security should extend beyond development across the lifecycle.
- AI-specific weaknesses: Lists incorrect statements presented as facts (“AI hallucination”), bias and susceptibility to leading questions, toxic-content generation, prompt injection attacks, and manipulation of training data through data poisoning. Prompt injection is described as input designed to make a model behave unintentionally, including revealing confidential information or triggering unintended consequences.
- Secure development guidance: Summarises the NCSC’s Guidelines for Secure AI System Development, developed with the US Cybersecurity and Infrastructure Security Agency and agencies from 17 other countries. These address design, development, deployment and operation, seeking secure outcomes rather than a static developer checklist.
- Leadership and organisational practice: Sets out secure by design as integrating security into AI projects and workflows from inception. It calls for leaders to understand consequences for integrity, availability and confidentiality; prepare for operational or reputational impacts; consider data security and legal compliance; and recognise that developers should take responsibility for customers’ security outcomes.
- Questions for organisational discussion: Provides nine questions for managers and boards, covering responsibility for AI/ML security, risk-aware deployment decisions, integration with governance processes, critical assets, worst-case failure, incident response, data/model/software supply chains, supplier assurance, and skills or knowledge gaps.
- Further reading: Directs readers to related NCSC publications on secure AI development, the near-term cyber threat, LLM risks, prompt injection, caution when building on LLMs, and principles for machine-learning security.
💡 Why it matters?
The guidance gives non-technical leaders a structured way to connect AI use with established security responsibilities. Its questions can help organisations assign accountability, bring AI/ML risks into existing governance processes, identify critical assets, and consider how they would respond to a serious incident. It also makes data, model and ML software supply chains an explicit management concern.
For teams building or deploying AI systems, the document reinforces that security is not confined to a development-stage review. It frames secure operation as dependent on lifecycle attention and on organisational leadership, while directing technical audiences to more detailed NCSC guidance and principles.
❓ What’s Missing
This is an introductory leadership guide rather than a detailed implementation manual. It names vulnerabilities and stresses secure-by-design practice, but does not prescribe technical controls, testing procedures, thresholds, or a method for assessing the severity of prompt injection, data poisoning, bias or hallucination in a particular system. Its organisational questions identify issues to discuss but do not provide model answers, maturity criteria, ownership structures or incident-response playbooks. Although it refers to legal compliance and established best practice for data handling, it does not identify particular legal regimes or set out legal obligations. Technical detail is instead signposted through the listed NCSC guidelines, principles and blogs.
👥 Best For
Managers, board members and senior executives overseeing AI or ML adoption, especially those preparing conversations with technical and security staff. It is also useful for risk owners who need to incorporate AI/ML risks into governance processes, plan for incidents, and question suppliers about data, model and software supply-chain security.
📄 Source Details
AI and cyber security: what you need to know is English-language guidance published by NCSC. It is version 1.0, marked “Published 13 February 2024” and “Reviewed 13 February 2024”, and is four pages long. No individual authors or document URL are printed in the supplied complete PDF.
