⚡ Quick Summary
Commissioned by the UK Department for Science, Innovation and Technology (DSIT) and conducted by Lancaster University, this report presents a comprehensive thematic review and gap analysis of peer-reviewed AI cybersecurity research published between January 2021 and January 2026. Applying a PRISMA literature review methodology across Scopus and Web of Science databases alongside semantic matching data science techniques, the researchers analysed 9,109 publications mapped against 12 core lifecycle themes aligned with ETSI EN 304 223 and ISO/IEC 22989 standards.
The study reveals a stark concentration of research in training-time security (41%) and data/privacy risks (26%), alongside notable under-researched areas. Specifically, it identifies five major research gaps: formal verification methods for data integrity, traditional versus AI attack surfaces in agentic/generative infrastructure, end-user operational risks, third-party model provenance tracking, and safe end-of-lifecycle model disposal.
🧩 What's Covered
The report maps the academic and corporate research landscape across 12 AI security themes covering the full system lifecycle:
- Training Time Security (41% / 2,101 papers): Examines attacks subverting models during training, covering 21 identified types of poisoning attacks, label flipping, and backdoor exploits (13%), while noting a lack of formal guarantees for training data integrity (<1%).
- Data and Privacy Risks (26% / 1,313 papers): Focuses on membership inference (7%), backdoor interactions (6%), model inversion, extraction, and privacy-enhancing defences, with gaps in low-level data verifiability (2%).
- Alignment, Inference Time, and Supply Chain Security (~8% each): Covers jailbreaks, prompt injection (4%), adversarial transferability (<1%), Hugging Face ecosystem risks (2%), and third-party model provenance (3%).
- Emerging and Under-Covered Themes: Evaluates System Infrastructure Security (132 papers), Agentic-AI / Autonomous Agent Security (91 papers, showing a 216% year-on-year surge in 2025), End-User Risks (91 papers), End of Lifecycle & Disposal (76 papers), Model & System Design Security (55 papers), Failure to Track Assets / Shadow AI (24 papers), and Security Governance & Regulation (21 papers).
Quantitative analysis demonstrates that AI security publications surged from around 500 annually in 2021–2022 to over 3,000 annually across 2024 and 2025. Author affiliation and funding analyses show academic institutions leading authorship, with corporate co-authorship peaking over 900 papers in 2024. Geographically and financially, China and the United States lead global publication counts and funding bodies, followed by the UK, Australia, Singapore, and South Korea.
💡 Why it matters?
As organisations accelerate the deployment of Generative and Agentic AI, security risks expand well beyond standalone mathematical models into runtime infrastructure, complex supply chains, and human-agent workflows. The review demonstrates that contemporary research disproportionately concentrates on training and inference mathematical exploits while leaving operational deployment blind spots under-examined.
Crucially, the report calls for public policy action to establish a standardised, open-source vulnerability database for AI systems—analogous to traditional CVE platforms and OECD AIM incident trackers—enabling enterprises and regulators to systematically track, disclose, and mitigate real-world AI vulnerabilities.
❓ What's Missing
By design, the review explicitly excludes AI safety harms, societal misuse (such as deepfakes or offensive cyber operations), non-English publications, grey literature, and non-peer-reviewed corporate or government whitepapers. Methodologically, 2,296 papers lacked author keywords and were excluded from specific keyword analyses. Furthermore, the report notes that sparse counts in governance (21 papers) and design security (55 papers) may reflect proprietary enterprise practices rather than true gaps in technical capability.
👥 Best For
This report is essential reading for AI security architects, CISOs, AI governance officers, cybersecurity researchers, and technology policymakers seeking an empirical baseline of AI security maturity, lifecycle vulnerability distributions, and critical gaps across agentic systems and third-party model supply chains.
📄 Source Details
- Title: Thematic review and gap analysis on AI security
- Publishing Body: Department for Science, Innovation & Technology (DSIT), UK Government
- Research Lead: Lancaster University
- Publication Date: 10 July 2026
- Coverage Period: January 2021 – January 2026 (9,109 peer-reviewed papers)
- Standard Alignments: ETSI EN 304 223, ISO/IEC 22989, ISO/IEC 42001
- Licence: Open Government Licence v3.0
- Official URL: https://www.gov.uk/government/publications/thematic-review-and-gap-analysis-on-ai-security/thematic-review-and-gap-analysis-on-ai-security
📝 Thanks to
Research conducted by Lancaster University under commission from the UK Department for Science, Innovation and Technology (DSIT).
