⚡ Quick Summary
Published by the Institute for AI Policy and Strategy, this issue brief compares Anthropic’s Responsible Scaling Policy (RSP) with responsible capability-scaling guidance from the UK Department for Science, Innovation and Technology (DSIT). It defines responsible capability scaling as a system of progressively higher risk levels, broadly corresponding to model size or capability, that trigger progressively more stringent responses. The analysis focuses on safety and security risks from advanced AI models, particularly catastrophic risks.
The authors select seven areas from DSIT’s responsible-scaling sub-recommendations and assess Anthropic’s RSP qualitatively against them. They describe Anthropic’s four AI Safety Levels (ASLs), which pair capability thresholds with containment and deployment measures. The brief recommends verifiable, absolute risk thresholds based on societal-risk tolerances; distinct thresholds for specific risk types such as biological and cyber misuse; monitoring across development and deployment; defined reporting to government; preparations to pause training or reduce access; and external verification of evaluation methods and results from ASL-3. It argues that eventual standardisation of RSP risk thresholds should be undertaken by a government or industry body.
🧩 What’s Covered
The brief proceeds from its definition of responsible scaling to a qualitative comparison with selected DSIT practices:
- Purpose and scope: The introduction distinguishes this focused assessment from an earlier, broader review of company safety policies. It limits the comparison to Anthropic’s RSP, described as the original example of such a policy, and to DSIT’s more concrete recommendations on responsible scaling.
- Anthropic’s AI Safety Levels: The brief outlines ASL-1 through ASL-4 and the prospective ASL-5+. ASL-2 includes evaluations for ASL-3 warning signs and basic cybersecurity; ASL-3 adds intermediate cybersecurity, compartmentalisation of training techniques and hyperparameters, strong misuse prevention, and expert red-teaming across deployed modalities.
- Selection method: Four IAPS researchers independently selected DSIT sub-recommendations they considered essential to reducing catastrophic risks. Eight selected sub-recommendations are grouped into seven areas for analysis, including assessment, monitoring, thresholds, mitigations, reporting, pauses and verification.
- Risk assessment: The authors recommend combining forecasts of future capabilities with evaluations of present capabilities. They identify the risk that evaluations reveal examples rather than upper bounds of dangerous capability, and discuss deceptive alignment as another possible source of underestimation; suggested responses include mechanistic interpretability research and deceptive-alignment evaluations.
- Lifecycle monitoring: The analysis notes Anthropic’s evaluation schedule during training and fine-tuning—every three months and after each fourfold increase in computing power—and recommends explicit post-deployment monitoring, combining automated detection with human-led vulnerability detection.
- Threshold design: It contrasts relative and absolute risk, recommending absolute societal-risk thresholds. Drawing on a cited survey, it discusses maximum tolerances for events involving 1,000 or more deaths in the range of 1 E-04 to 1 E-10 events per year, with broadly acceptable tolerances generally two orders of magnitude lower.
- Action, disclosure and assurance: The brief supports commitments to continue mitigations until risk is below a target, to plan for pauses and coordinated action, to alert relevant government agencies at a specified threshold, and to obtain external verification of individual evaluation results at ASL-3 rather than waiting for ASL-4.
💡 Why it matters?
For AI governance and safety teams, the brief turns responsible scaling from a general policy concept into a set of operational questions: what risk boundary has been crossed, what mitigation must follow, who must be informed, and how compliance can be checked. Its comparison highlights that capability evaluations, deployment controls and pause commitments depend on sufficiently defined thresholds and evidence that mitigations work.
The document also links organisational controls across the model lifecycle. It calls for monitoring before, during and after deployment, including automated harm detection and human elements of oversight. Its recommendations on reporting, cross-company coordination and external audits are directly relevant where a model’s capabilities may create catastrophic misuse or security risks.
❓ What’s Missing
The brief is deliberately narrow. It assesses Anthropic’s RSP rather than comparing a range of company policies, although it notes that OpenAI has a Preparedness Framework. It focuses primarily on safety and security risks from advanced models and states that it did not explicitly consider other risks, including bias and discrimination. The assessment is qualitative and does not quantify its judgements. Several recommendations are expressly tentative: the authors acknowledge that very low probabilities are hard to measure, that a complete risk taxonomy cannot be expected immediately, and that standardised operationalisations of RSP thresholds remain to be developed. It therefore does not provide a finished, universally applicable set of thresholds or a full implementation specification for audits, government disclosures or coordinated pauses.
👥 Best For
AI safety and governance leads designing or reviewing responsible scaling policies; evaluation teams defining capability and risk thresholds; and public-sector or industry bodies considering common approaches to catastrophic-risk reporting, model-development pauses and external verification. It is particularly useful for readers assessing how ASL-style commitments translate into monitoring, containment and deployment measures.
📄 Source Details
Responsible Scaling: Comparing Government Guidance and Company Policy is an English-language issue brief published by the Institute for AI Policy and Strategy on 11th March 2024. It names Bill Anderson-Samways, Shaun Ee, Joe O’Brien, Marie Buhl and Zoe Williams as authors. The complete PDF was available for review; it comprises a cover and 20 numbered report pages.
