⚡ Quick Summary
Published by the Centre for the Governance of AI (GovAI), this policy brief is the first in a series and analyses how frontier AI companies assess the risk their models add relative to competitors' models. It defines marginal risk as a difference in risk relative to a baseline, contrasts it with absolute risk, and lists possible baselines: human performance, the state of the world at a previous point in time, a company's own previous model, and a competitor's model. As of October 2025, thirteen organisations had published a safety framework, and the brief identifies at least six — Anthropic, OpenAI, Google DeepMind, Meta, Microsoft and Amazon — that explicitly or implicitly refer to marginal risk relative to competitors. It sets out the justifications given by Anthropic and OpenAI, breaks verification into three high-level steps, and identifies conditions under which total risk would not stay constant, including awareness, low switching costs and no complementarities. The conclusion is that this approach to marginal risk needs further scrutiny and should not be accepted as best practice without significant analysis and public discussion.
🧩 What’s Covered
The brief addresses five questions: what marginal risk is, which companies assess it relative to competitors, how they justify the approach, why the justification may be flawed, and why this is a problem.
- Definitions and baselines: absolute risk is the total amount of risk; marginal risk is a difference in risk relative to a baseline, and is easier to measure because catastrophic outcomes are rare and context-dependent. Baselines include human performance, models available in 2023, a company's own earlier model, and a competitor's model (Figures 1 and 2).
- Industry context: marginal risk is established in nuclear energy and transport, including the European GAMAB principle; the Frontier Model Forum has warned of "risk creep"; the EU AI Act and the EU GPAI Code of Practice leave unclear whether safety risks, as opposed to security risks, may be assessed against competitors.
- Company practice: profiles of Anthropic, OpenAI, Google DeepMind, Meta, Microsoft and Amazon, tracking which framework version introduced, narrowed or removed the concept, with excerpts reproduced in the Appendix.
- Justifications: Anthropic says the incremental increase in risk attributable to it would be "small"; OpenAI says a competitor's release would "limit the degree" to which it can reduce risk, and commits to staying "more protective" and sharing validating information; competitive disadvantage is discussed as an unstated driver.
- Three verification steps: deciding whether another model passed a capability threshold, whether its mitigations are equivalent, and whether lowering one's own would not meaningfully increase total risk. Failure modes include false positives, false negatives, withheld mitigation details and the absence of frameworks for evaluating mitigations.
- Risk types and conditions: Table 1 lists awareness, low switching costs and no complementarities as conditions for misuse risk to stay constant; accident and structural risks may scale with the number of exposed users; autonomy risks may compound, such as two models each carrying a 1% loss-of-control chance.
- Consequences: inaccurate assessment can raise total risk; standards can erode through a "race to the bottom" or a "boiling frog" effect; public trust may fall and legal liability may rise, illustrated by an asbestos analogy and doctrines that abandon "but-for" causation.
💡 Why it matters?
For teams drafting or reviewing frontier safety frameworks, the brief supplies a structured critique of one mechanism: allowing a competitor's weaker mitigations to justify lowering one's own. It sets out the evidence a company would need — capability assessment, mitigation equivalence, and an argument that total risk would not rise meaningfully — and shows where each step can fail. It also links the practice to existing expectations, noting the Frontier Model Forum's "risk creep" warning, the ambiguity in the EU AI Act and the EU GPAI Code of Practice, and liability exposure where a company weakens mitigations it had previously adopted.
❓ What’s Missing
The brief offers no method for conducting the assessments it criticises: it identifies three steps and their pitfalls but no benchmarks, thresholds or disclosure format. Only six of the thirteen published frameworks are analysed, and the authors note that most companies give too little detail for comparable depth. The legal liability discussion is explicitly out of scope, the brief has not been peer reviewed, and mitigation and policy responses are deferred to later briefs. Table 1's conditions are described as not exhaustive.
👥 Best For
Safety framework authors and reviewers at frontier AI developers, policy analysts comparing company commitments, regulators and standards bodies asking whether competitor-relative mitigation adjustments are acceptable, and risk or legal teams weighing the liability and reputational consequences of lowering mitigations in response to a rival's release.
📄 Source Details
Assessing Risk Relative to Competitors: An Analysis of Current AI Company Policies, a GovAI policy brief published by the Centre for the Governance of AI and dated October 2025 on the cover. Authors: Sophie Williams, Noemi Dreksler, Aidan Homewood, Markus Anderljung and Jonas Freund. The supplied file runs to 19 pages and the extraction covers all of them. The brief states that it represents the authors' views and has not undergone an official peer review. No URL for the document itself is printed in it.
