✒️ Foreword
A graduate student found a spreadsheet error inside one of the most influential economic arguments of the austerity era.
In 2013, Thomas Herndon was doing what graduate students are routinely asked to do: reproduce someone else’s result. His target was Growth in a Time of Debt, the much-cited paper by Carmen Reinhart and Kenneth Rogoff arguing that economic growth fell sharply once public debt crossed 90% of GDP. The finding had travelled far beyond academia. In the aftermath of the financial crisis, it became part of the policy argument for cutting public spending.
But Herndon could not make the numbers work.
He checked his calculations. His professors checked them too. Eventually, Reinhart and Rogoff shared the spreadsheet behind the paper. Inside it was a remarkably ordinary error: one averaging formula stopped five rows too early, excluding Australia, Austria, Belgium, Canada and Denmark. That mistake mattered, but it was not the whole story. Herndon and his co-authors also challenged which years had been excluded and how countries were weighted. Correcting the formula and changing those analytical choices substantially weakened the dramatic 90% threshold.
That is what makes the episode more interesting than “Excel error changes economics.” The headline result looked simple: above this number, growth collapses. The machinery producing it was not simple at all. It depended on a chain of data selection, formulas, weighting decisions and assumptions that remained largely invisible until somebody tried to reconstruct the result from the beginning.
Replication did not merely check the answer. It exposed the path by which the answer had been produced.
That is becoming one of the central problems in AI governance.
As AI systems become more capable, the consequential question is increasingly not just whether a model produces an acceptable output. It is whether an organization can reconstruct how that output—or action—came about: which system had authority, which credentials it used, which tool it called, what data it accessed, what safeguards were active, who approved the action, who was supposed to challenge it, and what evidence remains afterwards.
The materials in this issue all attack that problem from different angles.
The recent guidance on agentic AI makes traceability almost literal. Agents should have distinct cryptographic identities, short-lived credentials and tightly scoped permissions. High-impact actions should cross predefined human approval gates. Authorization chains should be visible. Third-party tools and agents should be treated as supply-chain dependencies rather than trusted extensions of the model. And external testing arrangements are becoming more structured, including evaluations designed to see what models can do when ordinary safeguards are reduced.
CSET’s implementation guide approaches the same problem organizationally. High-level principles are not enough if nobody can tell where they become engineering tasks, tests, deployment gates, monitoring routines or incident-response procedures. Its value lies in connecting recommendations to lifecycle stages, system layers and responsible functions. In other words: make the path from principle to control reconstructable.
The Three Lines of Defense framework adds another layer: make the path of responsibility reconstructable too. Product and engineering teams own the risks they create. Risk, legal and compliance functions challenge and support them. Internal audit independently examines both. The model is imperfect, and the paper is refreshingly explicit about that. Responsibilities can blur. Independence can be weak. Information can fail to travel. But those weaknesses themselves point to the same governance requirement: an organization needs to know not only what controls exist, but who operated them, who challenged them, and who independently checked whether any of it worked.
This is a shift from governance as declaration to governance as reproducibility.
A policy saying “human oversight is required” is useful. A system record showing which human approved which action, under which conditions, is stronger. A statement that a model was tested is useful. An evaluation protocol showing what was tested, with which safeguards enabled, and what happened is stronger. A risk framework is useful. A chain linking the framework to an owner, control, test, monitoring signal and reassessment trigger is stronger.
Herndon’s story is memorable because something so consequential eventually came down to opening the machinery and following the calculation cell by cell. AI governance is heading toward the same demand, only the machinery is becoming much larger: models, agents, tools, permissions, people, vendors and runtime decisions interacting continuously.
The harder AI systems become to inspect from the outside, the more important it becomes to preserve the path inside.
Because sometimes the most important governance question is not whether the final number looks plausible.
It is whether someone else can reproduce how you got there.
— Kuba
Curator, AIGL 📚
☀️Spotlight Resources
Institutionalizing AI Safety: CISA’s Agentic Guide and CAISI Agreements
What it is:
A Cloud Security Alliance analysis of two developments in U.S. and Five Eyes AI safety governance: joint cybersecurity guidance for the secure adoption of agentic AI and the expansion of NIST CAISI’s pre-deployment testing agreements with major frontier AI developers. It examines how government oversight is moving from broad voluntary commitments toward operational security guidance and structured external evaluation.
Why it’s worth reading:
The strongest material is the translation of emerging government policy into enterprise controls. The report organizes agentic AI risk into five categories—privilege, design and configuration, behavioral, structural, and accountability risks—and highlights concrete measures including distinct cryptographic identities for agents, short-lived credentials, least privilege, predefined human approval gates, traceable authorization chains, and supply-chain vetting of third-party tools and agents. It also explains why CAISI’s testing of frontier models—including evaluations conducted with safeguards reduced or removed—matters for vendor due diligence and internal model assurance.
Operationalizing AI Guidance: A Reference Guide for Translating High-Level Goals into Practical Implementation
What it is:
A major CSET implementation guide designed to bridge the gap between AI governance principles and operational practice. Building on CSET’s earlier harmonization of 7,741 recommendations from 52 guidance documents, the authors reviewed more than 1,200 additional resources and converted the resulting framework into 232 practical implementation steps covering the full AI adoption lifecycle.
Why it’s worth reading:
Its main strength is navigation: recommendations are organized simultaneously by lifecycle stage, organizational level, responsible function, and underlying governance objective. The lifecycle runs from self-assessment and planning through acquisition, development, controls, testing, approval, deployment, monitoring, incident response, and improvement. Guidance is also separated across organization, operations, workforce, infrastructure, data, model, and system levels. Instead of inventing another high-level framework, the guide acts as an implementation map, showing practitioners what needs to happen, when it should happen, who is likely to own it, and where to find detailed methods for doing it.
Three Lines of Defense Against Risks from AI
What it is:
Jonas Schuett’s application of the established Three Lines of Defense risk-management model to organizations developing and deploying advanced AI. The paper focuses less on identifying individual AI risks and more on the organizational question of who should own, challenge, independently assess, and oversee those risks.
Why it’s worth reading:
The paper provides a useful governance architecture for separating responsibilities. The first line—research, engineering, and product teams—owns risks created by AI development and deployment. The second line—risk, legal, compliance, and related functions—provides expertise, establishes frameworks, monitors controls, and challenges first-line decisions. The third line—internal audit—independently assesses both other lines and reports to the board. Particularly useful is the discussion of AI-specific audit activities, including model, impact, governance, compliance, and risk audits. The paper also avoids overselling the framework: it explicitly notes weaknesses such as blurred accountability, bureaucracy, weak independence, poor information flow, and the limited empirical evidence demonstrating that Three Lines of Defense actually reduces risk.
