⚡ Quick Summary
This paper proposes a user-centered framework for understanding and designing AI agent autonomy. Instead of treating autonomy as an inevitable product of increasing capabilities, the authors argue that autonomy should be a deliberate, designable property. The framework identifies five levels of autonomy, each defined by the role a user plays in the agent’s operation—from full control (L1: Operator) to full independence (L5: Observer). Each level is mapped to specific control mechanisms, agent behaviors, and open design questions. The authors also propose the concept of autonomy certificates—third-party issued documents that validate and govern an agent’s autonomy level. Finally, they recommend a novel assisted evaluation approach to assess autonomy independently from capability benchmarks.
🧩 What’s Covered
The core contribution is a five-level framework that classifies AI agents based on the user’s involvement:
- L1: Operator – User makes all decisions; the agent only acts on direct command (e.g., Microsoft Copilot).
- L2: Collaborator – User and agent share planning and execution, allowing fluid control handoffs (e.g., OpenAI Operator).
- L3: Consultant – Agent takes the lead, consulting the user only for expertise and preferences (e.g., Gemini Deep Research).
- L4: Approver – Agent operates independently, asking for user approval in high-risk or pre-defined cases (e.g., Devin).
- L5: Observer – Agent acts fully autonomously; user can only monitor or activate an emergency stop (e.g., Voyager).
Each level is illustrated with detailed examples, controls, user behaviors, and application contexts. The framework is intentionally decoupled from model capability, allowing a powerful model to be used at a low autonomy level for safety or user preference.
The second contribution is autonomy certificates, a governance mechanism that caps an agent’s permitted autonomy level. These are issued by third-party bodies based on “autonomy cases”—structured justifications akin to safety cases.
Finally, the authors propose assisted evaluations—a method for determining an agent’s autonomy level by incrementally increasing user involvement until task success is achieved. This offers a practical, testable alternative to code inspection or capability-only benchmarks.
💡 Why it matters?
As AI agents become more embedded in daily life and critical systems, questions of control, accountability, and safety intensify. This framework introduces a shared vocabulary and design tool for agent developers, policymakers, and safety researchers. By operationalizing autonomy as a configurable design parameter, it aligns AI agent behavior with governance goals and user preferences. The proposal of autonomy certificates helps bridge gaps in current safety regimes, offering a more nuanced mechanism than binary capability thresholds. Assisted evaluation further supports regulatory and technical audits by making autonomy quantifiable and testable in context.
❓ What’s Missing
While the framework is conceptually robust, it lacks empirical validation or user study results. The feasibility and adoption pathways for autonomy certificates are speculative, without detailing institutional incentives or legal grounding. There’s also limited guidance on how to practically integrate this framework into current AI development pipelines or product lifecycles. Finally, while multi-agent systems are mentioned, the nuances of agent-to-agent autonomy dynamics are underexplored.
👥 Best For
- AI product designers seeking user-aligned autonomy choices
- Governance researchers building agent oversight frameworks
- Safety auditors needing a taxonomy for agent evaluation
- Policymakers considering agent licensing or certification
- Developers contributing to multi-agent systems requiring coordination protocols
📄 Source Details
Title: Levels of Autonomy for AI Agents
Authors: K. J. Kevin Feng, David W. McDonald, Amy X. Zhang
Affiliation: University of Washington
Published by: Knight First Amendment Institute
Date: July 28, 2025
arXiv ID: arXiv:2506.12469v2 [cs.HC]
License: Public research publication
📝 Thanks to
Knight First Amendment Institute, MINT Lab, and reviewers including Seth Lazar and Beba Cibralic, for contributing to the theoretical and policy depth of this important work.
