Introduction

Developing the capacity to govern

We must improve our capacity to govern as deliberately as we improve the technologies being governed. That is the argument of this book. It asks us to give sustained attention to what institutions can understand, what they are entitled to decide, and whether their decisions can change what happens. Developing that capacity should be part of the work of developing artificial intelligence.

The purpose is practical. People should be able to benefit from useful capabilities without losing the means to question how those capabilities affect them. They need protection against harm and arbitrary power, dependable help, and a say in consequential choices. They should not have to become experts in every system on which their lives depend to obtain those things.

Artificial intelligence, or AI, is used here to describe computational systems that perform tasks such as generating language, analyzing information, making recommendations, and carrying out actions. Their governing significance depends on what they can do within the arrangements that give them access and permission. A recommendation considered by a reviewer and an instruction carried out directly can have different consequences, even when the underlying model is unchanged.

As capabilities and permissions change, the work of oversight can change with them. An institution may need to examine activities its earlier evaluations did not cover, or relationships its separate reviews did not reveal. It might receive more information while becoming less able to establish what matters. It might reach a sound conclusion and lack the authority or resources to respond.

AI governance inherits institutions as well as problems. Legislators, regulators, auditors, and courts work through established powers, professional skills, records, and procedures. These arrangements can preserve experience and protect people against arbitrary decisions. They also shape which questions institutions are equipped to ask and which answers they can use.

Past choices can make a method progressively harder to replace. Staff learn it; other procedures come to depend on it; alternatives require resources and justification that repeating the familiar method does not. This is a risk of path dependence: earlier choices reinforce a way of governing and make other paths harder to pursue. Continued use can still be justified. The concern arises when reinforcement keeps an inadequate method in place after its supporting conditions change.

A review can continue without examining whether its evidence still supports continued permission. Faster processing would leave that omission unresolved. Developing governing capacity includes examining what sustains the method, making justified alternatives practicable, and preserving the protections that change should serve.

These are possibilities to examine, not a verdict on all institutions. Adequacy depends on the whole arrangement, including its evidence, authority, resources, and effective response.

Better governing capacity can make worthwhile activity responsibly possible, expose a reason to narrow it, preserve an alternative, or correct a mistaken decision. Its value includes what it enables and what it protects.

The public question running through this book is therefore: What should governance make possible—and how will we know when it needs to change?

The arrangements we need to question

Governance is the set of arrangements through which a group or society sets direction, distributes and exercises authority, makes collective decisions, and determines how decisions and decision-makers can be challenged, corrected, or replaced.

This working definition describes governance without guaranteeing its quality. An institution can have clear responsibilities and orderly decisions while exercising power badly. We need to examine the purposes it pursues, the interests it protects, and the grounds on which it claims authority. We also need to ask whether its choices can be carried into effect and whether people can obtain correction when something goes wrong.

Procedures can make these achievements possible. A required review can bring a concern to attention before an irreversible action. A record can preserve reasons for later scrutiny. A stable rule can protect someone from the discretion of a powerful official or supplier. Restrictions can preserve people's ability to act by limiting what others may do to them.

The existence of these instruments leaves their performance open to examination. Completing a review establishes that a process occurred. Whether it supplied adequate grounds for a decision is a further question. An institution should be able to pursue that question without treating every deficiency as permission to discard its obligations.

The book connects three subjects. Governance of AI concerns the terms under which systems are developed and used, including their consequences for people. AI for governance concerns the assistance those systems might provide with evidence, analysis, coordination, and authorized action. Governance of AI-enabled governance concerns the authority and tools through which that assistance influences decisions. A tool used to scrutinize power can itself become a source of power that needs scrutiny.

These subjects belong together. We cannot judge an AI-assisted institution only by the efficiency of its work, or judge its governing tools only by the accuracy of their outputs. We must ask who may rely on them, who remains responsible, and what a person can do when the resulting decision needs to be questioned.

This book is a public manifesto, not a scientific study or a technical manual. It argues for a change in how we govern artificial intelligence—and in how governing institutions learn. It draws on existing ideas and evidence, but its central contribution is a set of principles, questions, and directions for action, not a scientifically validated model. Its purpose is to make an important public choice understandable: what should governance make possible, what must it protect, and how should it improve?

The reader needs no specialist qualification to take part in that choice. Evidence remains essential, and expertise deserves serious attention. But deciding what power should serve and how it should remain answerable is a public responsibility. The argument should be clear enough to question, including where its priorities differ from yours.

Priorities when worthwhile goods compete

The manifesto begins with a commitment to human agency:

AI should expand people's ability to understand, choose, and act—not diminish it.

The six priorities that follow give this commitment direction when worthwhile goods compete:

We value collective capability over procedural compliance.

We value learning through action over advance planning.

We value useful evidence over comprehensive reporting.

We value public accountability over administrative efficiency.

We value meaningful choice over operational convenience.

We value lasting benefit over immediate gains.

Both sides have value. The comparative form draws on the original Agile Manifesto's acknowledgment of that fact, while making a different argument about public governance.¹ These preferences need reasons and limits; they are not absolute rankings for every circumstance.

Procedural compliance supports dependable treatment and protects against arbitrariness. Collective capability asks what those procedures enable people and institutions to accomplish. A process that cannot recognize when its basis has changed needs examination through legitimate authority. Neither a valuable result nor an impatient developer can confer permission to disregard a binding duty.

Planning can prevent a harm that later learning would not undo. Learning through action asks us to pursue justified opportunities to acquire knowledge, including investigation, simulation, or a bounded test. Enforcing a limit can be action too. The preference does not make people involuntary experimental material or presume that deployment is always the next step.

Comprehensive reporting can reveal omissions and preserve information whose importance becomes clear later. Useful evidence asks whether the account supports understanding and consequential judgment. It may require more disclosure, independent inquiry, or an unwelcome finding. It cannot mean only the convenient subset an operator wishes to present.

Administrative efficiency can conserve resources and make help accessible. Public accountability asks whether those gains preserve usable scrutiny and challenge. Saving an institution effort by transferring the difficult work to someone seeking a remedy would need a different justification.

Operational convenience can make cooperation affordable and useful services easier to use. Meaningful choice asks whether people retain practical alternatives when an integrated arrangement no longer serves them. Convenience deserves weight without settling the terms of dependence.

Immediate gains matter to people with urgent needs. Lasting benefit asks whether help can endure, who bears its costs, and what today's choices leave others able to do. An unspecified future advantage cannot casually displace a pressing present need. Nor can a short-term success settle the dependence or loss of choice it creates.

The priorities do not erase rights, legal obligations, or essential protections. Benefits elsewhere cannot automatically compensate for their absence. Their purpose is to help us reason about a real choice, including its costs, rather than make every preferred course appear effortless. The chapters will develop those choices without assuming that reasonable disagreement disappears.

Learning that remains answerable

AI governance must maintain awareness as capability grows. Responsible actors need sufficiently current, evidence-supported, and correctable understanding of consequential activity, its effects, and the limits of their knowledge. That awareness includes useful opportunities. It must also leave room for independent inquiry and experience beyond the operator's preferred account.

Awareness is bounded by the responsibility and the stakes. It does not require knowing every internal computation or collecting everyone's private information. A reviewer needs grounds appropriate to the decision, with ways to discover a material omission. Acknowledging uncertainty is valuable when it changes what is investigated or permitted, rather than merely adding a disclaimer to an unchanged course.

The operating condition proposed in this book is that no AI system should operate at a scope, scale, or level of autonomy beyond the responsible institutions' demonstrated capacity to oversee it and constrain its consequential effects. This is a normative commitment. Where that capacity is insufficient, institutions should strengthen it, constrain the activity, or combine both.

A material loss of visibility on which authorization depended is itself a reason to reassess the affected activity. Waiting automatically for proven harm could defeat the purpose of preventive oversight. The appropriate response depends on the stakes and the safeguards that remain effective; not every missing record requires a shutdown.

Understanding must connect to legitimate authority and a response that can matter in time. A correct finding delivered to someone unable to act leaves the task unfinished. An accurate account received after an irreversible consequence can support investigation without having provided prevention. Better information cannot settle every political judgment or supply missing powers.

Computational assistance offers possibilities worth developing and testing. Conventional software or AI might help retain relevant context, examine connections, or check whether an authorized response took effect. Its usefulness needs demonstration in the work entrusted to it. More sophisticated analysis does not confer authority, and placing a person at the end of a process does not establish that the person can meaningfully question it.

Learning must also reach the questions institutions ask. An account can answer its assigned question well while omitting experience the decision needs. Governing capacity includes reconsidering what evidence is requested, whose experience enters judgment, and how a new concern can change the inquiry. Better preparation of the same incomplete account would leave the omission intact.

There is a serious objection to this ambition. Repeated revision can consume resources, exhaust people, and make rules unpredictable. A demand to adapt can become a convenient justification for weakening a safeguard. Dependable responsibilities and stable protections are part of what good governance achieves. Changes need authorization appropriate to their substance, evidence of usefulness, and scrutiny of who benefits and who bears the burden.

Learning can justify keeping an arrangement. Governance need not change whenever a model changes or reproduce the technology's internal complexity. It needs capacity adequate for the activity permitted and a way to recognize when that judgment no longer holds.

Learning can preserve a protection

An examination may establish that a safeguard should remain. A different method may fulfill the same duty more effectively, while a proposed update may weaken it. What learning establishes matters more than how often an institution announces a change.

Developing this capacity can be costly, and some useful activity may have to remain narrower while it is developed. Those costs deserve explanation. The obligation to improve should make institutions more answerable for their choices, including restraint, rather than give them an unlimited mandate to expand their powers.

The journey ahead

The argument proceeds through three parts. Purpose and Power examines what governance should protect and make possible, which concerns require attention, what governance must be able to do, and who may exercise governing authority. Knowing Enough to Act connects credible knowledge with computational assistance, justified response, and cooperation across divided responsibilities. Learning Without Losing What Matters considers practical choices, the assumptions behind reliance, institutional improvement, and scrutiny of the governors themselves.

The appendices collect the compact manifesto, governance amplifier questions, and illustrative assumptions register, followed by an optional exploration of the governance frontier. The public argument stands without that exploration. You should not need to memorize an architecture or complete a checklist to understand what is being proposed.

The expectation is demanding but intelligible: institutions should be able to explain why their capacity is adequate for what they permit, recognize when its grounds change, and respond under authority people can question. For someone seeking help, that means a consequential concern can reach competent attention and a justified decision can change what happens. For useful development, it means permission supported by dependable protection and effective oversight.

To judge either possibility, we must first ask what the governing arrangement is meant to accomplish. What is governance for?