More Capable, Still Answerable

What we should now expect

Governance should make worthwhile collective action possible under authority people can question. It should protect their practical ability to understand, choose, and act, including when they depend on institutions and technologies they cannot scrutinize alone. We should expect those institutions to recognize when their arrangements no longer suffice and to have a responsible way to change them.

The question with which this book began—what should governance make possible, and how will we know when it needs to change?—now has a more demanding answer. A governing arrangement must connect its purposes to what it can actually accomplish. Its decisions need grounds, its responsibilities need practical means, and its account of success must encounter evidence beyond the account it prefers to give.

For someone seeking help, these connections determine whether a concern can reach competent attention and whether an appropriate response follows. For those developing useful technology, they determine what can responsibly proceed and on what terms. They also matter to people outside either institution who may bear consequences without having chosen the activity. Governing capacity must include their interests and their ability to question consequential power.

AI governance must maintain awareness as capability grows. That awareness needs sufficiently current evidence and room for correction, including knowledge of what remains uncertain. Developers and operators have essential knowledge; independent investigators, workers, users, and affected people can reveal what the established account misses. Different responsibilities call for different access. Credible inquiry does not require everyone to receive the same information or accept one interpretation.

Understanding must connect to legitimate authority. Evidence can establish an omission without giving its discoverer power to correct it. An institution can possess a mandate without the resources to fulfill it. Divided responsibilities can leave a consequential relationship unattended. A useful governing arrangement makes these connections work, rather than require the person raising a concern to resolve its internal divisions.

Authority must also connect to effects. Recording a restriction does not establish that the affected activity changed. Receiving an accurate warning after an irreversible consequence can support investigation while having failed to provide prevention. Where intervention cannot arrive in time, preventive bounds or narrower activity may be needed. Later inquiry and remedy remain duties even when they cannot restore everything lost.

The operating condition proposed here follows from these relationships: no AI system should operate at a scope, scale, or level of autonomy beyond the responsible institutions' demonstrated capacity to oversee it and constrain its consequential effects. Where capacity is inadequate, strengthen it, constrain the activity, or combine both. This commitment does not require complete knowledge. It requires material uncertainty to inform the grounds and conditions of permission.

A material loss of visibility on which authorization depended warrants reassessment of the affected activity before proven harm. The response must fit the missing grounds, the stakes, and the safeguards that remain effective. Not every missing record demands suspension; an earlier approval does not supply evidence that is no longer available. Those responsible should explain what remains justified and who can question that judgment.

These obligations give capability and answerability a common purpose. Stronger governing institutions should make useful reliance possible while preserving ways to discover when it becomes misplaced. Their own tools and permissions must meet that expectation too.

Capacity that deserves development

We must improve our capacity to govern as deliberately as we improve the technologies being governed. Deliberate improvement means making governing work a subject of inquiry, investment, testing, and correction. It means examining whether the whole arrangement can handle consequential activity, including interactions that separate competent reviews might miss. Faster processing alone cannot establish that achievement.

This brings the book's three subjects together: governance of AI; AI for governance; governance of AI-enabled governance. Computational assistance might help examine the activity being governed, preserve the conditions supporting permission, and carry out an authorized response. The same reliance creates a duty to examine what the assistance contributes, what it omits, and how it influences the exercise of power.

Accountability to people does not require people to perform every operation unaided. Suitable conventional software or AI can make some duties more practicable where its usefulness is demonstrated. Retained context could help a reviewer identify which earlier permission a change affects. Analysis could make an interaction examinable; simulation could reveal a question worth investigating. None of these contributions would by itself settle whether the activity should be permitted.

The institution must distinguish the task entrusted to a tool from the judgment and authority that remain to be exercised. Choosing what to measure, whose experience enters analysis, or which burden is treated as acceptable can shape a decision before anyone signs it. Those choices need identifiable responsibility and routes to challenge. A human signature cannot repair a recommendation that the reviewer has no practical means to question.

Developing capacity also means developing expertise, evidence access, resources, and working relationships. A more sophisticated submission is useful when someone can check its grounds and pursue a material question. Cheaper preparation does not establish lower total burden if the recipient must spend more effort verifying it or affected people must supply the missing inquiry. Useful assistance should strengthen understanding at a proportionate cost.

Across institutions, shared capacity can make inquiry possible that none could sustain alone. The Governance Labs proposed in this book offer one setting for developing and testing those connections. Their contribution would be evidence of what an arrangement can accomplish and where its limits lie. Research could inform an authorized decision; it could not supply missing powers or make a controlled test proof of adequate operation elsewhere.

Cooperation must preserve the ability to investigate differently. A common method can help participants relate evidence while also carrying a common omission. Smaller institutions need resources to exercise their responsibilities, not merely an invitation to join. Shared computation should support distinct mandates and independent judgment. Agreement about a technical finding can leave a legitimate political choice unresolved.

Nor does successful assistance justify every possible expansion. Some interactions may need to remain simpler, some permissions narrower, and some activities refused. More capable governance can make additional worthwhile activity justifiable; it can also establish stronger grounds for restraint. The standard is what can be governed adequately under legitimate terms, with those terms open to challenge.

What learning must change and preserve

An institution's ability to learn depends partly on what can enter its attention. Better analysis of an incomplete account may leave the important omission untouched. Learning must reach the questions asked, the evidence produced and checked, the experience heard, and the resources available to examine it. The consequences for people must remain part of the judgment, even when they are harder to establish than the number of completed reviews.

Correcting a decision is valuable in its own right. People should not have to wait for institutional redesign to receive help already justified. But the correction can also reveal something about how future decisions should be made. What did the earlier question leave outside the account? How could another unfamiliar concern prompt reconsideration? Improving the arrangements through which those questions arise can make learning more dependable.

This work deserves continuity. Reasons, supporting evidence, and the conditions of earlier judgments should remain available to appropriate scrutiny. Otherwise institutions can lose the experience they need to recognize a changed basis for permission or repeat a proposal whose weaknesses were already examined. Memory supports reconsideration as well as stability.

Dependable foundations and adaptable ways of working should sustain each other. A method can change while fulfilling the same duty more effectively. A change described as technical can also expand sensitive access or remove a protection. The authorization required depends on its substance and consequences. Experience can justify reconsidering a foundational arrangement without giving the institution permission to enlarge its own authority.

Repeated revision has costs: it uses resources, can exhaust people, and can make treatment unpredictable. Yet even after identifying an inadequate method, an institution may be unable to change it, because expertise, contracts, and accepted practices remain organized around its use. Deliberate improvement includes making justified alternatives practicable. That work needs authority, resources, and scrutiny of its burdens. The capacity to change must preserve dependable protection, including the ability to question whoever controls the improvement process.

Stable procedures and efficient administration remain genuine goods. They preserve dependable treatment and can make competent help available sooner. Their value should survive a reform where it remains justified. Learning can support retaining a safeguard, declining a proposed update, or simplifying work that no longer helps. Its quality cannot be inferred from the frequency of change.

A claim of improvement has an audience

Those who bear a revision's burdens need grounds on which to question its success. Saving an authority time may leave a smaller partner with more work or make an unusual circumstance harder to hear. A credible account of improvement makes those consequences examinable alongside the gains it reports.

Evaluation therefore needs room to question both the result and what was counted as success. Ask what became better, for whom, at what cost, and with what remaining uncertainty. Useful findings from a test retain their conditions; a favorable outcome after a revision does not automatically establish its cause. Independent inquiry and affected people's experience can challenge the preferred account without having to prove every part of it false.

Future choice belongs in that evaluation. Useful integration may create dependence on a method that is hard to inspect or replace. Recovery requires the capacity to sustain essential functions and protections, not simply restore a tool to operation. Some consequences cannot be undone. Their seriousness can justify protection before exposure while leaving duties of investigation and appropriate remedy afterward.

Immediate needs deserve weight alongside these longer commitments. People waiting for help should not bear indefinite delay for an unspecified future advantage. Institutions need reasons about actual alternatives, the distribution of burdens, and the protections that must remain. Lasting benefit includes making present help dependable without silently closing the choices others will need.

A responsibility we can share

The difficulties remain real. Evidence will be incomplete, resources unequal, and some consequential choices contested after careful inquiry. Governing power will still need limits. The argument of this book is for institutions able to decide and learn responsibly under those conditions, with their grounds and conduct open to challenge.

Responsibility is shared without being identical. Developers and operators should provide appropriate grounds for scrutiny and help establish what oversight their activity requires. Governing institutions need competence, resources, and legitimate powers to discharge their duties. Those examining their claims need practical independence and a route from findings to response. Delegation and cooperation must keep these responsibilities identifiable.

People outside those institutions can question purposes, contribute experience, and influence judgments about acceptable consequences and worthwhile improvement. Participation should make relevant contributions matter, with support for those whose circumstances make contributing difficult. It need not make everyone an evaluator or give every objection a veto. Protection cannot depend on someone's ability to complete an elaborate complaint.

We should expect governing capacity to be deliberately developed and its adequacy demonstrated for the activity permitted. We should expect consequential decisions and claims of improvement to have grounds that others can examine, with someone answerable for a justified response. These commitments preserve useful action, necessary restraint, and the ability to change course. They give technological achievement a public relationship with the people it affects.

For someone seeking help, these commitments mean an intelligible reason, a way to question a material premise, and someone equipped to investigate and respond.

Competent assistance should leave practical alternatives available. The institution carries its share of inquiry, including discovering what its usual account leaves out.

Governance that learns makes useful collective action possible while preserving people's ability to shape its terms. Institutions remain answerable for what they permit and what they must change.

The promise of governance that learns is fulfilled when people can understand their situation, influence consequential choices, and obtain the help they need.