The Cipher

A rule can take effect before anyone has the chance to read it. A payment is refused, an offer disappears from view, a message is removed, or a record is assigned a score that changes what happens next. The decision arrives as an output. Somewhere behind it are instructions, data, thresholds, and choices about whose purposes the system is meant to serve. The earlier Author asked who writes the rules of a market. The Cipher follows that question into their execution. Software can make a rule consistent, rapid, and inexpensive to apply at scale. It can also make a consequential judgment difficult to locate: the person affected encounters the result while the authority to explain or change it is dispersed among the organization, the developer, and the system's operators.

The first distinction concerns what the machine has been asked to do. A program may execute an explicit rule, such as rejecting a message that exceeds a stated length. A statistical model may instead estimate a category from patterns learned in examples. The first can apply a clearly specified instruction to inaccurate data; the second can make an uncertain inference even when its inputs are recorded correctly. Both require choices about the objective, the relevant evidence, and the consequence attached to the output. Calling the result algorithmic does not identify which choice is disputed. A complaint may concern the policy itself, an erroneous record, a faulty implementation, an unsuitable model, or an excessive consequence imposed on an uncertain prediction.¹

A useful estimate still leaves a decision about errors and who bears them. Consider a hypothetical message classifier used on a thousand items, one hundred of which actually violate a specified rule. It flags ninety of those hundred and mistakenly flags forty-five of the nine hundred compliant items. Of the one hundred thirty-five flags, one third are wrong. Overall accuracy is nevertheless ninety-four and a half percent, because most items are compliant and correctly pass. The two percentages describe different features of the same system. A reader who sees only accuracy can miss the experience of the person whose harmless message was removed. The cost of each kind of error, and the procedure that follows a flag, belong in the design alongside the statistical score.²

Changing the population can change the meaning of the same measured error rates. Now suppose only one percent of ten thousand items violate the rule. At the same ninety-percent detection rate and five-percent false-positive rate, the system finds ninety real violations and flags four hundred ninety-five compliant items. Fewer than one in six flags is now correct. A threshold that was tolerable in one setting can become burdensome in another without any change to its source code. Conditions can also shift after deployment as language, conduct, or incentives change. The system therefore needs evidence from the context in which it operates and continuing checks on what its outputs mean there. A successful test at one moment supplies a starting point for governance, not an indefinite certificate of fitness.³

An explanation must reach the particular decision closely enough to make correction possible. Publishing source code may help an expert inspect implementation, but the affected person may need a different set of facts: which record was used, what criterion mattered, how uncertainty was treated, and who can correct a mistake. NIST's AI Risk Management Framework distinguishes transparency, explainability, and interpretability, and treats accountability as a matter for the people and organizations involved throughout a system's life. Those distinctions keep disclosure from becoming a ceremonial gesture. An intelligible general description is useful; an answer that cannot identify or change the basis of a harmful result may still leave the individual with no effective way to contest it.⁴

Human review is valuable when the reviewer has the information, authority, and time to exercise judgment. Placing a person after the machine does little if that person sees only its score, cannot obtain the underlying evidence, or is penalized for departing from it. A meaningful process records the applicable version, preserves the inputs needed to investigate, tests errors, and supplies a route to revise both individual outcomes and recurring defects. Different uses warrant different procedures: a reversible recommendation and an irreversible exclusion do not carry the same stakes. Logs and disclosure also need protection for private information and security. The aim is an accountable chain of decisions in which each responsible actor can explain a task and correct what lies within that actor's control.⁵

The cipher closes the institutions by returning their apparent impersonality to human authorship. The firm organizes, the contract binds, money settles, the tax finances, and the rule gives exchange its form. Software can carry each institution's instructions farther and faster, but the purposes, evidence, permissions, and remedies still require judgment. An executable rule becomes legitimate through the authority behind it and the treatment of the people subject to it; execution alone cannot supply those things. To read the code is one way of reading power. To govern it is to connect the result to reasons, responsibility, and correction. The Cycle now turns from the institutions that frame economic life to the movements of the whole economy as its many decisions rise and fall together.