The Day You Stop Reading

The following is an illustrative scenario, not a reported case. It is here to make the mechanism concrete. Appendix M tells you which claims in this book are documented and which are illustrative.

Imagine a clinician in a busy practice. She has adopted a triage tool that, by every metric she tracks, is a success. It gives her back ninety minutes a day. She uses those ninety minutes on her inbox. One afternoon, a four-year-old is brought in wheezing. The tool suggests a standard asthma pathway. The suggestion is reasonable. It matches the literature. She signs it and moves to the next patient.

The child's mother, on the way out, mentions — casually, as an afterthought — that the child started a new antibiotic two weeks ago, and the wheezing started three days after the antibiotic ended. The clinician is already turning to the next chart. She says, "That's common, keep an eye on it," and the mother leaves.

The antibiotic was a plausible culprit. The wheezing was not asthma. It was a drug reaction, and by Thursday the child is in the emergency department with respiratory distress that would have been manageable on Tuesday. The child recovers. The practice changes how it uses the tool.

What went wrong, in one sentence

A recommendation became a default without a named human owning the "no."

The pathway was not wrong in general. Whether the tool or the clinician erred is less useful than the shape of the failure: a good suggestion applied without the one check only a person could make. The error was human: a suggestion was accepted because it was good, and the one thing the tool could not do — read the message the way a person reads it, slowly, with the whole context, including the sentence said at the door — was not done.

This practice is not the practice in Chapters 19 and 24; those are separate illustrations of the same failure shape. This is illustrative, not reported. But the mechanism is not confined to medicine: the same shape appears wherever a good suggestion replaces a slow read — an AI-drafted email sent unread, an AI-summarized essay graded on the summary, an AI-suggested pull request merged on the strength of a green check. Only the setting changes.

The handoff

This is the quiet handoff. Not a coup. A convenience. A tool that worked, that the practice adopted, that gave the clinicians back ninety minutes, and that in exchange took the one thing they were supposed to be doing, which was reading.

The handoff is not "AI takes over." It is "no one can say whether it already has." That sentence is the thesis of this book. If you can finish that sentence for your own work — "I can still do this without the system, and here is the test" — you do not need most of this book. If you cannot, you are in the handoff, and you should keep reading.

The three myths

Myth one: the Terminator. The belief that danger looks like violence. It does not. Violence is usually legible, attributable, and interruptible. The handoff is none of those things. By the time anything looks like a Terminator, the relevant decisions were made years earlier by people who thought they were optimizing a workflow.

Myth two: nothing happens. The belief that if we have not seen catastrophe, the risk is overstated. But the handoff is not an event; it is an equilibrium. You cannot observe an equilibrium by waiting for an explosion. You observe it by checking whether the humans in a system can still do the system's job without the system. In the cases the editorial team examined — chosen because they interested us, not sampled — most could not. That is an impression, not a measurement, and Chapter 6 takes the objection seriously.

Myth three: someone is in charge. The belief that governments, labs, or standards bodies are managing this. Some are trying. None has a reliable mechanism for stopping a capability once it is cheap and useful. The default trajectory is deployment, not deliberation. The mechanism is in Chapter 2. The mistake is yours.