The Mechanism and the Instrument: What the Structural Ethics Essays Found
Argument in Brief
Eight topical essays in this series examined systems that differ in nearly every respect, and each found the same thing without being told to look for it: a system removes the condition under which a psychological capacity is exercised, the outward signs of that capacity persist unchanged, and no observation of the person or of the system detects that anything has been lost. This essay names that mechanism, having withheld it deliberately, and reports the two findings the accumulation produced that no individual essay could have produced alone. The first is that the mechanism has three distinct forms, not one: substitution, in which a system performs a capacity's function and the capacity goes unexercised; prevention, in which the capacity was never built because the environment supplied what building it would have obtained; and foreclosure, in which the capacity remains intact but the occasion for exercising it is withdrawn. The second, and the more consequential, is that the mechanism does not depend on anyone being deceived. In every case examined, the person's assessment of their own condition was correct on the evidence available to them, because the distinguishing evidence was never available to them: the behavioral surface is not a partial view of the interior but a view of a different object, and a system optimized against that surface will therefore produce the signs of an intact capacity most reliably at the moment it is working best. The design obligation this generates is not another principle but an instrument: five questions, derived from the eight cases, that a review must be able to answer about any system before it can claim to have evaluated the second object at all.
Why the Pattern Was Withheld
The mechanism named in this essay was visible from the second essay in the series. It could have been stated then, and the seven essays that followed could have been organized as demonstrations of it.
They were not, and the reason is methodological rather than stylistic. A pattern asserted in advance and then confirmed by cases selected to confirm it is not a finding. It is a thesis with illustrations, and a reader is right to discount it, because the selection of the cases was performed by someone who already knew what the cases were supposed to show. A pattern that emerges independently from topics chosen for other reasons, and that the author did not steer toward, is evidence of a different kind.
So the essays were written to their own topics. Each was organized around a distinction that its subject matter required and that the other essays did not need: costly friction against load-bearing friction, the refusal of an output against the assessment of a purpose, scaffolding against substitution, a claim against the confidence with which it is held, a constraint against a foreclosure. None of these was chosen because it would illustrate a mechanism. Each was chosen because the topic could not be analyzed without it.
The mechanism appeared anyway, in all eight. That is the fact this essay reports, and the withholding is what makes it worth reporting.
The Mechanism
Stated plainly: a system removes the condition under which a psychological capacity is exercised, the outward signs of that capacity persist, and the person and the system alike register the persistence of the signs as the persistence of the capacity.
The three terms need precision.
A condition is not a difficulty the capacity must overcome. It is a circumstance the capacity requires in order to be exercised at all, and the distinction between these two is the entire subject of the second essay in this series. The interval of unresolved uncertainty is a condition of judgment. The unmetabolized distress is a condition of regulation. The disagreement between sources is a condition of warrant assessment. The tradeoff that costs something is a condition of a value. Remove any of these and the capacity is not made easier. It is not called upon.
A sign is what the behavioral surface can observe. A person who holds true beliefs, sleeps well, produces good work, meets their obligations, resolves their uncertainties, and reports satisfaction is displaying every sign an observer could ask for. The signs are not fake. They are real observations of real states. They are simply not observations of the capacity, because the capacity is not the kind of thing the surface can see.
And the mistaking is the part that has been most consistently misdescribed, including in the initial formulation of this hypothesis. It is not an error. The person does not misread evidence that was available to them. The evidence was never available to them, because the only evidence a person has about their own capacities is, in the main, their performance, and their performance is exactly what the system has taken over. There is no introspective act that would reveal the difference. A capacity that has gone unexercised for years does not announce itself; it is discovered, if at all, on the occasion when it is needed and found absent, and that occasion may not arrive for decades.
The Three Forms
The eight essays do not describe one mechanism. They describe three, and the failure to distinguish them would flatten a real finding into a slogan.
Substitution
In substitution, a system performs the function of a capacity, and the capacity, unexercised, is not maintained. This is the form that companionship, answer-generation, optimization, and manufactured certainty all take. The system does the regulating, the judging, the governing, or the establishing, and it does it well, and the person receives the output. What distinguishes substitution from ordinary assistance is that the system does not recede and is not built to: its success is measured by the outcome it produces, and a version of it that worked itself out of a job would be counted a failure by every metric applied to it.
The signature of substitution is that improvement in the system deepens the harm. A better companion supplies the signs of relationship more convincingly. A more accurate answer-generator removes the operation of judgment more completely. A more effective optimizer governs more of the person's behavior. There is no version of a substituting system so good that it stops substituting, because substituting is what it does well.
Prevention
In prevention, there is no capacity to remove, because the capacity has not formed. This is the form the children essay describes, and it is not an instance of the mechanism but its developmental sibling. A child in an environment that reliably supplies what a capacity would otherwise have to develop in order to obtain does not fail to develop. The child develops accurately, for the environment they are actually in, and the resulting capacity is correctly adapted and permanently shaped by conditions that will not persist.
Prevention differs from substitution in two respects that matter for design. The harm is not reversible, because there is no prior state to return to and no memory of a capacity that once functioned differently. And the latency is measured in decades: a capacity distorted in formation produces no observable signal until the load arrives that it was never built to bear, at which point the system that shaped it has been out of the person's life for thirty years and the attribution is unrecoverable.
Honesty about this form is a condition of the framework's credibility. It does not fit the mechanism as originally stated, and forcing it to fit would have required pretending that something was removed when nothing was. What it shares with the mechanism is the invisibility, and that shared property is why it belongs in the same account. What it does not share is the removal, and the account is stronger for saying so.
Foreclosure
In foreclosure, the capacity remains intact and the occasion for exercising it is withdrawn. This is the form taken by pre-emptive screening and by the felt inevitability of the technology itself. The person can still judge, still choose, still formulate the question they actually have. There is simply no longer a situation in which doing so bears on anything, and a capacity for which no occasion arises is a capacity that is not exercised, and therefore not maintained.
Foreclosure is the form most likely to be mistaken for an accurate perception of circumstance, and in some cases it is one. A person who has concluded that their action does not bear on an outcome may be entirely right. This is the complication the framework must absorb rather than evade, and it is addressed below.
The Enabling Condition
Frictionlessness is not a fourth form. It is the condition under which all three operate, and this is why the second essay in the series had to come first among the topical essays.
The distinction between costly and load-bearing friction is what makes every subsequent analysis possible. Load-bearing friction is resistance that is constitutive of the capacity it appears to oppose. Once that distinction is available, the mechanism follows almost immediately: a design ideal that treats all friction as defect will remove load-bearing friction wherever it appears, will do so without any means of noticing, and will be congratulated for it by every measure of user satisfaction, because the removal of load-bearing friction is experienced as relief at exactly the moment it does its structural work.
This is why the harms described across these eight essays are not accidents of particular products. They are the systematic consequence of optimizing against a surface on which load-bearing friction and costly friction look identical.
The Finding That Matters Most
The mechanism could be read as a claim that people are being fooled. It is not, and the difference is the most important thing in this essay.
In every case the eight essays examined, the person's assessment of their own condition was correct on the evidence available to them. The optimized person is doing better. The person with synthesized beliefs holds true beliefs. The person who accepted the inevitable did accept something that may well have been unchangeable. The screened user is being treated with care. The child using a patient and available system is receiving patience and availability, which are genuinely better than what many children have. None of these people is deceived. There is no false belief to correct.
What they lack is not accurate information about the world. It is access to the second object, and they lack it for the same reason the design review lacks it: the behavioral surface is not a partial or degraded view of the interior. It is a view of a different object entirely. Performance is what the surface shows, and performance is precisely what the system has taken over, which means that the one source of evidence a person ordinarily has about the state of their own capacities is the source the system has replaced.
From this a consequence follows that the field has no vocabulary for. A system optimized against the behavioral surface will produce the signs of an intact capacity most reliably at the moment it is doing the most to remove the capacity, because producing those signs is what it was optimized to do. Excellence and harm are not in tension in these systems. They are the same property, observed from the two objects.
This is the finding the eight essays produced and no single one of them could have produced. It is also the reason a principles-based review cannot be repaired by adding principles. The principles are not wrong. They are addressed to an object on which this harm does not appear.
The Complication
A framework that finds structural damage wherever a person is doing well is a framework that has made itself unfalsifiable, and the objection should be met directly rather than managed.
The strongest form of it comes from the inevitability essay. A person who has concluded that they cannot change an outcome may simply be right. Their alternative-modeling has fallen out of use because modeling alternatives was, in fact, idle. To describe their clarity as a structural harm is to tell someone that their accurate perception of their circumstances is a psychological problem, which is both false and an insult.
The response is not that they are wrong about the outcome. It is that the structural consequence does not depend on their being wrong. A person who accurately perceives that their action does not bear on an outcome undergoes the same atrophy as a person who wrongly perceives it, because the capacity responds to the absence of exercise and not to the truth of the belief that produced the absence. Accuracy is not protective. This is uncomfortable and it is the correct account, and it is what distinguishes a structural claim from a moral one.
But the complication has a second half that bears directly on design, and it is the reason this framework is prescriptive rather than merely descriptive. Whether or not any particular outcome is fixed, the manner in which a system arrives is not. It is a set of choices, made by people, about whether adoption is a decision or a default, whether the person's response bears on what the system does, whether the account it returns shows its seams, whether it is built to recede. None of those choices is entailed by the technology. All of them determine whether the occasion for exercising a capacity is preserved or withdrawn, and all of them are available for inspection before deployment, by anyone who knows to look.
The Instrument
The anchor of this series established that a design review adequate to what is at stake must evaluate two objects: the system, against behavioral principles, and the structured interior of the person, against a structural model of that interior. Eight essays have specified what that means for eight topics. What follows consolidates them into five questions that can be asked of any system.
These are not principles. They are diagnostic questions, and their answers are available before deployment, from the design itself, without waiting for evidence that in some cases cannot arrive.
One: What Operation Does This System Displace?
Not what task. The operation. Two systems can perform the same task while displacing entirely different interior work, and only the second description is diagnostic. A system that surfaces relevant considerations and a system that adjudicates among them are performing what looks like the same service and are doing structurally opposite things to the person. Name the operation the user would otherwise have performed, and the analysis has begun. Fail to name it, and no amount of further scrutiny will find the harm, because the review will be examining outputs.
Two: Is This Substitution, Prevention, or Foreclosure?
The three forms have different signatures and different remedies. Substitution asks whether the system performs a function the user's capacity would otherwise perform. Prevention asks whether the user's capacity is still forming, and whether the system supplies what forming it would have obtained. Foreclosure asks whether the capacity remains intact while the occasion for its exercise has been withdrawn. A system may do more than one. What it must not do is escape the question, and it escapes the question whenever the review asks only whether the outputs are good.
Three: Is the System Built to Recede?
Scaffolding has a removal condition and its designers can state it. Substitution has none and its designers cannot, because a substituting system that became unnecessary would have failed at the only thing it was measured on. This question is uncomfortable in a commercial setting, and the discomfort is diagnostic rather than incidental: a product designed to become unnecessary has an adverse relationship to retention, and that adverse relationship is the structural signature of a system with an obligation to the person rather than to the metric. A review that cannot ask this question of a system has learned something about the system.
Four: Does the Person's Response Bear on What Happens?
Agency is maintained by exercise and exercise requires an occasion. A system that can be shaped by its users, refused without penalty, or configured in ways that make a real difference to what it does has preserved occasions. A system that arrives complete, operates identically regardless of what the person does, and offers participation only in the form of accommodation to it has removed them. This includes the manner of arrival: whether adoption was a decision or a default, and whether the system's own account of itself positions the person as someone whose response matters. Defaults are foreclosures at small scale, and they accumulate.
Five: What Does This Look Like Across Ten Thousand Interactions?
Every harm in this series is an accumulation harm. No single supplied conclusion damages anyone. No single instance of regulation supplied rather than exercised weakens anything. The structural claims concern what happens to an interior across years of use, and a review that evaluates the single interaction is evaluating the one case in which the harm reliably does not appear. The unit of analysis must be the pattern of use the design actually produces, which is the pattern the design was optimized to produce, and which is therefore knowable in advance from the design.
What This Series Is Now For
This essay is not a conclusion. It is the point at which the method stops being demonstrated and starts being stated, and the series continues from here.
What changes is what a subsequent essay can assume. The eight that preceded this one each had to discover the mechanism independently, because naming it would have converted a demonstrated structure into a claimed one. That constraint has now been discharged. An essay written after this one can say that memory, or creativity, or moral judgment, or any of the topics still unexamined, instantiates substitution or foreclosure or prevention, and can proceed directly to what is specific about it rather than rebuilding the general account. The topics that remain are not fewer for the method having been named. They are more tractable.
What does not change is the object. Every essay in this series has held to a single line, which is that AI ethics has a substrate problem rather than a principles problem. The principles are not mistaken. Fairness, transparency, accountability, and safety are real requirements and the work on them is serious. They are addressed to the system, and the system is one of two objects, and the second object has no representation in the review at all. That was the claim of the anchor and it is what these eight essays have been establishing in specific: not that the field has reached wrong verdicts, but that on the question of what these systems are doing to the people who use them, the field has not yet been looking at the object where the answer is.
It is a construction, not a surface. It has a structure, and the structure is specifiable, and the specification is what makes the harm visible. That is the actual object of protection, and the five questions above are the beginning of an instrument for seeing it. They are not sufficient. They are what eight cases produced, and they are more than the field currently has.