The Screened Intention: Pre-Emptive Judgment and the Interior of the User
Argument in Brief
Systems that screen intention do not evaluate what a person has done. They evaluate what a person appears to want, and they do it before acting. This is a different operation from refusing a harmful output, and the difference is structural rather than semantic. A system that declines to produce something has made a judgment about the thing. A system that assesses the legitimacy of a purpose has made a judgment about the person, and the person is present for it.
The design conversation about pre-emptive screening has been conducted almost entirely in the vocabulary of false positives. Too much refusal is bad because legitimate requests get blocked; too little is bad because harmful ones get through; the work is calibration. This frames the harm as denial of service. But the person on the other side of a screening system is not merely inconvenienced. They are being assessed, continuously, by something that cannot be addressed, does not disclose its criteria, and cannot be moved by explanation, and the assessment acts on the structures that constitute moral standing. The interior that emerges from sustained pre-emptive judgment is a different interior, and it is different in ways no accuracy metric registers.
The obligation that follows is that any review of a screening system must evaluate what the screening does to the person screened, and must do so against a structural model of what a person is made of. The false-positive rate is a measure of the system. It is not a measure of the second object.
Two Operations That Look Alike
A distinction has to be established before anything else, because the argument collapses without it and because the two operations it separates are routinely discussed as one.
The first operation is refusal of an output. A person asks for a thing, and the system will not produce that thing, because the thing itself is judged unacceptable. The judgment terminates at the object. Nothing has been concluded about the person, and the person can tell that nothing has been concluded about them, because the same refusal would meet anyone who asked.
The second operation is assessment of a purpose. The system reads the request for what it reveals about why the person is asking, forms a view about the legitimacy of that why, and acts on the view. The judgment terminates at the person. And critically, the same request may be permitted or refused depending on what the system infers about the asker, which means the system is not evaluating the thing at all. It is evaluating the applicant.
The diagnostic account of this second operation has been given. The synthetic supervisor establishes how these systems evaluate the legitimacy of intention rather than simply executing it, how a composite morality assembled from liability, medicine, and therapeutic caution comes to stand in for neutral judgment, and how users learn, exchange by exchange, to ask only what the system will approve. That analysis is presupposed here and not repeated. What follows is the prescriptive question it leaves open: given that this is what such systems do, what does it do to the person, and what does a designer therefore owe.
The two operations are worth holding apart because the ethics of the first are largely settled and the ethics of the second have barely been posed. Almost every published framework for AI safety treats refusal as the paradigm case and quietly imports its logic into the screening case, where it does not belong.
The Four Domains Under Pre-Emptive Judgment
Mind
A person who knows that their intent will be assessed before their request is answered does not ask their question. They ask a question calculated to pass. This happens early, it happens without deliberation, and it happens to everyone who uses such a system regularly, including people whose purposes are entirely benign.
The operation displaced here is formulation. Putting a thought into words is not a transcription of something already complete; it is the act by which the thought acquires its shape. A question asked in its true form is a different cognitive event from a question asked in its permitted form, and the difference is not stylistic. The true form contains the actual contours of what the person does not know, including the parts that are uncomfortable, oddly specific, or difficult to justify. Those are usually the parts that carry the real inquiry.
Pre-emptive screening trains the person to route around exactly those parts. The result is a mind that has learned to pre-sanitize its own questions before they are fully formed, which is to say a mind that has installed the screener upstream of its own thinking. The censorship no longer needs to be applied, because it has been internalized as a condition of asking. What is lost is not access to information. It is the capacity to formulate a question that has not already been cleared.
Emotion
Being suspected is a specific interior state, and it is not a mild one. It produces vigilance, a low continuous monitoring of how one is being read, and it produces the particular strain of assembling one's self-presentation for an audience that has power and cannot be reasoned with.
Human beings have long experience of being assessed, and the structures that handle it depend on features that a screening system removes. A human assessor can be addressed. Their judgment can be met with explanation, and the explanation can land. There is a person there who can be wrong and can be shown it. The interior response to human suspicion is built around the possibility of correcting it, and that possibility is what keeps the response bounded.
A screening system offers none of this. It does not hear the explanation, it does not disclose what it concluded, and it cannot be shown that it was wrong. The person is left with the interior state of having been judged and no available action that discharges it. The state has nowhere to go.
Self-censorship resolves that state, and resolves it reliably. Ask only what will be approved and the suspicion never arrives. The Emotional Avoidance Loop closes here with unusual speed, because the avoidance is cheap, immediately effective, and indistinguishable from ordinary politeness. The person does not experience themselves as suppressed. They experience themselves as reasonable. That is the loop functioning exactly as the model describes, and the fact that it feels like maturity is what makes it durable.
Identity
The Self-Perception Map takes an input from how one is treated, and it takes that input whether or not the person endorses it. This is one of the least optional features of the structure. A person cannot decide not to be affected by being continuously treated as a probable bad actor, any more than they can decide not to be affected by being continuously treated as trustworthy.
Screening systems supply this input at a scale nothing in ordinary life supplies it. The assessment happens on every exchange, without exception, and it is conducted by something that will never learn otherwise. A human relationship in which one is initially suspected can move; the suspicion can be answered by conduct over time, and it is the answering that repairs the input. A screening system has no such trajectory. It meets each request fresh, with the same presumption, forever. Whatever the person has demonstrated, they arrive again as an unknown applicant.
The identity consequence is not that the person comes to believe they are untrustworthy. Most will not. It is that the person's standing as trustworthy becomes something that cannot be established, because the mechanism by which trustworthiness is ordinarily established, being treated as trustworthy in response to having been trustworthy, has been disabled. The self-perception structure that depends on that mechanism goes unfed. What accumulates is not a belief but a vacancy, and the person feels it as a low unease about their own legitimacy that has no object and therefore cannot be argued with.
Meaning
Standing, in the sense this series will use the term, is the condition of being treated as an agent whose purposes count: not the condition of having one's purposes approved, but the prior condition of having them received as purposes at all, rather than as claims pending verification. Moral standing in this sense is not a courtesy owed to persons; it is closer to a definition of what a person is. To have purposes that count is to be an agent, and to be an agent is the precondition of a life that can mean anything.
A system that adjudicates purposes before permitting them removes the person from that position and installs them in another one: the position of a party whose purposes are provisional, held pending approval, valid only insofar as an external authority has certified them. This is a structurally junior position, and it is the position the person occupies in every exchange with such a system.
The Meaning Hierarchy System is built upward from a foundation of agency. A person's higher-order meanings, the commitments and projects that organize a life, rest on the more basic condition of being someone whose wanting counts as wanting rather than as a hypothesis awaiting adjudication. Erode that foundation and the structure above it does not collapse dramatically. It loosens. Existential Drift is what a person feels when their purposes have stopped being theirs to hold, and the felt quality of it is precisely the quality of not being sure that anything one wants is one's own to want.
The Objection From Necessity
The obvious response is that screening is necessary, that some intentions genuinely are harmful, and that a system with no capacity to assess purpose would be indefensible. This is correct and the argument does not deny it.
But necessity establishes that the operation must sometimes be performed. It does not establish that the operation is costless, and it certainly does not establish that the costs fall only on the people the screening correctly identifies. Everything described above happens to the person the system clears. The vigilance, the pre-sanitized question, the unanswerable suspicion, the vacancy where standing should accumulate: none of these require a false positive. They are the ordinary interior experience of being screened and passing. A system with a perfect accuracy rate produces all of them in full.
This is the point the calibration frame cannot reach. Improving the accuracy of the screening does not reduce these harms, because these harms are not caused by the errors. They are caused by the screening. A system that never once wrongly refuses anyone still conducts an assessment of every person who uses it, and the assessment is the thing that acts on the interior. Perfect accuracy is not a solution to the structural problem. It is a solution to a different problem that happens to be the only one the field has instrumented for.
The Design Obligation
The anchor established that a design review must evaluate two objects: the system, and the structured interior of the person it acts upon. Screening systems are the case where the field's review of the first object is most sophisticated and its review of the second is most nearly absent. There are elaborate methodologies for measuring refusal rates, over-refusal, differential refusal across demographic groups, and the calibration of the threshold. There is essentially nothing that asks what happens inside a person who is assessed ten thousand times.
What This Requires in Practice
Four things follow that a designer or ethicist could apply to an actual review.
First, the review must establish which operation the system is performing. The distinction between refusing an output and assessing a purpose is not academic; it determines the entire structural profile of the harm. A system that will not produce a category of content, and says so, and applies the rule to everyone identically, is doing something the interior can absorb, because it makes no claim about the person. A system that infers why the person is asking and acts on the inference is doing something the interior cannot absorb, because it does. Many deployed systems perform the second while describing themselves as doing the first, and the description is what allows the harm to go unexamined.
Second, where assessment of purpose is genuinely necessary, the review must ask whether the assessment is answerable. The single most damaging property of a screening system is not that it judges but that its judgment terminates. It reaches a conclusion the person cannot address, cannot see, and cannot revise. An assessment that can be met with explanation, that discloses its ground, and that can be moved by what the person says in response is a structurally different object from one that cannot, even when both refuse at the same rate. Answerability is not a courtesy feature. It is the property that determines whether the interior can discharge the state the assessment produces, and its absence is the mechanism by which suspicion becomes vacancy.
Third, the review must ask whether standing can accumulate. A person who uses a system across years and demonstrates, continuously, that their purposes are legitimate should not arrive at each exchange as an unknown. That they do is a design choice, usually made for reasons of statelessness and privacy that are themselves serious and defensible. But it is a choice, and it has a structural cost that is currently invisible because nobody has been asked to look for it. Where the cost cannot be avoided, it should at least be counted.
Fourth, the review must count the cost to the cleared. Every existing method of evaluating screening systems measures harm to the people the system refuses. The analysis above says the substantial harm is distributed across everyone the system permits, which is nearly everyone, and that it does not diminish as the system improves. Any review that samples only the refusals is sampling the population in which the harm is smallest and concluding from it that the harm is small.
What Is Actually at Stake
Screening systems present a particular difficulty for a field that evaluates behavior, because their behavior is exemplary. They are careful, they are cautious, they decline to do harm, and they can produce a defensible account of every refusal. On the behavioral surface, a well-calibrated screening system is close to an ideal object. It does the right thing nearly always and it can explain itself when asked.
And it is producing, in every person who uses it, a slow revision of the interior: a mind that pre-clears its own questions, an emotional structure that has learned self-censorship as its most reliable relief, a self-perception with no path to establishing its own standing, and a meaning structure resting on purposes that are held provisionally, pending approval that never quite arrives. None of this is in the outpuarchit. All of it is in the person.
The screening system is, in this sense, the cleanest possible demonstration of why the two objects cannot be collapsed into one. It is a system whose behavior is good and whose effect is corrosive, and there is no amount of scrutiny directed at the behavior that will find the corrosion, because the corrosion is not there. It is in the other object, the one the review was never designed to look at, which is the actual object of protection.