Human in the loop is not enough: the 10-second AI approval test

The comfort of the Allow button

Every serious AI tool now puts a human in the loop. The agent wants to send an email, run a command, or update a record — and a screen appears: Allow?

That screen is where all the safety promises live. The vendor points at it. Your IT policy points at it. You point at it, when someone asks whether the AI could do something wrong: "It can't. I approve everything."

The uncomfortable question is whether that approval is real, or whether it is theatre. Recent evidence says: at normal working speed, it is theatre more often than anyone would like.

What the study actually found

A developer-tools company called Scalex ran an experiment in which people approved or rejected commands proposed by AI agents, across roughly 40,000 simulated runs (study, discussion).

The headline result: reviewers missed about one in three threats. The dangerous action was on screen. The reviewer had the power to stop it. They clicked approve anyway.

The honest caveats first. It is one study, in a gamified setting, and it has not been independently audited. Treat the number as a prompt, not a statistic.

But the direction should surprise nobody who has ever clicked through a cookie banner. Showing a human a confirmation screen is not the same as the human confirming anything.

Why smart people approve bad actions

Approval fatigue is not a character flaw. It is what happens when screens are written for the tool's convenience instead of yours.

Three forces do the damage:

• Repetition. The tenth "Allow?" of the morning gets the same click as the first — and the tenth is the one that sends the wrong draft to the wrong person.
• Framing. Most approval prompts show the technical how (a wall of parameters) and hide the consequential what ("this sends an external email to 40 recipients").
• Default momentum. Approve is the button that makes work continue. Reject is the button that makes work stop. At 4:45pm, that is not a fair fight.

None of this is fixed by telling people to be more careful. It is fixed by changing what a person must be able to say before they click.

A worked example: the tenth approval of the morning

Picture an operations executive using an AI assistant that drafts and sends supplier emails.

1. Nine times this morning, the screen said the same thing: Agent wants to send email. Allow? Nine times, the email was routine. Nine approvals, each faster than the last.
2. The tenth request looks identical — but this draft quotes the confidential rate card, and the recipient field says the wrong supplier.
3. The information needed to catch it was technically on screen, two taps deep, in a preview nobody opens by the tenth time.

The reviewer did not fail. The screen did. It trained them, nine times in a row, that reading was unnecessary.

The 10-Second Approval Test

Here is the practice. Next time an AI tool asks you to approve an action, answer four questions before you click. Out loud if you like. Ten seconds.

1. Action — Can I say what it is about to do in plain words, without reading the prompt again?
2. Blast radius — If this is wrong, who sees it and what does it touch? Me only? A client? Money?
3. Scope — Is it asking for this one action, or for standing permission that outlives the task?
4. Undo — Do I know how to reverse it — and is reversal actually possible once it is out?

If you cannot answer all four in ten seconds, do not approve. Reject, and make the tool restate the action until you can.

The rejection is not a failure state. It is the system working: an action you could not summarise is an action you were about to rubber-stamp.

The screen design that produces the misses

Once you run the test a few times, you start noticing which tools make it easy and which make it impossible.

The pattern to watch for: any approval screen that shows you the HOW — a wall of technical detail, a raw command, a JSON blob — but not the WHAT, the plain-language consequence. That design gap is where the one-in-three lives.

A good approval screen answers the four questions before you ask them: plain action description, named recipients or systems affected, one-time versus standing scope, and the undo path. If a tool you rely on cannot show you that, the ten-second test is your compensating control — and worth raising with the vendor.

Building the habit in a team

One person running the test protects one inbox. A team habit protects the workflow. Three ways to make it stick:

• Weekly autopilot review. Once a week, look at the approvals you granted on autopilot. If you cannot remember approving something, that approval was theatre — discuss it, do not hide it.
• Reject without penalty. Make it explicit that rejecting an AI action to demand a clearer restatement is good work, not slow work.
• Escalate the blanks. When nobody on the team can answer the Undo question for a tool, that is a finding for whoever owns the tool relationship — before an incident, not after.

A human in the loop only counts if the human is actually in the loop. Approval is a skill, not a click — and the ten seconds it costs is the cheapest risk control your team will ever buy.

Frequently asked questions

What does "human in the loop" mean in AI?

It means an AI system is designed so that a human reviews or approves certain actions before they take effect — sending a message, executing a trade instruction, changing a record. It is the most common safety control in workplace AI. Its weakness is that it assumes the review actually happens, rather than becoming a reflexive click.

Does human oversight of AI actually work?

Only when the oversight is designed to be performable. Evidence like the Scalex simulation — reviewers missing roughly one in three threats across 40,000 runs — suggests that a bare confirmation dialog does very little. Oversight works when the screen states the action plainly, shows consequences, limits scope, and offers an undo — or when the reviewer imposes that standard themselves.

What is approval fatigue?

The decay in attention that comes from approving many similar requests in a row. By the tenth identical-looking prompt, most people click from pattern memory, not from reading. It is a design problem: screens that front-load technical detail and hide consequences train users to stop reading.

What are the four questions of the 10-Second Approval Test?

Action (can I state what it will do in plain words?), Blast radius (who or what is affected if it is wrong?), Scope (one action, or standing permission?), and Undo (do I know the reversal path, and does one exist?). All four answerable in ten seconds, or you reject and demand a clearer restatement.

Should employees be allowed to reject AI agent actions?

Yes, and it should be costless. A rejection that forces the tool to restate an action more clearly is the control working as intended. Teams that treat rejection as slowing things down are training their people to rubber-stamp — which converts every future incident into "but a human approved it."

Is one-in-three a reliable statistic?

Treat it as directional. It comes from a single vendor-published, gamified study that has not been independently audited. The reason to act on it anyway is that its direction matches everyday behaviour everyone recognises — banner blindness, cookie-consent clicking, terms-of-service scrolling. The test costs ten seconds; being wrong about the exact percentage costs nothing.