Operations team reviewing an exception queue of AI agent actions awaiting approval

Human in the Loop AI Agents: Where to Put Approval Points

Move from automation to AI agents without approving everything. Tier actions by risk, reversibility and value, design exception queues, and measure whether human oversight catches real errors.

By Shayan Salehi 8 min read

Image: ColdAI

Put human approval where an action is costly or hard to undo, not on every step. Tier agent actions by risk, reversibility and value, let low-tier actions run with logging, route uncertain cases to an exception queue, and measure whether reviewers catch real errors. Approval that nobody can meaningfully exercise adds delay and false comfort, and regulators now say so.

Why approving everything does not work

The first instinct when a team moves from scripted automation to AI agents is to insert a person before every action. It feels safe. In practice it produces reviewers who click approve forty times an hour on outputs they cannot check.

The EU AI Act names the underlying problem. Article 14(4)(b) says people overseeing a high-risk system must stay aware of the possible tendency of automatically relying or over-relying on its output. That is automation bias, and it grows when reviewers see mostly correct outputs and have no time or information to catch the rare wrong one.

Researcher Ben Green reviewed policies that require human oversight of government algorithms and argued that the evidence shows people are often unable to perform the oversight functions those policies assume. His conclusion is not that people should be removed. It is that oversight must be justified and tested, not assumed. The same applies inside a company.

NIST's AI Risk Management Framework 1.0, published in January 2023, frames the choice as a range of human-AI configurations. A system can decide autonomously, defer to a human expert, or serve as an additional opinion to a human decision maker. NIST also notes that some systems need no human oversight at all. The design question is which configuration fits which action.

What the rules ask of you

Start with what is actually required, because it is narrower than many teams assume and more specific than they expect.

Article 14 of the EU AI Act applies to high-risk AI systems. It requires that oversight measures be commensurate with the risks, level of autonomy and context of use. It also lists what the overseer must be able to do: understand the system's capacities and limitations, interpret its output, decide not to use it, override or reverse its output, and stop it through a stop button or similar procedure. Article 26(2) tells deployers to assign oversight to people with the necessary competence, training and authority.

Timing has moved. Cuatrecasas reports that Regulation (EU) 2026/1744, the Digital Omnibus on AI, was published in the Official Journal on July 24, 2026. It postpones obligations for stand-alone Annex III high-risk systems, such as those used in employment or access to essential services, to December 2, 2027, and for AI embedded in Annex I products to August 2, 2028. Check the current consolidated text of the Act before you rely on any article wording or date.

In the United Kingdom, the Data (Use and Access) Act 2025 inserted Articles 22A to 22C into the UK GDPR. Article 22A defines a decision as solely automated when there is no meaningful human involvement. For significant solely automated decisions, Article 22C requires safeguards: telling the person about the decision, letting them make representations, obtain human intervention and contest it. The Information Commissioner's Office published draft updated guidance in March 2026. Slaughter and May's summary says the draft treats ad hoc spot checks as insufficient and expects the reviewer to be trained to understand the system's logic, outputs and limitations, with involvement before the decision is applied. Treat that as a draft that may change.

Management-system standards complete the picture. ISO/IEC 42001, published in December 2023, specifies requirements for establishing, implementing, maintaining and continually improving an AI management system. It does not set your approval thresholds. It asks that you define them, assign them to named roles, check that they work and improve them. NIST makes the same point in GOVERN 3.2, which calls for policies that define roles and responsibilities for human-AI configurations and oversight, and in MAP 3.5, which asks that oversight processes be defined, assessed and documented.

Tier agent actions by what they can do

A practical design starts from the action, not the agent. List every action an agent can take through its tools: send, post, pay, update a record, delete, grant access, message a customer. Then assign each action to a tier. The tiers below are a design pattern to adapt, not a standard.

  • Tier 0, observe: the agent reads and drafts. No approval, but every output is logged.
  • Tier 1, act and report: the agent takes a reversible, low-value action. No pre-approval. A sample is reviewed after the fact and the action can be undone.
  • Tier 2, propose and approve: the agent prepares the action and a named person approves it before it executes. Used for irreversible actions or those above a value threshold.
  • Tier 3, human decides: the agent supplies evidence and a recommendation, and a person with authority decides. Used where law requires human involvement or where harm to a person is likely.

The tier belongs to the action and its context, not to the model. The same agent can issue a small refund at Tier 1 and a large one at Tier 2.

Separate reversible from irreversible

Reversibility is the most useful single question. An agent that reschedules a delivery slot can be corrected tomorrow. An agent that sends a payment, deletes a customer record, or tells a customer their claim is denied creates facts that are costly to unwind.

Ask three things of each action. Can it be undone, and by whom? How long is the window before it becomes final? What happens downstream before anyone notices? An action that can be reversed within minutes by the same system can run with after-the-fact review. An action that triggers an external message, a payment rail or a regulatory record needs a gate before it executes.

Build the undo path before you grant the permission. If you cannot describe how an action is reversed, treat it as irreversible until you can. Article 14(4)(e) of the Act points the same way, since it expects a way to bring the system to a halt in a safe state.

Set value and confidence thresholds

Value thresholds turn the tiers into rules a team can operate. Choose them from your own loss data and your authority matrix, not from a vendor default. Many organizations already have delegation-of-authority limits for people, such as who may approve a purchase order or a credit memo up to a given amount. Apply the same limits to agents, so an agent never holds more authority than the role it replaces.

Add three more triggers beside value.

  • Novelty: the case does not resemble anything the agent has handled, or a required field is missing.
  • Sensitivity: the case involves a vulnerable person, special category data or a regulated decision.
  • Disagreement: two checks, such as a second model and a rules engine, reach different answers.

Be careful with model confidence scores. They are useful for ranking work in a queue but can be poorly calibrated, so test them against outcomes before using them as a gate. If you cannot show that low-confidence cases are wrong more often, do not route on them.

Design the exception queue as a work product

Most oversight fails in the queue, not in the policy. A reviewer who receives a bare recommendation, no evidence and a ten-second target will approve it. The UK draft guidance and Article 14(4) both point to the same conditions: the reviewer must understand the output and have real authority to change it.

Give each queue item the inputs the agent used, the action it proposes, the reason it was escalated, and the cost of waiting. Make rejection as easy as approval. Record the reviewer's reason in a structured field so that the data can improve the agent.

Set service levels by tier. A queue with no deadline becomes a backlog, and a backlog invites bulk approval. Define who covers when the owner is absent, and what the agent does while it waits: hold, proceed with a safe default, or stop. Define the escalation path when the reviewer disagrees with a colleague or doubts the system itself.

Measure whether oversight works

Counting approvals measures activity, not oversight. NIST's AI RMF 1.0 observes that the degree to which humans are given the means and incentives to challenge AI output requires further study, and suggests that data on how often, and why, humans overrule AI output may be useful. That gives you a starting measure. These are worth tracking.

  • Override rate by tier and reviewer. A rate near zero across thousands of items may mean excellent output or rubber-stamping. A rate that differs sharply between reviewers shows inconsistent judgment.
  • Seeded error detection. Insert known-bad cases into the queue at a low, steady rate and record the share reviewers catch. This is the most direct test of whether oversight works.
  • Time per decision against the time needed. Reviews that take a few seconds on complex items are a warning sign.
  • Escaped errors. Incidents found later by customers, audits or downstream systems on items a person approved, and on items that ran without approval.
  • Reversal and appeal volume. Under the UK rules, people can contest significant automated decisions, so appeals are a direct signal.
  • Queue health. Age of oldest item, share resolved within the service level, and share escalated again.

Review these monthly, and rerun them after any change to the model, the prompt, the tools or the data. A new model version can change error patterns without changing the interface your reviewers see.

Where to start

You can begin with one workflow and a few weeks of effort. Start by inventorying the actions an agent can take, because the tier follows from the action. These steps work in order.

  1. List every tool and permission your agent holds, and describe for each one how it is undone and how long that takes.
  1. Assign each action a tier using reversibility, value and sensitivity. Write the thresholds down and get the owner of the process to sign them.
  1. Name the reviewers for Tier 2 and Tier 3, confirm they have the competence, training and authority Article 26(2) describes, and give them the stop and override controls.
  1. Build the exception queue with evidence, reasons, deadlines and cover arrangements before go-live.
  1. Seed test errors and set baselines for override rate, time per decision and escaped errors in the first month.
  1. Decide in advance what result moves an action up or down a tier, and review that decision quarterly with risk and compliance.

If any of the agent's decisions affect hiring, credit, access to essential services or other significant outcomes for people, ask counsel which of the EU or UK rules apply to you before you set the tiers. The design above gives you the evidence they will ask to see.

Tags:human-in-the-loopai-agentshuman-oversightapproval-workflowsrisk-tieringexception-handlingoperations

Your starting point

What would you like to move forward?

Start with the outcome that feels closest. We'll help give the next step a useful shape.

Let's find your next move

One conversation. A clearer direction.

Tell us what you want to improve. A few sentences are enough to start.

shayan@coldai.org