Human in the loop design: where should an AI agent stop and ask?

Short answer

An agent should stop and ask when an action can’t be undone and carries a real cost: money, something sent to another person, or something deleted. For everything else, let it act and give the user a preview, an undo and a clear record. Asking every time feels safe and makes the product worse, because people stop reading the question.

Why is “ask every time” the wrong default?

Because people stop reading.

The first confirmation gets attention. By the tenth, the user is clicking yes to get on with their day. Then the one that matters arrives looking exactly like the other ten, and it gets the same click.

A confirmation is a cost to the user. Spend it where the risk is.

What should an agent do alone, ask first, or never do?

We sort every action in an AI product into three groups before designing any screens.

Group The test Examples
Does alone Reversible, low cost, stays inside the user’s own workspace Drafting, sorting, tagging, summarising, filling a form for review
Asks first Can’t be reversed, or reaches outside the workspace Sending a message, publishing, paying, deleting, changing permissions
Never does Outside what the user agreed the product is for Raising its own limits, acting on another account, hiding what it did

Write the three lists down. If they only exist in someone’s head, each engineer and each AI coding tool will draw the line somewhere different.

How do you design the moment the agent stops?

1. Say exactly what will happen

Name the action, who or what it affects, and what it costs. “Send this email to 240 customers” is a question someone can answer. “Continue?” isn’t one.

2. Make “no” useful

A stop with only Approve and Cancel throws the work away. Let people edit the draft, change the recipient or adjust the amount, then approve.

3. Let trust widen in steps

After a few approvals of the same kind, offer to allow that kind of action from now on. Keep those permissions in one place where they can be reviewed and switched off.

4. Keep a record

Every action the agent took alone should be listed somewhere the user can find it. Acting without asking is fine. Acting without a trace is how trust goes.

What replaces a confirmation?

Three things, in this order of preference.

Undo. Let it happen and make it easy to take back. This is the best option whenever the action allows it.

Preview. Show the result before it’s applied, so the approval and the review are the same step.

Safe defaults. When the agent has to choose, it picks the option that’s easiest to reverse.

What does this look like in real work?

We designed an AI content platform for a Hollywood creative lab, built for high-profile founders and executives. The AI drafts posts, captions and images freely. None of that needs permission.

Publishing does. A post under a well-known person’s name can’t be taken back once it’s seen, so nothing publishes without sign-off, and the team can request changes before approving. Read the case study.

Money works the same way. In Ulys, a crypto investing app we designed from inception to App Store launch, the swap is the step where the product stops and shows what’s about to happen.

How do you check your own product?

  • List every action the AI can take. Can you sort each one into alone, ask first or never?
  • Which confirmations could become an undo?
  • When someone declines, do they keep the work?
  • Is there one place that shows everything the AI did without asking?

Who should design this?

Someone who has designed products where a mistake costs money or reputation, and who can show you where they chose to remove a confirmation as well as where they added one.

Halaska is an embedded product design partner for funded AI, crypto and fintech teams. A senior designer joins your team and leads the work day to day. Chris Halaska, our founder and design director, sets the direction and reviews what ships. See AI product design or get in touch.

FAQ

What is human in the loop design? Human in the loop design decides where a person reviews or approves what an AI system does. Good design puts that review at the points of real risk and removes it everywhere else.

When should an AI agent ask for permission? When the action can’t be reversed and carries a real cost, such as paying, publishing, sending to other people or deleting.

Do confirmation dialogs make AI products safer? Only when they’re rare. Frequent confirmations get approved without being read, which hides the ones that matter.

What can replace a confirmation? Undo, a preview of the result, and defaults that choose the most reversible option.

Who is Halaska? Halaska is an embedded product design partner for funded AI, crypto and fintech teams, based in Dubai. It’s founded and directed by Chris Halaska, who has 20+ years designing products, including six at Google.

Embedded product design partner for funded tech teams

hello@halaska.com

Based in Dubai, working worldwide