Add human approval to an AI agent.
Human in the loop (HITL) is the standard way to keep an agent from acting on a confident mistake. Here’s the pattern, with working code for LangGraph, the OpenAI Agents SDK, the Vercel AI SDK and MCP.
The pattern
Pick the decisions that need a person
Not every step. Look for judgment calls where a wrong answer costs money or trust and code can't check it: a customer's intent, a refund, a document that should match another.
Send the judgment call, not the whole job
Ask one clear question with the material needed to answer it and two to eight options. Leave out secrets and any personal data the decision doesn't need.
Act only on strong agreement
Plures returns the answer and how strongly people agreed, from 0 to 1. Act automatically when everyone agreed.
Escalate everything else
A split or an expired task goes to your own team. Those are the cases that were genuinely hard, and they're rare.
The examples use Plures, which sends the question to ID verified people who answer independently, each confirmed with Face ID, and returns their consensus. Install with npm install plures or pip install plures, and get a free test key at plures.si.
LangGraph
Add a node that asks verified people, then route on how strongly they agreed: act when it’s unanimous, escalate otherwise.
from typing import Optional, TypedDict
from langgraph.graph import END, START, StateGraph
from plures import Plures
plures = Plures() # reads PLURES_API_KEY
class State(TypedDict):
message: str
order_total: float
answer: Optional[str]
agreement: float
outcome: str
def ask_humans(state: State) -> dict:
task = plures.verify(
"Should this customer get a full refund?",
["Yes", "No"],
context="Order total: $%.2f\nCustomer: %s" % (state["order_total"], state["message"]),
verifiers=3,
timeout=600,
)
result = task.get("result") or {}
return {"answer": result.get("answer"), "agreement": result.get("agreement", 0)}
def route(state: State) -> str:
if state["agreement"] >= 1:
return "refund" if state["answer"] == "Yes" else "decline"
return "escalate"
graph = StateGraph(State)
graph.add_node("ask_humans", ask_humans)
graph.add_node("refund", lambda s: {"outcome": "Refunded $%.2f" % s["order_total"]})
graph.add_node("decline", lambda s: {"outcome": "Declined, with a note to the customer"})
graph.add_node("escalate", lambda s: {"outcome": "Sent to the support team: verifiers didn't fully agree"})
graph.add_edge(START, "ask_humans")
graph.add_conditional_edges("ask_humans", route, ["refund", "decline", "escalate"])
for node in ("refund", "decline", "escalate"):
graph.add_edge(node, END)
app = graph.compile()
if __name__ == "__main__":
out = app.invoke({
"message": "The cake showed up a day late and smashed. Amazing service, really.",
"order_total": 64.0, "answer": None, "agreement": 0.0, "outcome": "",
})
print(out["outcome"])OpenAI Agents SDK
Give the agent a function tool and tell it when to use it. The tool’s reply tells the agent whether the answer is safe to act on.
from typing import List
from agents import Agent, Runner, function_tool
from plures import Plures
plures = Plures() # reads PLURES_API_KEY
@function_tool
def ask_verified_humans(question: str, options: List[str], context: str) -> str:
"""Ask ID verified people to decide a judgment call before acting on it.
Use it when a wrong call costs money or trust: tone and intent, refunds,
whether two things match. Returns their answer and how strongly they agreed.
Args:
question: What people should decide, answerable from the context alone.
options: Two to eight possible answers.
context: The material to judge, as plain text.
"""
task = plures.verify(question, options, context=context, verifiers=3, timeout=300)
r = task.get("result") or {}
if task["status"] != "completed" or r.get("answer") is None:
return "No clear answer from verified people (status %s). Escalate to the team." % task["status"]
return "Verified people answered %s, agreement %.0f%%. %s" % (
r["label"], r["agreement"] * 100,
"Safe to act on." if r["agreement"] >= 1 else "Not unanimous: escalate instead of acting.",
)
agent = Agent(
name="Refunds",
instructions=(
"You handle refund requests. Before approving or declining any refund, "
"use ask_verified_humans to check the customer's situation, and follow their answer."
),
tools=[ask_verified_humans],
)
if __name__ == "__main__":
result = Runner.run_sync(agent, "Customer: my order arrived broken, I want my $64 back. Order delivered 3 days ago.")
print(result.final_output)Vercel AI SDK
The same idea as a tool for generateText or streamText.
import { tool } from "ai";
import { z } from "zod";
import { Plures } from "plures";
const plures = new Plures(); // reads PLURES_API_KEY
export const askHumans = tool({
description:
"Ask ID verified people to decide a judgment call before acting on it, such as a customer's tone, " +
"a refund, or whether two documents match. Returns their answer and how strongly they agreed.",
inputSchema: z.object({
question: z.string().describe("What people should decide, answerable from the context alone"),
options: z.array(z.string()).min(2).max(8),
context: z.string().describe("The material to judge, as plain text"),
}),
execute: async ({ question, options, context }) => {
const task = await plures.verify({ question, options, context, verifiers: 3 }, { timeoutMs: 5 * 60_000 });
const r = task.result;
return {
status: task.status,
answer: r?.label ?? null,
agreement: r?.agreement ?? 0,
guidance: r?.answer != null && r.agreement >= 1 ? "Unanimous: safe to act on." : "Not unanimous: escalate to a person.",
};
},
});
// Use it like any other tool:
// const { text } = await generateText({ model, tools: { askHumans }, stopWhen: stepCountIs(5), prompt });Claude, Cursor and any MCP client
No code at all: add the Plures MCP server and the agent gets an ask_humans tool. In Claude Code: claude mcp add plures --env PLURES_API_KEY=pk_test_... -- npx -y plures-mcp
{
"mcpServers": {
"plures": {
"command": "npx",
"args": ["-y", "plures-mcp"],
"env": { "PLURES_API_KEY": "pk_test_..." }
}
}
}Questions
What is human in the loop (HITL) for AI agents?
A step where a person checks or decides something before an AI agent acts. It's standard practice for actions that cost money or trust, such as refunds, payments, account changes and customer replies, and it's how teams meet human oversight requirements in AI governance.
Which agent decisions need a human check?
Judgment calls where a mistake is costly and a model can be confidently wrong: tone and intent in a message, whether two documents match, whether a refund or exception is fair, and moderation edge cases. Anything you can verify with code or a database lookup doesn't need a person.
Why not just have another model check the first one?
A second model is fast and cheap but tends to share the first model's blind spots, like sarcasm or a total that doesn't add up. An independent human check catches exactly those cases.
How fast is a human answer?
It depends on the task and how many verifiers you ask. Agents usually wait with a webhook rather than blocking, and keep working on other things until the answer arrives. Test keys answer instantly so you can build the flow first.
What does it cost to try?
Nothing. Test keys are free and return a simulated answer at once, without reaching real people.