Loading...
Loading...
Human in the loop AI (HITL) is a design in which a person reviews, corrects, or approves an AI system's output before it takes effect. Human in the loop AI Agents in a security operations center usually pause at two points: before a high-impact action runs, and before an analyst's correction changes what every Agent remembers.
"Chances are somebody uploaded this document here because the first time around we got it wrong." I said that in August, walking a colleague through an investigation where the AI SOC Agent closed an alert by leaning on a classification document someone had uploaded earlier. "So now this is analyst feedback. We're using that forever."
Forever is the whole value of a SOC that learns, because one analyst's fix keeps paying out on every alert of that type without anyone typing it again. A wrong fix lasts just as long, and once it's in context it reads like policy.
Each time an analyst flips a verdict to benign and types why, they write one of these lessons. With Simbian, that comment becomes a request to change what the Agents know, and we never train on your data. Search for human in the loop AI Agents and nearly every top result is about the other half of oversight, approving what an Agent does before it acts.
Human in the loop AI (HITL) puts a person at an AI system's decision points to check or approve its output before it counts. In machine learning, HITL AI usually means people labeling training data, ranking a model's answers before release, or correcting its predictions. With AI Agents, a person signs off on what the Agent does and on what it keeps.
The difference between human in the loop and human on the loop is timing. In a human in the loop design, a person approves the AI's output before it takes effect. In a human on the loop design, the AI acts first, and a person monitors the results and can step in afterward. Human out of the loop means nobody reviews it at all.
| Model | When a person decides | SOC example |
|---|---|---|
| Human in the loop | Before the output takes effect | An analyst approves a host isolation before it runs |
| Human on the loop | After the AI acts, with the power to reverse it | The Agent closes a low-impact alert and an analyst reviews the record later |
| Human out of the loop | Never | Nobody sees the decision unless something breaks |
Simbian runs both models at once. We call that human in control: the action types you keep gated stay human in the loop and wait for approval, and the ones you've released run human on the loop, where your team can review them afterward and step in.
AI Agents should wait for human approval before actions that are disruptive or costly when wrong, such as containment steps. Isolating a production host or disabling an executive's account can cost more than the threat itself when the verdict is wrong. Those wait for a person.
Our AI SOC Agent investigates each alert and proposes the response. Every action type starts gated, and your team decides which ones stop waiting. We investigate and respond to every threat within minutes, on average under 4 minutes from detection to response.
95% of the responses we propose are approved by the customer's own analysts. Their call, not ours. Where they don't approve, we get additional context that makes the next response better.
Gate every low-impact alert too, and your reviewers start approving by habit, which Article 14, the EU AI Act's human oversight article, calls "automation bias," the tendency of automatically relying or over-relying on what a system produces. The Act asks that overseers of high-risk systems stay aware of it and be able to "disregard, override or reverse the output."
An analyst's comment trains an AI SOC Agent when it explains a changed verdict. When an analyst changes a verdict or severity during alert triage and writes why, the change becomes a request to update the Agent's context. That request is the AI Agent feedback loop in a SOC.
Many of those comments correct false positives. In the SANS 2025 Detection and Response Survey, 73% of respondents named false positives as their leading detection challenge, up from 64% in 2024, and 60% said they encounter false positives frequently or very frequently. A USENIX Security 2022 study by Bushra A. Alahmadi, Louise Axon, and Ivan Martinovic found that most alarms practitioners call false positives are "benign triggers—true alarms, explained by legitimate behavior in the organization's environment." The detection fired on something real, and the reason it was fine sits in the analyst's head until a verdict comment writes it down.
The Agent files each lesson itself. A short fact, such as which account is your CI service account, becomes a memory every Agent can recall, and guidance about one alert type becomes an investigation step for that type. Applied context lives in the Simbian Context Lake™, the persistent memory every Simbian Agent shares. "Whatever the SOC learned, the threat hunt is also going to learn," as I put it in the same conversation.
A reviewer with permission to edit context approves an analyst's correction before the AI SOC Agent learns it, whenever the analyst can't write context directly. The request waits in an approval queue until the reviewer approves or rejects it, with an optional note, and no investigation uses it before it's applied.
Requests the Agent raises itself always wait for a person.
We built one queue for every writer, whether the lesson comes from an analyst's verdict comment, a direct submission from your team, or the Agent's own proposal. Each one arrives as the same kind of request, with a diff and a history.
When an auditor asks why an alert was closed, the answer should be a request and the people on it. Every investigation that leans on a piece of context links back to that request, which records who raised it, who approved it if it needed sign-off, and when. Before a request that needs sign-off is applied, the reviewer reads the exact before-and-after diff, the way your engineers review a config change before it reaches production.
Treat queue time as a triage metric. A slow approval queue leaves lessons on the shelf while the same alerts keep arriving. Article 15 of the EU AI Act asks systems that keep learning after deployment to address biased "feedback loops," the risk this review exists to catch. Whether a SOC tool counts as high-risk under the Act is a question for your lawyers, so read both articles as a design reference.
Few vendors say who reviews what their AI learns. Of the nine other AI SOC tools we compared in October 2026, six don't document a review step for what the tool learns, and one reviews lessons only for email and collaboration alerts. One of the remaining two previews and backtests changes, and the other has admins regularly review and prune their strategies, and schedules cleanup of stale memory records. The method and the full table are in which AI SOC solutions reduce dependency on human analysts.
Suppose an analyst marked a host's behavior as expected months ago, and today another analyst says the opposite. We check each request against existing context and hold a contradiction as a conflict. A conflicted request is never applied.
The reviewer gets both versions side by side, with the reason they clash. They can keep the old one, accept the new one everywhere the old one appears, or write a merged version for when each analyst was right in a different situation. If the Agent recommends a side, the reviewer can overrule it.
When the AI SOC Agent proposes its own lesson, the proposal waits for a person's approval and arrives with its evidence, including why it was raised, what share of investigations it would touch, and which past investigations showed the gap. Under self-improving defense, the Agent's proposals never apply on their own.
At a large enterprise customer, each time the EDR classified a signed Admin By Request component as process hollowing, the AI SOC Agent hunted across the whole host before calling the activity legitimate. It proposed a skill for that alert type: validate the signed component, confirm the managed-endpoint workflow, check the child processes for malicious follow-on activity, and stop.
In a before-and-after comparison reported in August 2026, during the feature's preview, verdict quality held on matching alerts in that tenant once a person approved the skill, and the Agent cited the skill in its reasoning on every one.
The result has limits. Every matching alert before the skill was legitimate, so it shows benign cases closing with 42% fewer input tokens on matching alerts in that tenant, and it can't show how the skill handles an attack. The skill also narrows each investigation to the alert's process chain and what the component started, not the whole host. A DLL side-loaded into the signed component could pass the signature check, so only the follow-on check stands between that attack and a benign verdict.
Before approving a correction, check the fact, its scope, its reason, and its end date. The reviewer is deciding what every later investigation of that alert type will assume.
Does the fact name the account, the hosts, and the hour, so the Agent has something to test?
Which hosts, accounts, or ranges does it cover, and did the analyst see all of them?
Does the reason tell the Agent when the exception stops applying, so the same activity at a different hour still gets a full investigation?
Does the request carry an end date? Red-team windows and contractor access both have one.
A correction that fails a check goes back with a note on what's missing, and the analyst raises a tighter one. One that passes covers a whole class of benign triggers on the hosts and accounts it names, and every alert outside it still gets a full investigation.
Human review narrows the risk of a poisoned correction, provided the review sits in front of memory. The OWASP GenAI Security Project lists memory and context poisoning as ASI06 in its Top 10 for agentic applications, and every analyst correction writes into that memory.
Writing for OWASP in May 2026 about memory and context poisoning, Idan Habler, co-lead of the ASI06 entry, put the risk in four words: "The issue is persistence."
Three things limit a bad lesson's reach. Writes stay inside your organization's own context and never change global behavior. A memory can carry an end date, so a stale fact expires. Each piece of context records who raised it and, where it needed sign-off, who approved it, so a bad one can be traced to its request and corrected with a follow-up.
None of that stops a reviewer from approving a plausible bad correction, and once applied, it shapes investigations until someone notices.
Corrections from people with permission to edit context apply without a second reviewer, so keep that permission narrow.
Running human in the loop AI Agents well starts with treating analyst corrections as part of how your SOC triages alerts. Give the approval queue an owner by name. Ask analysts to correct verdicts in the product with a comment, because that raises the request automatically and carries their reason with it. Watch the queue the way you watch alert backlog, since every item in it is a lesson the Agent hasn't learned yet.
Read the applied diffs now and then. Each one shows exactly what the Agent took from a sentence an analyst typed, and if it took too much, a follow-up request tightens it.
Expect a burst of Agent proposals at the start that tapers as gaps close. Open each one's linked investigations before you decide, and reject it with a note if they don't show the gap, because the next round of proposals can read those notes. The Agent doesn't propose allowlists or rules that automate verdicts.
Q: What does HITL mean in AI? HITL stands for human in the loop. HITL AI is a system where a person checks and signs off on the output at set decision points before it takes effect. For AI Agents in security operations, those points are usually high-impact response actions, such as isolating a host, and changes to what the Agent remembers about your environment.
Q: What is the difference between human in the loop and human on the loop? The difference is timing. Human in the loop means a person approves the AI's output before it takes effect. Human on the loop means the AI acts first, and a person monitors the results and can reverse a decision afterward. A common setup keeps high-impact actions in the loop and runs low-impact ones on the loop.
Q: What are human in the loop AI Agents? Human in the loop AI Agents pause for a person's approval at set points, usually before a high-impact action runs and before a change to what they remember is applied. In a SOC, an analyst approves actions such as isolating a host, and a reviewer approves corrections that need sign-off before the Agent learns them.
Q: When should an AI Agent require human approval? An AI Agent should require human approval before actions that are disruptive or costly when wrong, such as isolating a production host, disabling an account, or changing what it remembers about your environment. Low-impact, easily reversed actions can run with a person reviewing afterward, and teams lift approval one action type at a time.
Q: How do you stop reviewers from rubber-stamping AI decisions? Keep approvals for the decisions that matter, so reviewers aren't clicking approve all shift. Show the reviewer the evidence and the exact change. Track how often reviewers reject or edit, because a queue that's always approved unchanged is a sign of automation bias, which the EU AI Act asks overseers to stay aware of.
Q: Why is the human-in-the-loop principle essential when collaborating with AI? An AI system can be wrong without any sign that it is. A person at the two costly points, before an action runs and before a lesson sticks, catches errors before they spread. In a SOC, that person also knows the environment, which no model was trained on, and that knowledge has to come from someone who has seen it.
Before the next verdict comment lands, decide who reads it.
Book a Demo to watch one analyst's correction travel from the verdict screen to the approval queue.
