The term "human in the loop" is used to describe four meaningfully different supervisory relationships between a human operator and an autonomous system. Treating them as interchangeable produces designs that are either operationally useless — because the human is inserted at a point where they cannot effectively intervene — or unsafe — because the assumption of human oversight is not matched by an interface that makes oversight achievable under mission conditions.
This article distinguishes the four supervisory patterns, explains what each requires from the underlying architecture, and identifies where each is appropriate for autonomous ISR platforms. The framing is explicitly engineering-focused: we are not making policy arguments about appropriate levels of autonomy. We are documenting the technical constraints that define what "human oversight" actually means in each supervisory mode.
The OODA loop position defines the supervisory relationship
The OODA framework — Observe, Orient, Decide, Act — provides a useful structural reference for locating where a human is inserted in an autonomous system's decision cycle. Human-in-the-loop in the classical sense means the human participates in the Decide phase before every Action: the system observes and orients autonomously, presents a decision recommendation, and waits for human approval before executing. Human-on-the-loop means the human can observe the system's actions in real time and veto or interrupt them, but does not approve each decision before it is executed. Post-hoc audit means the human reviews actions after the fact, with no real-time intervention capability.
The correct supervisory pattern for a given autonomous ISR system depends on: (1) the action latency budget — how quickly must the system act for the action to be mission-relevant; (2) the communication latency — what is the round-trip time for the human operator to receive a decision request and respond; and (3) the consequence severity of incorrect autonomous action — what is the cost of acting incorrectly without human review.
For a surveillance UAS classifying objects in a wide-area search pattern, the decision latency budget is typically 2–5 seconds (the object will be observable for multiple frames), the communication link may have 500 ms to 2 second latency, and the consequence of a misclassification is an incorrect annotation in the track log, not an irreversible action. This profile supports human-in-the-loop approval for high-confidence actions, with human-on-the-loop fallback for time-critical cues. For a UGV executing autonomous obstacle avoidance at 3 m/s, the action latency budget is under 100 ms — far below any human round-trip time on any realistic communication link. That system must act autonomously in the Observe-Orient-Decide-Act cycle and the human role is supervisory observation, not approval.
Pattern 1: Commit — human approves before each action
The commit pattern requires explicit human approval before the system executes each decision. The system's output is a recommendation with supporting evidence (classification confidence, sensor data, alternative hypotheses), and execution waits for a human "approve" or "reject" signal.
For this pattern to function correctly, the interface must present decision-relevant information in under 2 seconds, the operator must be able to evaluate that information in the context of their situational awareness, and the communication link must support round-trip latency within the action time budget. Any of these failing — a cluttered display that requires 8 seconds to interpret, a communication dropout, an operator who is managing 12 simultaneous tasks — either delays the action past mission relevance or induces the operator to approve blindly to clear the queue.
The commit pattern is appropriate for actions with irreversible or high-consequence outcomes where the autonomous confidence is below a mission-defined threshold. It is not appropriate as a blanket approach to all autonomous decisions, because it degrades the autonomous system to a decision-support tool that is only as fast as the human operator. We are not saying the commit pattern is excessive in high-stakes contexts; we are saying it is the wrong pattern for actions where the decision latency budget is tighter than human-in-the-loop round-trip time.
Pattern 2: Veto — human can interrupt but doesn't pre-approve
The veto pattern inverts the commit pattern: the system executes decisions autonomously, but publishes its intended action with a configurable pre-execution window during which the operator can interrupt. If no veto is received within the window, the action executes. This is the classic human-on-the-loop architecture and it shifts the operator's role from approver to monitor.
The pre-execution window is the critical design parameter. A 500 ms veto window is theoretically adequate for simple binary decisions (proceed vs. abort), but in practice an operator who is monitoring multiple assets simultaneously may not be attending to a specific asset's pre-execution notification within that window. The design must account for the operator's attention allocation under realistic task load, not just the mechanically achievable reaction time in an isolated test.
Interface design for the veto pattern requires that the operator can recognize an impending action, evaluate it, and issue a veto through a low-friction physical interaction (a dedicated abort control or a clearly visible on-screen control that does not require precision pointing) within the veto window. Any interface where the veto action requires menu navigation is a design failure for the veto pattern. The interrupt path must be at the top of the interaction hierarchy with no cognitive or motor overhead between operator intent and interrupt signal.
For a persistent surveillance UGV executing a patrol route autonomously, the veto pattern applied to route deviation decisions — the vehicle generates a proposed course adjustment and the operator has 3 seconds to override — is a practical implementation of human-on-the-loop that preserves operator authority without requiring approval of each waypoint.
Pattern 3: Suggest — human acts on recommendations, system supports
In the suggest pattern, the autonomous system generates recommendations but the human executes all actions manually. The system's output is advisory: a track annotation, a highlighted object of interest, a suggested search area. The human decides what to do with the recommendation and executes via their normal operator interface.
This pattern is common in ISR surveillance systems where the human operator retains full control of the platform and payload, and the autonomous system's role is to reduce the cognitive load of search by triaging the sensor data and surfacing candidates for human attention. The suggest pattern has the advantage that it is unambiguously compliant with any requirement for human control of the system — the system never takes an autonomous action, it only provides information.
The failure mode is alert fatigue. A suggest-pattern system that generates 40 candidate alerts per hour, most of which are false positives, trains the operator to ignore recommendations. A well-calibrated suggest-pattern system operates at high precision (low false alarm rate) at the cost of some recall — it surfaces fewer candidates but the ones it surfaces are worth the operator's attention. Tuning this calibration point for the specific operational environment (cluttered urban vs. open desert) is an operational deployment problem, not just a model accuracy problem.
Pattern 4: Observe — post-hoc audit only
The observe pattern provides no real-time oversight. The system acts fully autonomously within its mission parameters, and human review occurs post-mission via log data and decision audit trails. This is not the same as "no human involvement" — mission parameters and behavioral bounds are defined by humans prior to launch — but there is no operator who can interrupt or redirect during execution.
The observe pattern is appropriate for missions where communication constraints preclude real-time operator interaction: beyond-radio-horizon UAS operations or communications-denied environments. It is not appropriate when mission parameters may change during execution in ways requiring human judgment to resolve. The engineering requirements are substantially higher than for the other three patterns. A system operating under the observe pattern must log its decisions with sufficient fidelity that post-mission audit can reconstruct what the system decided, why, and whether that decision was within the specified mission bounds. The log must be tamper-evident and survive foreseeable platform loss scenarios. Behavioral certification — demonstrating before deployment that the system's autonomous behavior is bounded within acceptable limits — is the hardest qualifying requirement for observe-pattern systems.
Latency budgets and fail behavior
The supervisory pattern choice drives the latency budget for the operator interface. Programs that design the interface first and choose the pattern second often discover incompatibility too late to change it.
For commit pattern: total latency from event detection to operator display must stay below the degraded-decision threshold. Under moderate cognitive load, a human operator can process a simple binary decision in roughly 3–5 seconds; the interface pipeline (edge processing + communications + display rendering) must fit within this envelope. For veto pattern: the effective veto window is the configured pre-execution window minus interface pipeline latency. A system with an 800 ms veto window and 400 ms pipeline latency gives the operator 400 ms of actual decision time — which is often insufficient under task load. Veto window and interface latency must be designed jointly, not independently. For suggest pattern, latency requirements are relaxed because the operator is not acting against a time window; 1–3 seconds is generally acceptable. For observe pattern, there is no real-time interface latency requirement; the requirement shifts to log write atomicity and post-mission replay fidelity.
Every autonomous ISR system must specify its behavior on loss of the supervisory link. The two endpoints are fail-safe (cease autonomous action, execute return-to-base or hold) and fail-operative (continue the mission, because the operational requirement demands it). Most systems appropriately fall between these: continue autonomous operation for a configurable mission-hold period after link loss, then transition to safe behavior if the link is not re-established. The mission-hold duration reflects the operational requirement. A system completing a time-critical ISR pass may tolerate a 120-second hold; a persistent surveillance asset doing loop coverage can transition to hold state immediately on link loss without mission impact. The fail behavior must be explicitly designed, tested under simulated link-loss conditions, and documented in the mission configuration. Systems that treat link loss as an edge case will behave unpredictably in contested communication environments where link degradation is not an exception — it is the expected operational condition.