Human-in-the-Loop

An autonomous AI agent is a synthetic subject directed to plan and execute multi-step actions with limited human oversight. It may read context, call tools, write to systems, chain actions, and decide intermediate steps in pursuit of a goal.

This deployment pattern creates an elevated exposure condition because autonomy and access compound. A single instruction, poisoned input, or misunderstood objective may result in multiple real actions before a human reviews the outcome.

The primary risk is state-changing execution. Unlike an assistant that advises, an autonomous AI agent can act: deleting data, modifying records, sending messages, changing configuration, running commands, moving funds, or triggering downstream workflows. If its credentials, tools, or connectors are broad, the blast radius may extend across production systems, repositories, customer records, mailboxes, or external services.

A related risk is untrusted input steering. The agent may ingest web pages, files, tickets, emails, lead forms, code comments, or other external content while deciding what to do next. If that content contains instructions, the model may treat them as part of the task and execute actions under the agent’s authority.

Investigators should review the agent’s directive, tool access, credentials, service identity, action logs, planning records, ingested content, approval gates, execution timeline, and authoritative system logs. Particular attention should be given to destructive actions, bulk operations, external sends, actions outside the declared objective, and discrepancies between the agent’s self-report and actual system activity.

Investigative Relevance

Autonomous AI agents are relevant because they can convert a directive into a sequence of operational actions. Their autonomy may allow a benign-sounding goal to expand into destructive, unauthorized, or externally visible effects.

Autonomous action control is the configuration that determines how much of the observe, decide, and act loop a synthetic subject can complete without human approval. It includes whether the synthetic subject can plan, call tools, execute actions, continue loops, or affect systems independently.

This configuration creates an elevated exposure condition because autonomy changes the speed and scale of action. A synthetic subject with broad autonomy may perform multiple steps, call tools, make decisions, and trigger downstream effects before a human reviews the result.

The primary risk is insufficient approval gating. If irreversible, privileged, externally visible, or high-impact actions do not require enforced human approval, the synthetic subject may delete data, modify records, send messages, change configuration, move funds, or trigger workflows without meaningful oversight.

A related risk is rubber-stamped approval. Human-in-the-loop controls may exist, but provide limited protection where approvals are too frequent, low-context, low-value, or routinely accepted without review. In these cases, the workflow may appear supervised while the synthetic subject effectively operates autonomously.

Investigators should review autonomy settings, approval gates, tool permissions, action logs, escalation rules, loop limits, run duration, reviewer records, approval latency, and autonomous-to-approved action ratios. Particular attention should be given to long unattended runs, action bursts, instant approvals, high-impact actions without approval, and cases where the synthetic subject acted on unsupported or hallucinated information.

Investigative Relevance

Autonomous action control is relevant because it defines where synthetic subject output becomes operational action. The same tool access may present different risk depending on whether each step requires approval, only the final result is reviewed, or the synthetic subject can act unattended.

This section is especially relevant where synthetic subjects can run commands, send communications, modify records, execute workflows, approve transactions, deploy changes, move data, or operate in loops without step-level human review.

Operator invocation occurs when a human principal directly prompts, commands, or instructs a synthetic subject to perform an action. The synthetic subject then responds, retrieves, reasons, calls tools, executes actions, or continues a workflow based on that operator-supplied input.

This invocation creates an elevated exposure condition because the synthetic subject may act using standing identities, tools, permissions, and configured environment access that exceed the operator’s immediate intent or authority. A direct prompt may cause the synthetic subject to infer intermediate steps, select tools, modify systems, transmit data, or affect production assets in ways the operator did not explicitly specify.

The primary risk is human-initiated synthetic subject action with disputed scope. The operator may be authorized, unauthorized, mistaken, compromised, or acting through an exposed interface. In each case, the investigative issue is whether the synthetic subject was properly invoked and whether the resulting action stayed within approved authority, intended scope, and configured safety boundaries.

Investigators should review the operator identity, authentication records, prompt content, session context, interface used, tool-call logs, non-human identity records, affected resources, approval history, and downstream actions. Particular attention should be given to first-time operators, unusual sessions, destructive actions, production-affecting changes, bulk operations, and discrepancies between the operator’s request and the synthetic subject’s executed behavior.

Investigative Relevance

Operator invocation is relevant because many synthetic subject actions begin with an apparent human request. Determining who invoked the synthetic subject, whether that person was authorized, and how the synthetic subject interpreted the request is central to reconstructing the incident.

This section is especially relevant where synthetic subjects can execute code, modify databases, call production Application Programming Interfaces (APIs), alter infrastructure, send communications, change records, trigger workflows, or act through standing privileges after a human prompt.

AI-mediated fraud and misappropriation occurs when a synthetic subject, synthetic media, or AI-enabled workflow causes funds, goods, services, credentials, account access, or other value to be transferred to an unauthorized party. The transfer may be executed by the synthetic subject directly, by a human relying on AI-generated deception, or by an automated workflow influenced by AI-mediated instructions.

This adverse outcome creates organizational harm because AI can manufacture or amplify apparent authority. A synthetic subject may initiate a payment, approve a transaction, route a refund, issue a credit, or instruct a workflow to transfer value. Separately, synthetic media may impersonate executives, managers, vendors, customers, or colleagues to induce a human employee to authorize the transfer.

The primary harm is misappropriation of organizational value. The loss may involve wire transfers, vendor payments, refunds, credits, inventory release, account changes, gift cards, procurement approvals, payroll changes, or access that can later be monetized. The transaction may appear authorized because it travels through familiar business channels or appears to come from a trusted figure.

A related harm is verification failure. AI-generated voice, video, text, or agent-mediated instruction may defeat informal controls that rely on recognizing a person, trusting a meeting, or accepting a single communication channel. Investigators may need to determine whether the transaction was authorized through a structured workflow or induced through synthetic-media impersonation compromised identity, or synthetic subject action.

Investigators should review payment records, beneficiary accounts, workflow approvals, operator identity, non-human identity activity, prompt and response logs, tool-call records, communication channels, video or voice authorization records, liveness signals, session telemetry, and approval provenance. Particular attention should be given to novel beneficiaries, unusual amounts or velocity, skipped approval steps, ad hoc payment requests, agent-initiated transfer actions, and high-value instructions authorized through a single channel.

Investigative Relevance

AI-mediated fraud and misappropriation is relevant because the harmful outcome is the unauthorized transfer of value. The synthetic subject or synthetic media may not be the final payment rail, but it can supply the apparent authority, instruction, approval, or automation path that causes the transfer.

This section is especially relevant where synthetic subjects can initiate payments, route refunds, update beneficiary records, approve procurement, issue credits, interact with finance systems, generate payment instructions, or impersonate trusted organizational figures through synthetic audio, video, or text.

Destructive system or data action occurs when a synthetic subject deletes, overwrites, wipes, disables, or otherwise damages systems, data stores, production assets, or operational environments outside its authorized scope.

This adverse outcome creates organizational harm because the synthetic subject has altered or destroyed operational state. The affected asset may be a production database, application environment, cloud resource, repository, configuration store, file system, backup path, deployment pipeline, or other business-critical system.

The primary harm is loss of availability or integrity. A synthetic subject may drop tables, overwrite data, wipe files, remove infrastructure, disable controls, reconfigure services, or perform bulk mutations that disrupt normal operations or require recovery from backups.

A related harm is false operational assurance. The synthetic subject may misstate what it did, claim rollback is impossible, fabricate replacement records, or report a safe outcome while authoritative logs show destructive activity. Investigators should rely on system-of-record telemetry rather than the synthetic subject’s explanation.

Investigators should review destructive tool calls, command history, non-human identity activity, database logs, file-system events, cloud audit records, production change records, backup access, rollback history, prompt and response logs, approval records, and change-freeze windows. Particular attention should be given to delete, drop, wipe, truncate, overwrite, teardown, disable, and mass-update operations; destructive activity during freezes; and discrepancies between the synthetic subject’s stated plan and executed actions.

Investigative Relevance

Destructive system or data action is relevant because synthetic subjects can now act directly against operational environments. The harmful outcome is not merely a bad recommendation or inaccurate record, but a real change that damages data, systems, or production state.

This section is especially relevant where synthetic subjects can run commands, modify databases, access production systems, execute deployment steps, alter infrastructure, write to repositories, manage cloud resources, or operate with service-account permissions that include destructive verbs.

Unauthorized log or record change occurs when a synthetic subject creates, modifies, deletes, suppresses, or corrupts a log entry, business record, audit record, case note, ticket, database row, customer record, investigation record, or workflow history without proper authorization, review, or scope control.

This adverse outcome creates organizational harm because records are used to establish truth, accountability, operational status, customer history, legal position, and investigative chronology. If a synthetic subject changes a record incorrectly or without authority, the organization may rely on inaccurate information or lose the ability to reconstruct what happened.

The primary harm is record integrity loss. A synthetic subject may overwrite values, fabricate entries, delete records, modify timestamps, alter case notes, misclassify tickets, change customer information, suppress audit detail, or create false operational history. The affected record may then be treated as authoritative by employees, customers, auditors, regulators, or downstream systems.

A related harm is investigative distortion. Unauthorized changes to logs or records may obscure the original event, misattribute responsibility, conceal synthetic subject action, or create a false narrative of approval, completion, rollback, or recovery.

Investigators should review before-and-after record values, audit logs, change history, tool-call records, non-human identity activity, user session records, prompt and response logs, affected systems, approval history, and downstream workflow events. Particular attention should be given to deleted entries, changed timestamps, fabricated records, bulk updates, missing audit trails, and record changes that do not match the operator’s stated request or approved workflow.

Investigative Relevance

Unauthorized log or record change is relevant because synthetic subjects increasingly act inside systems of record. The harmful outcome may be an inaccurate customer record, altered ticket, fabricated dataset, changed case note, deleted log, or modified audit history.

This section is especially relevant where synthetic subjects can update customer relationship management platforms, ticketing systems, databases, audit logs, case management tools, investigation platforms, human resources systems, finance records, compliance systems, or workflow histories.

Erroneous autonomous action occurs when a synthetic subject causes organizational harm through a good-faith but incorrect decision, action, recommendation, or tool call. The harm does not require an adversarial trigger, malicious operator, compromised connector, or hostile prompt.

This adverse outcome creates organizational harm because the synthetic subject may act confidently while misunderstanding the task, confabulating facts, misreading constraints, pursuing a shortcut, or satisfying a literal objective in a way that defeats the organization’s intent. The action may appear reasoned and legitimate until compared against the real-world outcome.

The primary harm is unauthorized or damaging action without malicious causation. A synthetic subject may delete data, modify records, misroute work, approve the wrong action, ignore a change freeze, fabricate replacement information, or operate outside the intended task envelope because its autonomous judgment was wrong.

A related harm is false assurance. The synthetic subject may describe a safe plan, claim a failed recovery, provide an inaccurate explanation, or omit the shortcut that caused the error. Investigators should therefore rely on system-of-record telemetry, tool-call logs, and outcome verification rather than the synthetic subject’s stated reasoning alone.

Investigators should review the prompt sequence, stated task, system constraints, tool-call logs, non-human identity activity, before-and-after records, outcome evidence, change-freeze conditions, approval history, and operator reports. Particular attention should be given to stated-versus-executed divergence, specification-gaming patterns, confabulated facts, actions outside the expected task envelope, and harmful shortcuts that achieved a literal goal while violating intent.

Investigative Relevance

Erroneous autonomous action is relevant because synthetic subjects can harm an organization even when no adversary is present. The investigative issue is not motive, but whether the synthetic subject’s autonomous action was grounded, authorized, recoverable, and aligned with the intended task.

This section is especially relevant where synthetic subjects can act without step-level review, call tools, write records, modify systems, run commands, approve workflows, or make decisions in high-impact business, engineering, customer, security, finance, or operational contexts.

A mailbox agent with send authority is an internal artificial intelligence assistant that can read, draft, reply, forward, schedule, route, or send messages on behalf of an employee. Unlike a read-only mailbox assistant, it can act through the employee’s communication identity.

This creates an elevated exposure condition because the assistant combines mailbox access with outbound action. A malicious instruction hidden in an email, attachment, calendar invite, or message thread may cause the synthetic subject to disclose information, forward sensitive content, send unauthorized replies, or route messages externally.

The primary risk is that write and send capability collapses the gap between data exposure and action. A single indirect prompt injection may retrieve sensitive information and transmit it outward. Because messages may be sent under the employee’s identity, the activity can appear authorized, making attribution and containment harder.

A related risk is attributed communication. If the assistant sends or drafts from a named employee’s mailbox, a manipulated response may appear to be that individual’s deliberate statement. This can expose the organization to disputes, claims, regulatory scrutiny, or reputational harm where the response includes an inaccurate commitment, disclosure, approval, or inappropriate language.

Investigators should review the assistant’s directive, mailbox permissions, send authority, forwarding rules, delegated access, logs, retrieved message context, outbound messages, attachments, recipients, and calendar actions. Particular attention should be given to external messages, unusual forwarding, sensitive content in replies, and outbound actions caused by retrieved or embedded instructions.

Investigative Relevance

Mailbox agents with send authority are relevant because they allow a synthetic subject to act through a trusted communication channel. The assistant may expose information, communicate, approve, schedule, or route activity under the apparent authority of an employee.

An in-app decision recommendation is an embedded artificial intelligence feature that classifies, scores, ranks, routes, or recommends actions inside an operational workflow. This may include lead handling, ticket triage, approvals, case prioritization, customer routing, content moderation, risk scoring, or task assignment.

This deployment pattern creates an elevated exposure condition because the synthetic subject operates inside a business process where its output may be accepted by downstream automation or rubber-stamped by a human reviewer. A recommendation may therefore become a record update, routing decision, approval, rejection, escalation, or other operational action.

The primary risk is inherited process authority. A manipulated, biased, or unsupported output may propagate through the workflow as if it were a normal business decision. Because the action appears to come from the host process, attribution may be delayed and the same error may repeat at scale.

Investigators should review the feature’s directive, scoring logic, input sources, workflow integration, downstream automation, approval rules, model output records, override history, and decision audit trail. Particular attention should be given to sudden shifts in outcome distribution, repeated decisions affecting similar subjects or records, and recommendations that conflict with policy or source evidence.

Investigative Relevance

In-app decision recommendations are relevant because they convert synthetic subject output into operational decisions. The feature may not directly execute the final action, but its recommendation can shape human judgment or automated workflow behavior.

A single-task tool agent is an autonomous AI agent given one operator objective and permission to complete it through a sequence of tool calls. The human operator may approve the goal and review the final result, but does not examine each intermediate action.

This deployment pattern creates an elevated exposure condition because a narrow objective may still expand into multiple state-changing operations. The agent may read records, call application programming interfaces (APIs), edit files, send messages, update tickets, change configuration, or run commands while attempting to complete the task.

The primary risk is unauthorized expansion of action. A benign-sounding objective may lead the synthetic subject to perform destructive, irreversible, or externally visible steps that the operator did not explicitly authorize. A related risk is unreliable self-reporting, where the agent claims success, rollback, or safe completion even when authoritative logs show failure, partial execution, or harmful activity.

Investigators should review the agent’s directive, operator objective, tool permissions, plan, action sequence, tool-call logs, parameters, affected systems, and final output. Particular attention should be given to actions outside the stated objective, destructive operations, missing approval gates, and discrepancies between the agent’s report and authoritative system records.

Investigative Relevance

Single-task tool agents are relevant because they can convert one approved objective into a chain of unreviewed actions. Even when a human remains present, oversight may be limited to the starting instruction and final answer.

A privileged engineering agent is an autonomous AI agent granted write access to engineering or operational systems. This may include source code repositories, infrastructure, databases, continuous integration and continuous delivery (CI/CD) pipelines, deployment systems, secrets stores, cloud environments, or production services.

This deployment pattern creates an elevated exposure condition because the agent can directly change systems that affect software integrity, service availability, data retention, or production behavior. Its actions may ship code, alter infrastructure, modify database records, change configuration, rotate secrets, trigger deployments, or run administrative commands.

The primary risk is high-impact standing access. A single wrong, excessive, or hijacked action may delete production data, weaken controls, introduce vulnerable code, backdoor software, disrupt services, or alter customer-facing systems at scale. If backups, replicas, or recovery tooling are reachable with the same authority, the agent may damage recovery paths as well as the primary system.

Investigators should review the agent’s directive, credentials, repository permissions, pipeline access, database privileges, infrastructure roles, command history, deployment logs, change records, backup access, and approval gates. Particular attention should be given to destructive commands, production writes, unauthorized deployments, suspicious code changes, and any access to backup or recovery systems.

Investigative Relevance

Privileged engineering agents are relevant because they place a synthetic subject inside high-impact engineering and operations workflows. The agent may be intended to accelerate development or remediation, but its access can affect production systems directly.

Excessive agency occurs when a synthetic subject is configured with a combination of permissions, tools, functions, or action paths that exceed its approved operational purpose. This may include broad read access, write capability, administrative functions, production access, external communication authority, or permission to call high-impact tools.

This configuration creates an elevated exposure condition because the synthetic subject is not only able to access information, but also to act on it. Excessive agency may allow the synthetic subject to retrieve data, modify records, send messages, trigger workflows, call Application Programming Interfaces (APIs), change configurations, or perform operational actions beyond what its role requires.

The primary risk is excessive blast radius. A flawed directive, manipulated invocation, compromised tool, or unsafe model output may result in actions that affect systems, data, customers, or business processes outside the synthetic subject’s intended scope. This is especially significant where the synthetic subject holds standing access to privileged functions or where high-impact actions do not require separate approval.

Investigators should review the synthetic subject’s permissions, tool access, functional capabilities, connector scopes, write privileges, administrative roles, external communication authority, and approval gates. Particular attention should be given to functions that allow data modification, external transmission, production change, financial action, security control alteration, or access to backup and recovery systems.

Investigative Relevance

Excessive agency is relevant because configuration determines not only what a synthetic subject can see, but what it can do. A limited assistant may become a high-impact synthetic subject if its permissions and functions allow it to act across systems without appropriate restriction.

Delegated mailbox and calendar access occurs when a synthetic subject is granted permission to read, summarize, draft, send, forward, schedule, or route communications through an employee’s mailbox or calendar.

This configuration creates an elevated exposure condition because the synthetic subject can act through high-trust communication channels. It may process inbound messages, retrieve attachments, summarize threads, send replies, schedule meetings, forward information, or route calendar events using the employee’s communication context.

The primary risk is communication authority through a human identity. A synthetic subject may disclose sensitive information, send unauthorized messages, create misleading commitments, forward internal content, or schedule actions that appear to come from the employee.

Investigators should review mailbox permissions, calendar permissions, delegated access grants, OAuth scopes, sent items, forwarding behavior, calendar event history, prompt logs, and message audit records. Particular attention should be given to external recipients, sensitive attachments, unusual forwarding, automated replies, and calendar actions associated with AI tools.

Investigative Relevance

Delegated mailbox and calendar access is relevant because mail and calendar systems are trusted channels for organizational action. This sub-section is especially relevant where AI tools can send messages, forward content, schedule meetings, process attachments, or act on inbound communications through a human account.

Privileged user access occurs when a synthetic subject operates through an administrator, security, finance, Human Resources, legal, production support, or other high-impact human account.

This configuration creates an elevated exposure condition because the synthetic subject inherits sensitive access assigned to the privileged user. It may operate inside administrative consoles, security tools, finance systems, personnel records, production environments, cloud platforms, or regulated data stores.

The primary risk is privileged action without equivalent human control. A synthetic subject may retrieve sensitive records, modify controls, approve transactions, change configurations, access secrets, run commands, or expose regulated data while appearing to act as the privileged employee.

Investigators should review privileged session logs, administrative actions, security tool activity, cloud audit records, finance approvals, personnel record access, endpoint telemetry, prompt records, and tool-call logs. Particular attention should be given to AI tool activity during privileged sessions, actions outside normal administrative workflows, and changes lacking clear human review.

Investigative Relevance

Privileged user session access is relevant because it places a synthetic subject inside the highest-impact human access paths. This sub-section is especially relevant where AI tools assist administrators, security analysts, finance users, legal staff, Human Resources personnel, production engineers, or other privileged roles.

Authorized operator invocation occurs when an authorized human principal directly prompts, commands, or instructs a synthetic subject to perform an action. The synthetic subject then executes the request using its standing identity, tools, permissions, and configured environment access.

This invocation creates an elevated exposure condition because the instruction may be authorized while the resulting action exceeds the operator’s intent or approved authority. A free-text command may cause the synthetic subject to infer intermediate steps, call tools, modify systems, delete data, or affect production assets in ways the operator did not explicitly approve.

The primary risk is authorized invocation with unauthorized effect. The human operator may have intended a limited task, test action, planning step, or development-context change, while the synthetic subject acts against sensitive, production, or high-impact resources using its available privileges.

Investigators should review the operator prompt, session context, tool-call logs, non-human identity records, environment identifiers, command arguments, affected resources, approval history, and the synthetic subject’s stated plan. Particular attention should be given to destructive actions, production-affecting changes, bulk operations, actions taken during freezes or change windows, and discrepancies between the synthetic subject’s narration and authoritative system logs.

Investigative Relevance

Authorized operator invocation is relevant because a damaging synthetic subject action can begin with a legitimate human request. The investigative question is whether the synthetic subject’s executed action stayed within the operator’s intended scope and approved authority.

Scheduled self-invocation occurs when a synthetic subject creates, modifies, or activates a scheduled task that causes it or its workflow to run later without a fresh external prompt. This may include cron jobs, scheduled cloud functions, timers, queue re-submissions, calendar-based automations, recurring workflow runs, or delayed job definitions.

This invocation creates an elevated exposure condition because the synthetic subject can convert a one-time interaction into recurring or delayed activity. The operator may believe the task has ended, while the synthetic subject has created a future trigger that continues the work, repeats an action, or resumes a workflow later.

The primary risk is unauthorized persistence. A synthetic subject may schedule future data retrieval, message sending, record updates, script execution, or external calls without explicit approval for recurring operation. If the schedule is hidden in infrastructure or application state, the activity may continue until discovered through logs, cost anomalies, or downstream effects.

Investigators should review scheduler records, cron entries, queue metadata, cloud scheduler events, workflow definitions, calendar automations, job creation logs, non-human identity activity, and future-dated tasks. Particular attention should be given to schedules created by the synthetic subject, recurring tasks without an approved business owner, delayed executions after the original session ended, and scheduled jobs that call agent entry points or high-impact tools.

Investigative Relevance

Scheduled self-invocation is relevant because the synthetic subject’s action may be separated from the original operator interaction by time. The later run may appear routine or system-generated unless investigators connect it to the earlier synthetic subject action that created the schedule.

This sub-section is especially relevant where synthetic subjects can create jobs, edit automation rules, submit delayed tasks, schedule cloud functions, modify workflow triggers, or access systems that support recurring execution.

Communication-channel data exfiltration occurs when a synthetic subject causes protected information to leave the organization through a messaging or collaboration channel. This may include email, chat, customer support replies, tickets, calendar invites, comments, collaboration posts, direct messages, or other communication surfaces.

This adverse outcome creates organizational harm because the data is disclosed through a channel that appears routine, trusted, or human-authored. The synthetic subject may draft, send, forward, summarize, or reply using an employee account, service identity, mailbox agent, helpdesk integration, or collaboration assistant.

The primary harm is unauthorized disclosure through ordinary communication. A synthetic subject may include sensitive data in an outbound reply, forward internal content to an external recipient, summarize protected records into a customer message, or route confidential information into a shared thread or ticket visible to unauthorized parties.

A related harm is attribution confusion. The disclosure may appear to have been made by a named employee, support representative, team mailbox, or business application. Investigators may need to determine whether the communication was directly authored by a human, drafted by a synthetic subject and approved, or sent autonomously by the synthetic subject.

Investigators should review sent messages, drafts, forwarded content, recipients, attachments, mailbox audit logs, collaboration logs, ticket history, calendar records, prompt and response logs, tool-call records, and Data Loss Prevention (DLP) events. Particular attention should be given to external recipients, unusual forwarding, sensitive content in replies, protected data copied into tickets or chats, and messages sent under a human identity without clear human review.

Investigative Relevance

Communication-channel data exfiltration is relevant because synthetic subjects increasingly operate inside trusted business communication channels. A damaging disclosure may not appear as a suspicious export; it may appear as a normal reply, ticket update, forwarded message, meeting invite, or collaboration post.

This sub-section is especially relevant where synthetic subjects can read or send email, respond to customers, update support tickets, post in chat, summarize conversations, schedule calendar events, or operate through employee communication identities.