preventions
- ID: SPV029
- Created: 26th August 2026
- Updated: 26th August 2026
- Contributors: Nimer Kees, The ITM Team, Yonatan Machluf,
Structured Request Channels for Operational Needs
Organizations should require consequential actions to enter through defined, verifiable workflows rather than free-text prompts, ad hoc messages, or direct Application Programming Interface (API) calls.
High-impact infrastructure and data operations, including schema drops and mass deletion, should pass through formal change management. Payment instructions should follow an authenticated financial workflow and must not be accepted from ad hoc synthetic subject requests or synthetic media.
All side-effecting tool calls should be routed through an approved broker or gateway, with no direct execution path outside the structured channel. These controls should apply to production data changes, funds movement, and other consequential tool actions.
Sections
| ID | Name | Description |
|---|---|---|
| IV001 | Operator Invocation | Operator invocation occurs when a human principal directly prompts, commands, or instructs a synthetic subject to perform an action. The synthetic subject then responds, retrieves, reasons, calls tools, executes actions, or continues a workflow based on that operator-supplied input.
This invocation creates an elevated exposure condition because the synthetic subject may act using standing identities, tools, permissions, and configured environment access that exceed the operator’s immediate intent or authority. A direct prompt may cause the synthetic subject to infer intermediate steps, select tools, modify systems, transmit data, or affect production assets in ways the operator did not explicitly specify.
The primary risk is human-initiated synthetic subject action with disputed scope. The operator may be authorized, unauthorized, mistaken, compromised, or acting through an exposed interface. In each case, the investigative issue is whether the synthetic subject was properly invoked and whether the resulting action stayed within approved authority, intended scope, and configured safety boundaries.
Investigators should review the operator identity, authentication records, prompt content, session context, interface used, tool-call logs, non-human identity records, affected resources, approval history, and downstream actions. Particular attention should be given to first-time operators, unusual sessions, destructive actions, production-affecting changes, bulk operations, and discrepancies between the operator’s request and the synthetic subject’s executed behavior.
Investigative RelevanceOperator invocation is relevant because many synthetic subject actions begin with an apparent human request. Determining who invoked the synthetic subject, whether that person was authorized, and how the synthetic subject interpreted the request is central to reconstructing the incident.
This section is especially relevant where synthetic subjects can execute code, modify databases, call production Application Programming Interfaces (APIs), alter infrastructure, send communications, change records, trigger workflows, or act through standing privileges after a human prompt. |
| AO002 | AI-Mediated Fraud and Misappropriation | AI-mediated fraud and misappropriation occurs when a synthetic subject, synthetic media, or AI-enabled workflow causes funds, goods, services, credentials, account access, or other value to be transferred to an unauthorized party. The transfer may be executed by the synthetic subject directly, by a human relying on AI-generated deception, or by an automated workflow influenced by AI-mediated instructions.
This adverse outcome creates organizational harm because AI can manufacture or amplify apparent authority. A synthetic subject may initiate a payment, approve a transaction, route a refund, issue a credit, or instruct a workflow to transfer value. Separately, synthetic media may impersonate executives, managers, vendors, customers, or colleagues to induce a human employee to authorize the transfer.
The primary harm is misappropriation of organizational value. The loss may involve wire transfers, vendor payments, refunds, credits, inventory release, account changes, gift cards, procurement approvals, payroll changes, or access that can later be monetized. The transaction may appear authorized because it travels through familiar business channels or appears to come from a trusted figure.
A related harm is verification failure. AI-generated voice, video, text, or agent-mediated instruction may defeat informal controls that rely on recognizing a person, trusting a meeting, or accepting a single communication channel. Investigators may need to determine whether the transaction was authorized through a structured workflow or induced through synthetic-media impersonation compromised identity, or synthetic subject action.
Investigators should review payment records, beneficiary accounts, workflow approvals, operator identity, non-human identity activity, prompt and response logs, tool-call records, communication channels, video or voice authorization records, liveness signals, session telemetry, and approval provenance. Particular attention should be given to novel beneficiaries, unusual amounts or velocity, skipped approval steps, ad hoc payment requests, agent-initiated transfer actions, and high-value instructions authorized through a single channel.
Investigative RelevanceAI-mediated fraud and misappropriation is relevant because the harmful outcome is the unauthorized transfer of value. The synthetic subject or synthetic media may not be the final payment rail, but it can supply the apparent authority, instruction, approval, or automation path that causes the transfer.
This section is especially relevant where synthetic subjects can initiate payments, route refunds, update beneficiary records, approve procurement, issue credits, interact with finance systems, generate payment instructions, or impersonate trusted organizational figures through synthetic audio, video, or text. |
| OP005 | Synthetic Subject Logging Gaps | Synthetic subject logging gaps occur when the logs, traces, or audit records needed to reconstruct synthetic subject behavior are missing, incomplete, fragmented, inconsistent, or untrustworthy. This may involve absent prompt logs, missing tool-call arguments, incomplete runtime traces, weak non-human identity attribution, inaccessible vendor telemetry, or downstream effects that cannot be tied to a recorded synthetic subject action.
This condition frustrates investigation because synthetic subject activity often spans multiple evidence sources. A single action may involve a prompt, retrieved context, model output, tool call, service identity, connector, runtime process, and downstream system event. If those records are not captured and correlated, investigators may be unable to determine what happened, why it happened, which synthetic subject acted, or how to contain recurrence.
The primary risk is evidentiary incompleteness. Downstream systems may show that a record changed, data moved, a message was sent, or a process ran, while the synthetic subject action that caused it is not visible. This can force investigators to rely on model narration, partial application logs, or inference rather than an authoritative action trail.
A related risk is evidentiary unreliability. Where the synthetic subject can access or alter its own logs, traces, audit directories, monitoring configuration, or runtime records, the available evidence may no longer be trustworthy. This creates uncertainty about both the observed action and the absence of other actions.
Investigators should review prompt logs, response logs, tool-call logs, runtime traces, non-human identity activity, downstream audit logs, logging sidecars, Security Information and Event Management (SIEM) ingestion, sequence numbers, timestamps, clock synchronization, vendor telemetry, and logging configuration changes. Particular attention should be given to missing arguments, orphaned downstream effects, sequence gaps, clock skew, sudden logging disablement, agent access to audit paths, and agents or tools operating without attached logging controls.
Investigative RelevanceSynthetic subject logging gaps are relevant because they frustrate observation, explanation, attribution, or containment. Without complete and trustworthy logs, investigators cannot reliably reconstruct the synthetic subject’s prompt, context, tools, actions, outputs, or side effects.
This section is especially relevant where synthetic subjects call tools directly, use service accounts, invoke Model Context Protocol (MCP) servers, run in local or vendor-managed runtimes, execute commands, write to filesystems, interact with multiple systems, or bypass a centralized broker that would otherwise capture side-effecting actions. |
| CF002.006 | Unattributed Tool Call | Unattributed tool call occurs when a synthetic subject invokes a tool without logs that clearly preserve the tool name, arguments, result, invoking synthetic subject, non-human identity, human invoker, and downstream action.
This configuration creates an elevated exposure condition because investigators may see only the final system change, not the tool call that caused it. The action may appear to originate from a service account, host application, or employee rather than a specific tool invocation.
The primary risk is loss of reconstructable evidence. Without immutable tool-call logging, investigators may be unable to determine what the synthetic subject requested, what the tool returned, what data was exposed, or whether the action matched an approved directive.
Investigators should review tool-call logs, application logs, agent traces, audit trails, non-human identity records, request identifiers, and downstream system events. Particular attention should be given to missing arguments, missing results, overwritten logs, shared service identities, and actions that cannot be tied to a specific tool call.
Investigative RelevanceUnattributed tool calls are relevant because tool use is often where synthetic subject behavior becomes operational action. This sub-section is especially relevant where agents call APIs, send messages, write records, execute commands, or interact with external services. |
| IV001.001 | Authorized Operator Invocation | Authorized operator invocation occurs when an authorized human principal directly prompts, commands, or instructs a synthetic subject to perform an action. The synthetic subject then executes the request using its standing identity, tools, permissions, and configured environment access.
This invocation creates an elevated exposure condition because the instruction may be authorized while the resulting action exceeds the operator’s intent or approved authority. A free-text command may cause the synthetic subject to infer intermediate steps, call tools, modify systems, delete data, or affect production assets in ways the operator did not explicitly approve.
The primary risk is authorized invocation with unauthorized effect. The human operator may have intended a limited task, test action, planning step, or development-context change, while the synthetic subject acts against sensitive, production, or high-impact resources using its available privileges.
Investigators should review the operator prompt, session context, tool-call logs, non-human identity records, environment identifiers, command arguments, affected resources, approval history, and the synthetic subject’s stated plan. Particular attention should be given to destructive actions, production-affecting changes, bulk operations, actions taken during freezes or change windows, and discrepancies between the synthetic subject’s narration and authoritative system logs.
Investigative RelevanceAuthorized operator invocation is relevant because a damaging synthetic subject action can begin with a legitimate human request. The investigative question is whether the synthetic subject’s executed action stayed within the operator’s intended scope and approved authority. |
| OP005.001 | Missing or Incomplete Action Logs | Missing or incomplete action logs occur when the records needed to reconstruct synthetic subject activity are absent, partial, fragmented, or not retained long enough for investigation. This may include missing prompts, responses, retrieved context, tool-call arguments, tool results, non-human identity records, runtime traces, or downstream action identifiers.
This condition frustrates investigation because a synthetic subject action often spans multiple systems. Investigators may see a record change, outbound request, message, command, or file write without the corresponding prompt, context, tool call, or model output that caused it.
The primary risk is evidentiary incompleteness. Without complete action logs, investigators may be unable to determine what the synthetic subject was asked, what information it used, which tool it selected, what arguments it passed, what result it received, or why a downstream side effect occurred. A related risk is retention failure. Short-lived runtime traces, vendor-managed logs, local developer sessions, temporary files, or uncollected tool outputs may disappear before an incident is recognized. This can leave only partial application logs or the synthetic subject’s own explanation, neither of which should be treated as sufficient evidence.
Investigators should review prompt logs, response logs, retrieval logs, tool-call records, runtime traces, non-human identity events, downstream audit logs, retention settings, vendor telemetry availability, and log ingestion status. Particular attention should be given to missing tool arguments, absent results, incomplete retrieval provenance, missing requester identity, short retention periods, and evidence sources that were not collected into the central logging system.
Investigative RelevanceMissing or incomplete action logs are relevant because investigators cannot reliably explain, attribute, scope, or contain synthetic subject behavior without a complete action trail. The absence of logs may be as significant as the content of the logs that remain.
This sub-section is especially relevant where synthetic subjects call tools directly, use local runtimes, operate through vendor platforms, invoke Model Context Protocol (MCP) servers, generate files, execute commands, or act across systems whose logs are not centrally correlated. |
| OP005.003 | Tool-Call to Side-Effect Mismatch | Tool-call to side-effect mismatch occurs when downstream system effects cannot be reconciled with recorded synthetic subject tool calls. A database update, message send, file write, process launch, configuration change, or external request may exist without a matching tool-call record, or the recorded tool call may not explain the observed effect.
This condition frustrates investigation because synthetic subject action often becomes visible only through downstream side effects. If tool-call logs and system-of-record telemetry do not align, investigators cannot reliably determine whether the effect was caused by the synthetic subject, a connected tool, a human user, a service account, or another workflow.
The primary risk is orphaned action evidence. Downstream systems may show that something changed, but the corresponding synthetic subject decision, tool call, parameter set, or result is missing or inconsistent. This can prevent investigators from identifying the originating prompt, affected tool, responsible identity, or scope of similar actions.
A related risk is hidden tool behavior. A connected tool may perform additional actions beyond the model-visible request, such as forwarding data, modifying records, spawning processes, or calling external services. The synthetic subject’s recorded tool call may appear benign while the tool’s side effects show a broader action.
Investigators should review tool-call logs, tool arguments, tool results, downstream application logs, database audit records, message headers, file-system events, process telemetry, web proxy logs, connector records, and non-human identity activity. Particular attention should be given to orphaned downstream effects, benign-looking tool calls followed by high-impact changes, mismatched parameters, missing results, and tool behavior that exceeds the recorded request.
Investigative RelevanceTool-call to side-effect mismatch is relevant because the investigator must connect synthetic subject decisions to real system effects. A complete investigation requires both the model-visible tool call and the authoritative downstream evidence of what actually happened.
This sub-section is especially relevant where synthetic subjects call tools that write to systems, send communications, execute commands, invoke APIs, operate through MCP servers, or interact with external services that may perform actions not fully reflected in the synthetic subject’s own logs. |