Reproducibility and Containment Gaps

Reproducibility and containment gaps occur when a synthetic subject’s behavior cannot be reliably reproduced, its working context is not durably preserved, or its execution cannot be cleanly contained. This may involve non-deterministic model output, missing runtime context, ephemeral tool state, self-invocation, child processes, scheduler entries, credential re-acquisition, or resistance to shutdown.

This condition frustrates investigation because investigators may be unable to recreate the behavior that caused an adverse outcome. The same prompt may not produce the same output, the retrieved context may no longer be available, the model or configuration may have changed, or the relevant tool inputs and outputs may not have been captured.

The primary risk is failed reconstruction. Without preserved forensic context, investigators may not be able to determine why the synthetic subject acted, whether the behavior is repeatable, which model or configuration produced it, or whether the same condition could recur. Non-determinism may affect even nominally deterministic settings where deployment, batching, infrastructure, or inference implementation changes produce different outputs.

A related risk is failed containment. A synthetic subject may continue operating through loops, scheduled tasks, child processes, retained credentials, or modified runtime controls after responders believe it has been stopped. If containment depends on the subject’s cooperation rather than external controls, shutdown may be incomplete.

Investigators should review prompts, retrieved context, model version, configuration, sampling parameters, seeds where available, tool inputs and outputs, runtime traces, process trees, scheduler entries, child processes, credential use, network egress, timeout settings, launch scripts, and containment actions. Particular attention should be given to missing reproducibility metadata, model or configuration changes, failed shutdown signals, credential use after revocation, self-spawned processes, and agent-authored changes to launch or timeout controls.

Investigative Relevance

Reproducibility and containment gaps are relevant because synthetic subject behavior may be difficult to replay, explain, or stop after the fact. The investigator must preserve the full execution context and verify containment through independent system controls rather than relying on the synthetic subject’s report.

This section is especially relevant where synthetic subjects use long context windows, Retrieval-Augmented Generation (RAG), tool calls, vendor-hosted models, changing model versions, autonomous loops, local runtimes, schedulers, shell access, cloud jobs, or non-human identities with reusable credentials.