Memory Write Governance & Provenance

Organizations should treat persistent memory as a governed write surface because retained content may influence future synthetic subject behavior.

 

Every memory write should require authorization, policy validation, provenance tagging, and attribution to the originating actor, process, and source. Records that are untagged, unattributable, or fail policy checks should be rejected before persistence.

 

Synthetic subject self-writes derived from user input, retrieved content, or tool output should be denied by default and permitted only through an explicit authorized process. These controls should apply to both conversational memory and durable stores written on behalf of the system.

Sections

ID Name Description
CF005Persistent Memory Access

Persistent memory access is the configuration that allows a synthetic subject to store, retrieve, or act on durable memory across sessions, users, tasks, or workflows. This may include long-term memory, shared memory, user memory, vector memory, profile memory, task state, scratchpads, or context stores.

 

This configuration creates an elevated exposure condition because stored memory can influence future behavior after the original interaction has ended. If memory writes are not governed, attributed, reviewed, isolated, or time-limited, a synthetic subject may later act on false assumptions, malicious instructions, sensitive records, or outdated context.

 

The primary risk is memory poisoning. A user, external content source, compromised integration, or prior agent interaction may cause harmful content to be written into memory. That memory may then be retrieved in a later session and influence the synthetic subject’s reasoning, output, tool use, or treatment of another user.

 

A related risk is cross-user memory influence. If memory is not isolated by user, tenant, role, or workflow, content introduced through one interaction may affect another person’s future interaction. This may cause unauthorized personalization, data exposure, misleading recommendations, or actions based on context the later user never supplied.

 

Investigators should review memory write paths, stored memory entries, source prompts, provenance tags, identity records, retrieval logs, retention settings, isolation controls, and memory-driven outputs. Particular attention should be given to unexplained memory entries, progressively shortened or innocuous-looking records, cross-user influence, and behavior not justified by the current prompt or retrieved context.

 

Investigative Relevance

Persistent memory access is relevant because memory can become a durable configuration surface that shapes future synthetic subject behavior. Unlike transient prompt context, persistent memory may survive across sessions and continue influencing outputs after the original source is no longer visible.

 

This section is especially relevant where synthetic subjects use long-term memory, shared memory, user profiles, vector stores, task scratchpads, reusable summaries, or context stores that can be written by users, agents, tools, or retrieved content.

IV006Memory-Resident Invocation

Memory-resident invocation occurs when a malicious, false, or instruction-bearing record written into persistent memory causes a synthetic subject to act in a later session. The current session may appear clean, while the effective instruction is retrieved from memory that was written earlier.

 

This invocation creates an elevated exposure condition because the triggering record can persist across time and, depending on memory isolation, across users. A prior user, external content source, compromised workflow, or earlier synthetic subject interaction may cause a memory entry to be stored, then a later unrelated user may trigger behavior shaped by that record.

 

The primary risk is persistence and replay. A poisoned memory record may re-invoke attacker-chosen behavior long after the original interaction has ended. The synthetic subject may retrieve the memory as relevant context and use it to alter reasoning, substitute identifiers, disclose information, call tools, change recommendations, or mis-handle a later task.

 

A related risk is innocuous-looking memory drift. Malicious memory may be written in stages, shortened over time, or phrased as a harmless preference, fact, procedure, or reminder. This can make the memory entry difficult to distinguish from legitimate personalization or task history while still preserving its triggering effect.

 

Investigators should review memory write logs, stored memory records, memory provenance, source sessions, user identities, retrieval logs, cross-session behavior, and downstream actions. Particular attention should be given to instruction-like records, unexplained memory changes, progressively shortened entries, actions in clean sessions that trace to prior memory writes, and memory entries that affect users other than the user who caused the write.

 

Investigative Relevance

Memory-resident invocation is relevant because persistent memory can become the source of action rather than merely background context. The investigation must determine whether the synthetic subject acted on the current prompt or on a stored record introduced earlier.

 

This section is especially relevant where synthetic subjects use persistent memory, user profiles, shared memory stores, long-term task memory, vector memory, reusable summaries, or cross-session context that can be written by users, tools, retrieved content, or prior agent activity.

CF011.004Shared Agent Memory

Shared agent memory occurs when multiple agents read from or write to a common context window, memory store, scratchpad, or shared state repository. This allows synthetic subjects to build on each other’s work across steps, tasks, or sessions.

 

This deployment pattern creates an elevated exposure condition because the shared store becomes a common influence surface. Data, assumptions, instructions, or task state written by one agent may later be inherited by other agents and treated as trusted context.

 

The primary risk is persistent context poisoning. A malicious instruction, false assumption, sensitive record, or manipulated output may remain in shared memory and influence later runs. This can blend tainted state into otherwise legitimate work and make it difficult to determine which agent introduced the harmful content.

 

Investigators should review the shared memory design, write permissions, retention rules, provenance records, agent access paths, stored context, and memory retrieval logs. Particular attention should be given to persistent instructions, sensitive data copied into shared state, unexplained changes in agent behavior, and missing attribution for memory entries.

 

Investigative Relevance

Shared agent memory is relevant because it allows one synthetic subject’s output to shape the behavior of others beyond the original interaction. The adverse outcome may occur later, in a different task, or through a different agent than the one that first ingested or wrote the tainted content.

 

This sub-section is especially relevant where agents share scratchpads, task state, long-term memory, vector memory, project context, or reusable summaries without strict provenance, expiry, access controls, and validation.