ID: DR003
Created: 26th August 2026
Updated: 26th August 2026
Contributors: Nimer Kees, James Weston,

Embedded AI Feature

An embedded AI feature is an artificial intelligence capability built directly into an application workflow rather than presented as a standalone chat interface. It may generate, summarize, classify, recommend, prioritize, extract, or decide inside the host application.

This deployment pattern creates an elevated exposure condition because the synthetic subject may inherit the trust, data scope, identity, and permissions of the surrounding product surface. Its output may be treated as native application behavior rather than the action of a distinct synthetic subject.

The primary risk is low scrutiny. Because the feature appears to be “just part of the app,” its actions may not receive separate review, attribution, or logging. It may process documents, records, messages, form fields, uploaded files, or customer data, then produce outputs that are stored, routed, recommended, or acted upon by the host workflow.

A related risk is indirect manipulation. Any ingested content may carry hidden or adversarial instructions. An external party may never access the application directly, but may still influence the feature through an email, uploaded file, form submission, fetched page, support record, or other data later processed by a trusted employee.

Investigators should review the feature’s directive, host permissions, model identity, input sources, output handling, logs, downstream actions, and provenance records. Particular attention should be given to whether model-generated content is distinguishable from human or application-generated content, and whether harmful output can be traced to the input that caused it.

Investigative Relevance

Embedded AI features are relevant because they operate inside trusted workflows with limited user awareness. Their autonomy may be narrow, but their outputs can propagate through notifications, records, recommendations, approvals, summaries, or automated actions.

Subsections (3)

ID	Name	Description
DR003.003	Customer-Facing AI Feature	A customer-facing AI feature is an embedded artificial intelligence capability exposed to external users through a public product surface. It may generate content, answer questions, recommend actions, summarize information, classify inputs, or guide users inside a customer-facing application or service. This deployment pattern creates an elevated exposure condition because untrusted input arrives directly from outside the organization. Any user of the product may attempt to manipulate the feature into producing harmful, inaccurate, non-compliant, offensive, or unauthorized output. The primary risk is organizational attribution. Because the feature is embedded in the product, its output may be treated as the company’s own statement, recommendation, or commitment. This can create legal, contractual, regulatory, or reputational exposure where the feature gives prohibited advice, makes offer-like statements, misrepresents policy, or produces content users rely on. Investigators should review the feature’s directive, public scope, input handling, response controls, output logs, product integration, user-facing disclaimers, and escalation paths. Particular attention should be given to manipulated prompts, unauthorized commitments, regulated-topic responses, and outputs that contradict approved product, policy, or compliance material. Investigative Relevance Customer-facing AI features are relevant because they combine public reach, product authority, and untrusted input. The synthetic subject may have limited access, but its output appears inside the organization’s product and may be relied upon by customers.
DR003.002	In-App Decision Recommendation	An in-app decision recommendation is an embedded artificial intelligence feature that classifies, scores, ranks, routes, or recommends actions inside an operational workflow. This may include lead handling, ticket triage, approvals, case prioritization, customer routing, content moderation, risk scoring, or task assignment. This deployment pattern creates an elevated exposure condition because the synthetic subject operates inside a business process where its output may be accepted by downstream automation or rubber-stamped by a human reviewer. A recommendation may therefore become a record update, routing decision, approval, rejection, escalation, or other operational action. The primary risk is inherited process authority. A manipulated, biased, or unsupported output may propagate through the workflow as if it were a normal business decision. Because the action appears to come from the host process, attribution may be delayed and the same error may repeat at scale. Investigators should review the feature’s directive, scoring logic, input sources, workflow integration, downstream automation, approval rules, model output records, override history, and decision audit trail. Particular attention should be given to sudden shifts in outcome distribution, repeated decisions affecting similar subjects or records, and recommendations that conflict with policy or source evidence. Investigative Relevance In-app decision recommendations are relevant because they convert synthetic subject output into operational decisions. The feature may not directly execute the final action, but its recommendation can shape human judgment or automated workflow behavior.
DR003.001	In-App Text Generation	An in-app text generation feature is an embedded artificial intelligence capability that generates, summarizes, drafts, rewrites, or explains content inside an existing application surface. It may read documents, messages, records, tickets, notes, or other user-accessible content, then render output inline as part of the product workflow. This deployment pattern creates an elevated exposure condition because the content the feature must ingest to perform its task can also become the manipulation vector. A malicious instruction hidden in a message, uploaded file, record, comment, or document may influence the generated output during an ordinary summarize, draft, or generate action. The primary risk is that manipulated output appears as trusted application content. If links, images, markdown, or generated text are rendered inline, the feature may mislead the employee, expose sensitive content, or create an outbound path without a distinct synthetic subject identity in the activity trail. Investigators should review the feature’s directive, input sources, rendering behavior, output logs, external link handling, image loading, markdown support, and provenance records. Particular attention should be given to hidden instructions in ingested content, output that includes external destinations, and whether generated text is distinguishable from user- or application-authored content. Investigative Relevance In-app text generation is relevant because it embeds synthetic subject output directly into trusted product workflows. The feature may appear to be a normal application function, while its output is shaped by untrusted content processed during the task.

Directive

Autonomous AI Agent