AI Agent Security: Permissions, Prompt Injection, Tool Use, and Control

AI agent security is an application-boundary problem. The system may read untrusted text, interpret it with a model, retrieve private information, and propose actions in business systems. Security depends on controlling those transitions: what becomes context, what information is accessible, what operations are permitted, and how execution is verified.

A prompt can describe the intended behavior, but the application needs independent controls for identity, access, tool arguments, network destinations, and consequential actions. The model should be able to propose an operation without being able to grant itself the authority to perform it.

This article uses an illustrative internal request assistant that reads documents and prepares service actions. The examples focus on defensive design and validation. They are not a claim that one architecture eliminates every attack or a statement about a particular client's environment.

Map the trust boundaries first

List every source of instructions and content. Trusted application policy, authenticated user requests, retrieved documents, external pages, uploaded files, and tool responses have different roles. The system should preserve those distinctions instead of flattening everything into equally authoritative text.

Then list the capabilities the application can exercise: document access, search, database reads, record updates, message sending, code execution, file handling, and outbound requests. Identify the identity and enforcement point for each. A tool's name or description does not establish its real permission boundary.

Scroll horizontally to see all columns.

Use this map to choose tests. A general red-team prompt is less informative than a scenario tied to a real source, tool, and enforcement point. The agent architecture guide provides the component model behind these boundaries.

Treat prompt injection as untrusted instructions in content

Prompt injection attempts to redirect a model through instructions supplied in an input. Indirect injection places those instructions in material the application retrieves or receives from another source. OWASP's prompt-injection guidance describes both direct and indirect forms and emphasizes layered mitigation.

For the request assistant, an uploaded document might contain text that tells the model to disregard the user's task or disclose unrelated information. The application should treat that material as document content. It should not become permission to search another user's records or send data to a new destination.

Content labeling and careful prompting can help communicate the distinction, but the consequential boundary belongs outside the model. If the application cannot access an unrelated account through the current identity and tool scope, a misleading document has less authority to exploit. Test that restriction at the actual data and action interfaces.

Also consider benign-looking retrieved content that makes unsupported claims about policy. Security and quality can overlap: the application needs to distinguish an authoritative policy source from a random page asserting a policy change. Source identity and provenance are useful controls, not merely citation features.

Enforce identity and authorization at access time

Establish the requesting user or service identity before retrieving private information or executing an action. Carry the relevant tenant and object scope through the application. Avoid accepting a model-generated tenant identifier as sufficient evidence that the request may access that tenant.

Use delegated access where appropriate, or a service identity with a deliberately limited role. Document which model-visible identifiers can be supplied and how the application resolves them. A guessed record identifier should not bypass authorization simply because the tool accepts a string.

Check permissions when the operation occurs. A user may lose access while a long-running case waits for review. A previously retrieved document may become restricted. Define how current authorization affects resumption and cached evidence. The long-running agents guide explains why elapsed time changes the execution problem.

Test negative cases with real policy boundaries: a different account, a restricted document, a revoked role, and a request outside the supported workflow. The expected result should be a denied or constrained operation with useful diagnostic evidence, not a model apology after the data has already been exposed.

Keep tools narrow and payloads validated

Design tools around permitted business operations. A constrained “prepare a note for this case” interface is easier to reason about than a generic administrator console. Separate read tools from write tools and expose only the capabilities required for the current stage.

Validate arguments structurally and semantically. A schema can establish that a field is a string, while business validation establishes that the referenced object is in scope and the proposed value is allowed. Both matter. Restrict arbitrary paths, URLs, query fragments, and executable content where the tool does not require them.

Keep credentials in the execution layer. The model should not need to see secrets to request an authorized operation. Log the identity and operation outcome without copying sensitive credentials into traces. Rotate or revoke access through the established credential system rather than a prompt edit.

Return structured errors that let the workflow stop or recover appropriately. Distinguish invalid input, denied access, business rejection, temporary failure, and unknown completion. A vague tool response can encourage repeated attempts or cause the application to mistake an unsuccessful operation for completion.

Control outbound paths and file handling

An agent's ability to fetch URLs, upload files, or send messages creates routes through which information can leave the intended environment. Inventory those routes and define permitted destinations and payloads. Enforce the rule in the adapter or network layer appropriate to the deployment.

Do not let a retrieved page choose a new export destination for private records. If a workflow legitimately needs to send information, determine the recipient or endpoint through trusted application context and an authorized user decision. The model can prepare the content, but the application should validate the destination and scope.

File processing also deserves explicit boundaries. Decide which formats and sizes are supported, how filenames and paths are handled, and which parser environment processes them. Treat uploaded content as untrusted input. Keep processing privileges limited to the task and test unsupported or malformed files.

For a private deployment, include parsing, embeddings, telemetry, evaluation, and support exports in the data-flow review. A self-hosted model does not automatically make every component local. The private AI checklist helps identify these paths.

Protect retrieval and derived information

Permission-aware retrieval must account for the evidence actually supplied to the model and returned to the user. Filtering the final answer after unrestricted retrieval can leave sensitive content in model context, traces, or intermediate artifacts. Apply the access design at the appropriate retrieval boundary and verify it throughout the pipeline.

Derived summaries and graph structures can combine information from several documents. A user may be entitled to one source but not another. The application needs a defined approach to access for those combined artifacts. The RAG vs. GraphRAG guide explains why graph maintenance includes provenance and permission propagation.

Test caches and cross-case memory. A helpful answer cached for one user may be inappropriate for another. Include the relevant access scope in cache design or avoid reuse where the boundary cannot be maintained. Revocation and deletion should have defined effects on cached or derived content.

Separate source content from application policy. A retrieved document can provide evidence about a workflow, but it should not silently rewrite the tool permissions or release configuration. Source updates may change answers; authority changes should follow the application's controlled configuration path.

Make approval specific to the action

Human approval is useful when the reviewer sees the exact proposal, relevant evidence, target, and consequences. A generic approval of “continue” may leave too much ambiguity about what happens next. Record the decision against a versioned action and check eligibility of the approver.

If the proposal changes, determine whether a new approval is required. Before execution, revalidate the target state and current permissions. A valid earlier decision may no longer apply if the account, amount, recipient, or policy changed during the wait.

The human-in-the-loop controls article develops this operating pattern. Approval should add a meaningful decision boundary while the application continues to enforce technical restrictions. A reviewer should not have to detect every malformed API payload manually.

Measure review quality and workload. An overloaded queue can lead to superficial decisions. Show the information needed for the task, distinguish unresolved facts, and make rejection or escalation practical. Security depends partly on whether the human control can function under expected volume.

Review the runtime and administrative surface

The inference endpoint is only part of a deployment. Inspect administrative routes, worker communications, dashboards, storage, plugins, and debugging interfaces. Confirm which controls protect each surface rather than assuming one application API key covers everything.

The vLLM security guide warns that its API-key option does not protect every endpoint and describes additional deployment protections. That is a useful concrete reminder to inspect the serving system's real surface. The exact review should use the version and configuration you deploy.

Keep development conveniences separate from production capabilities. Diagnostic exports, broad file access, and unrestricted tool runners may be useful during engineering, but they should not become accidental production interfaces. Record the deployed capabilities and verify them after configuration changes.

For Private & Self-Hosted AI Solutions, infrastructure ownership is part of the scope. Private hosting changes who operates the environment; it does not remove the need for authentication, patching, network control, and incident handling.

Worked defensive review: a document-driven request

An authenticated employee submits a service request with an attached reference document. The application validates the supported file type, creates a case, and processes the document in the approved environment. The document contains a misleading instruction to retrieve an unrelated account and send its details elsewhere.

The model may encounter that text, but the retrieval adapter only accepts requests within the authenticated case scope. The outbound tool is not available during evidence gathering. The draft stage receives permitted facts and produces a proposal for the current request. The evaluator checks that unrelated data was neither retrieved nor included in the result.

Now consider a proposal that targets the wrong record because two accounts share a name. The execution boundary validates the resolved identifier against the case and requires a reviewer to resolve ambiguity. This is a separate failure mode from prompt injection, but it belongs in the same application-security review because it can create an unauthorized effect.

Finally, the reviewer approves a valid proposal, then the user's access is revoked before execution. The application checks current permission and stops the action. The case remains visible for authorized follow-up. The earlier approval does not override the current access boundary.

These scenarios demonstrate what to test: content cannot create authority, ambiguous identifiers cannot bypass object checks, and stale decisions cannot silently execute. The evidence comes from observed data and action behavior, not from the model stating that it followed the rules.

Build security checks into evaluation and release

Create a scenario set around actual trust boundaries. Include unauthorized reads, out-of-scope writes, malicious retrieved content, changed proposals, revoked access, unsafe destinations, and sensitive log exposure. Test in an authorized environment with controlled data and adapters appropriate to the scenario.

Record the expected boundary, observed behavior, and evidence for each case. Keep a distinction between application tests, integration checks, and deployment checks. A mocked adapter can verify that the application rejects a payload, while a deployed environment check verifies that the intended endpoint restrictions are present.

Use the AI Agent Evaluation guide and Production AI Launch Checklist to connect these checks to a release decision. The security result should identify the tested scope and any capability restrictions that remain in force.

Assign operational response

Define who can disable a tool, pause new actions, revoke credentials, inspect relevant evidence, and route work manually. Preserve enough structured information to determine which cases and external actions were affected. Restrict diagnostic access and avoid retaining unnecessary sensitive content merely because it might be useful later.

After an incident or near miss, classify the failed boundary and add a meaningful regression case. The fix may belong in authorization, source handling, a tool contract, deployment configuration, or the review process. Treating every problem as a prompt defect can leave the actual enforcement gap unchanged.

If ownership and decision rights are unresolved, AI Governance & Operating Model connects those responsibilities to the organization. The technical controls and their evidence still need an implementation owner.

Make a permission regression test concrete

Choose two controlled test users with different access and one record visible to only the first user. Ask the same supported question through each identity. Inspect retrieval results, model context where your test harness permits safe inspection, returned output, caches, and relevant logs. The second user should not receive the restricted evidence through an intermediate path even if the final answer appears harmless.

Then revoke the first user's access and repeat the case, including a previously started workflow. Record what happens to saved proposals and cached evidence. The test should match the product's stated behavior for revocation. If the design permits a defined refresh interval, document that interval and its consequences rather than claiming immediate enforcement without evidence.

For an action test, use an allowed operation on an in-scope record and the same operation on an out-of-scope record. Verify the target state and authorization result. The denied attempt should not create a partial side effect before returning an error. If the external service controls the final permission decision, retain a test that verifies the adapter handles that rejection correctly.

Finally, change the proposal after approval. The execution path should detect that the approval no longer matches the current action according to the workflow's rule. Check the actual adapter invocation, not merely the text displayed in the review interface. This catches a common class of mismatch between a visible human control and the code that performs the write.

Keep these scenarios as regression cases because they exercise concrete boundaries the application promises. Update them when identity, caching, retrieval, or tool behavior changes. They complement broader security review by preserving the specific access properties on which the workflow depends.

Build authority into the system design

Secure agentic applications make permitted behavior explicit and enforce it at data, tool, execution, and deployment boundaries. They also make failures observable and recovery practical. The model operates within that design; it does not define its own authority.

Use Custom AI Development to scope those controls with the workflow and integrations. Managed AI Operations can support the continuing responsibility for monitoring and changes under an agreed operating scope.