# Part 4: Why LLM agents need governance, permissions and evidence

Canonical page: https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/why-llm-agents-need-governance/

An LLM agent is an application in which a large language model (LLM) can use tools to pursue a task over several steps. Governance defines who may delegate that work, what authority the agent receives and who is responsible for accepting or challenging the outcome. It also covers the evidence needed for decisions and what happens when the task changes or a control fails. These questions arise because an agent can move beyond generating text to changing files or invoking external operations. Permissions and review arrangements let people decide how much action to delegate and assess what the application actually did.

## Who authorises the work and accepts the outcome?

Suppose an assistant is asked to update a name-formatting function in a software repository so that it handles empty input, add relevant tests and prepare a release note. This is a hypothetical task; the expected empty-input result remains to be specified.

The requester defines the intended outcome. A repository owner may determine who can accept changes to the project's files. A separate release decision concerns whether the change reaches users. These responsibilities can belong to the same person in a small project, but accepting an edit and publishing a release are different decisions.

NIST's voluntary AI Risk Management Framework addresses documented responsibilities, communication, leadership responsibility and oversight of human and AI arrangements. Its GOVERN function spans risk identification, measurement and management. The framework provides guidance; it does not certify a particular implementation or establish its compliance. [NIST AI RMF 1.0, GOVERN](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf)

Accountability requires the responsible person to understand the decision and have access to relevant evidence. Naming a reviewer is insufficient if that person cannot inspect the proposed change or influence what happens next.

## Capability, permission and instruction

Capability is what an application can technically do. Permission is what it is authorised to do in the present situation. An instruction tells the model how to behave. A tool may expose a broader range of operations than a particular assignment authorises.

Suppose the assistant can edit the repository and invoke a release command. The request to prepare a release note does not, by itself, answer whether publishing a release is authorised. That depends on the request and the permission arrangement for the project.

An instruction such as "prepare the release for review" guides the model. Executable enforcement is a separate mechanism: the application checks whether an operation is permitted before executing it. The surrounding software, often called the agent's *harness*, can perform this check independently of the model's generated request. Both mechanisms can support the same restriction, but a written instruction alone is not evidence that an executable restriction exists.



![Authorised roles define task scope, set permissions and assign acceptance or scope decision rights. The task leads to a model proposal, permission enforcement and, where permitted, a tool operation. Results and refusal records feed responsible review against the criteria. An authorised decision can accept the work or change the task's scope.](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/figures/fig-04-01-decision-rights-connect-permissions-to-acceptance.svg)

*Figure 4.1. Assigned roles define the task, permissions and decision rights. The application executes permitted operations and exposes evidence for review. Acceptance and authorised changes in scope remain distinct from the model's proposal, and permissions may be assigned in advance.*

## What controls do and how they handle failure

A control is a measure used to manage a risk. Prevention, detection and recovery describe different purposes a control can serve. The repository task provides examples of each:

| Purpose | Hypothetical measure | Question it addresses |
| --- | --- | --- |
| Prevention | Block an operation outside the authorised scope before execution | Can an unauthorised action proceed? |
| Detection | Compare the resulting diff, the record of file changes, with the request | Did the work introduce an unexpected alteration? |
| Recovery | Restore a reversible change after an unsuccessful outcome | How can the project return to an acceptable state? |

Inspection after an action serves a different purpose from controlling whether the action can occur. Recovery depends on the operation as well: restoring a local file and reversing something already published can involve different practical possibilities.

A control also needs a defined response when it cannot evaluate an operation. *Fail closed* means withholding the operation in that situation; *fail open* means allowing continuation. This describes behaviour during a control failure, separately from whether its normal decisions are correct. The [Claude Code guidance in Part 5](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/claude-code-guidance-and-enforcement/) sets out different failure-handling choices for hooks, the programs invoked at specified points in the agent's operation.

## Retrieved content does not inherit the requester's authority

Prompt injection is an attempt to redirect a model by introducing instructions that conflict with an application's authorised task or controls. In an indirect prompt injection, those instructions arrive through material the application retrieves, rather than through the direct user request. Research by Greshake and colleagues demonstrated how an external author could use this route to influence an LLM application's behaviour. [Greshake et al.: Not What You've Signed Up For](https://arxiv.org/html/2302.12173v2)

An assistant examining the repository might encounter a comment, an issue description or an external documentation page. Such material can provide useful evidence about the task. Its author does not acquire permission to expand the assistant's work simply because the content was retrieved.

If a document includes a release instruction, its *provenance*, meaning its origin and authorship, becomes relevant. Who wrote it, and what authority does that source have for this task? Reading documentation to understand a function and receiving an authorised instruction from the requester remain distinct events, even when both use natural language.



![Authorised task instructions and externally retrieved content enter context assembly through separate lanes. Only authorised permissions connect to the execution-control policy. The model's proposed action passes to configured controls before permitted execution. Results or refusal feedback return to context. Retrieved material has no connection that grants permission, although it can still influence the model's response.](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/figures/fig-04-02-retrieved-content-supplies-evidence-not-permission.svg)

*Figure 4.2. Both authorised instructions and retrieved material can enter a model's context. Their authority differs: a document's contents do not grant permission to expand the task. Provenance identifies the source, while configured runtime controls govern proposed execution; neither the labels nor this schematic establish immunity to prompt injection.*

## Give reviewers evidence and a decision they can make

A human review is useful when the reviewer understands the proposed decision, has relevant evidence and can influence the outcome. For the name-formatting task, a review package could contain the requested behaviour, the diff and the release note, together with records of any tests executed. The reviewer could then check whether the implementation and description match the request. Merely generating this package would not establish that the change had been accepted.

If the empty-input change exposes a larger compatibility issue, the next decision may concern the scope of the assignment. The assistant can explain the issue and the available options. The person or role authorised to change that scope can then decide whether to expand the work, clarify the requirement or postpone the change. A record of that decision gives subsequent work an identifiable basis. For Claude Code, [Part 5: How to govern Claude Code by separating guidance from enforcement](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/claude-code-guidance-and-enforcement/) sets out the technical controls an organisation can adopt to apply these principles, and [Part 6: How to diagnose Claude Code refusals and test governance hooks](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/claude-code-refusals-and-hook-testing/) explains how to test them.

Part 4 of 8 in the series [LLM harnesses and governance](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/).

Previous part: [Part 3: How LLM context, memory and handovers support long-running work](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/llm-context-memory-and-handovers/)

Next part: [Part 5: How to govern Claude Code by separating guidance from enforcement](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/claude-code-guidance-and-enforcement/)

Vikram Mohan is not a solicitor or barrister. Nothing on this site is legal advice. For legal advice, speak to a regulated legal professional.

© Vikram Mohan - All rights reserved. No liability accepted for content on this website. [LinkedIn](https://www.linkedin.com/in/mohanvikram/) · [Privacy](https://vikrammohan.com/privacy)