# Part 1: How large language models work and what they need to complete a task

Canonical page: https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/how-large-language-models-work/

A large language model (LLM) generates language using patterns learned during training and the information supplied when it runs. It can explain concepts, draft text and propose code. Completing a task also depends on the intended outcome and how success will be checked. A coding assistant, for example, needs current project information and a way to apply and inspect changes. The model generates responses; the surrounding application provides access to files and tools. A useful explanation may require only one response. Work involving several actions needs a system that carries information and results between those actions.

## Training, tokens and inference

Training adjusts a model's parameters, the numerical values that encode its learned patterns. Inference is the use of that trained model to process inputs and generate outputs. Text is represented through tokens, which need not be whole words. In typical autoregressive generation, the model predicts each next token using the available context and preceding output. These statistical relationships do not themselves establish that a statement corresponds to an external fact. NIST calls confidently expressed but erroneous or internally inconsistent content *confabulation*, also commonly called hallucination. [NIST AI 600-1, section 2.2](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf)

An answer can therefore be coherent and useful while still needing verification. A proposed implementation may identify a workable design, but assessing it requires information about the project where it will be used. Reading the generated code answers different questions from applying it and checking its behaviour.



![Task instructions and selected project evidence feed context tokens. Context tokens and trained parameters feed autoregressive inference, which produces the next token. Generated tokens return as preceding output for later generation. No connection changes the trained parameters or executes an external action.](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/figures/fig-01-01-inputs-to-autoregressive-inference.svg)

*Figure 1.1. During typical autoregressive inference, generation uses the supplied context, trained parameters and preceding output. The output may describe an action or request a tool; execution belongs to the surrounding application.*

## Define the outcome the assistant should achieve

Consider a hypothetical assignment: update a name-formatting function in a software repository so that it handles empty input, add relevant tests and prepare a release note. A repository contains the project's files and their version history. The assignment gives a direction, but the expected behaviour for empty input remains unspecified.

What should the function return? Which existing behaviour should remain compatible? Where do tests belong? Is the release note intended for users or maintainers? Answers to these questions let the assistant assess a proposed change against the requester's intention. Without them, several different implementations could appear reasonable.

Acceptance criteria are the conditions used to judge whether a task is complete. For this assignment, they could identify the expected empty-input result and the behaviour that must remain compatible. They give the assistant and reviewer a shared basis for assessment without prescribing every implementation step. Any unresolved requirement remains a question to answer before claiming that the change satisfies it.

## Provide relevant information about the current project

General programming knowledge cannot establish this function's current implementation, the code that calls it or the project's test conventions. The assistant needs evidence from the task environment.

The model's *context* is the information supplied for its present inference. It can include instructions, conversation history, tool information and retrieved material. Context engineering concerns selecting and managing that information as work progresses. A repository being available on disk is different from its relevant contents being present in the model's context. [Anthropic: Effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)

In the example, reading the function reveals its current behaviour. Examining a caller, meaning code that invokes the function, can expose a compatibility concern. Nearby tests show established conventions. Each provides evidence for a particular decision. Reading more files is useful when they resolve relevant uncertainty; the file count alone says little about whether the assistant understands the requested change.

## Use tools to act and inspect the result

A tool gives an LLM application access to an external operation, such as reading a file or running a test command. Anthropic documents a model producing a structured request, application code or provider infrastructure executing it, and the result returning to the model. Generating the request and executing the operation are separate events. [Anthropic: How tool use works](https://platform.claude.com/docs/en/agents-and-tools/tool-use/how-tool-use-works)

For the hypothetical repository task, different records would support different parts of a completion claim:

| Evidence | What it helps establish |
| --- | --- |
| A proposed patch, describing edits to files | The change being suggested |
| A repository diff, showing differences between file versions | The changes present in the inspected working copy |
| A record of a test command and its output | What was executed and what that execution reported |
| A release note compared with the diff | Whether the description corresponds to the change |

If tests are executed, their results need to be assessed against the code tested and the behaviour they exercise. A passing result does not establish every possible property of the function. A release note can accurately describe the intended change even when the implementation has a defect. A useful completion report connects each requested outcome to the available evidence and identifies anything unresolved.



![Task acceptance criteria, a completion claim, an inspected repository diff, a test record or missing result, and a release note feed an evidence review. The review can lead to accepted work when criteria are met or unresolved work when evidence is missing or mismatched. The completion claim has a distinct dashed connection.](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/figures/fig-01-02-evidence-supporting-a-completion-decision.svg)

*Figure 1.2. For the hypothetical name-formatting change, acceptance depends on comparing the requested behaviour with the changed files, available test evidence and release note. A completion claim is one item to assess, and an unavailable test outcome remains a gap.*

## What an agent harness adds

An LLM agent is an application that lets a model use tools over successive steps to pursue a task. Its *harness* is the surrounding software that manages those interactions, carries information between steps and provides the environment for external operations.

A model does not need a full agent harness to perform inference or provide a useful explanation. The additional machinery becomes relevant when work involves editing files, executing tests, retaining progress or handling a failed operation. These are responsibilities of the complete application, even when the model chooses which action to request next.

For the name-formatting change, the application could return a failed test result to the model for further investigation. The next decision would then use that observed result alongside the task requirements and relevant code. If the expected empty-input behaviour remained unspecified, the test result alone would not resolve that requirement.

Part 1 of 8 in the series [LLM harnesses and governance](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/).

Next part: [Part 2: What an LLM harness is and how it runs tools](https://vikrammohan.com/thought-leadership/llm-harnesses-and-governance/what-is-an-llm-harness/)

Vikram Mohan is not a solicitor or barrister. Nothing on this site is legal advice. For legal advice, speak to a regulated legal professional.

© Vikram Mohan - All rights reserved. No liability accepted for content on this website. [LinkedIn](https://www.linkedin.com/in/mohanvikram/) · [Privacy](https://vikrammohan.com/privacy)