Designing AI agents people can review and trust

Trust is a judgement, not a product claim

Fluency can make an answer feel more reliable than its evidence warrants. Labels such as “AI-generated” or generic warnings may disclose the technology, but they do not tell a reviewer whether this particular output is suitable for use. Nor does a detailed explanation automatically establish correctness.

NIST separates transparency, explainability and interpretability. Transparency helps answer what happened; explainability concerns how the system operates; interpretability concerns what an output means in its intended context (NIST, 2023a). A useful agent experience needs an appropriate combination. A decision-maker may need evidence and limitations; an operator may also need tool status and recovery options; an investigator may need the complete trace.

The relevant question is not “Is the agent trustworthy?” in the abstract. It is “What information and controls does this person need to decide whether to rely on this output for this purpose?”

Design for review, not explanation theatre

A review interface should reduce the distance between a material claim and the information needed to challenge it. It should not simulate certainty with a confidence gauge that has no validated meaning, or expose a stream of technical activity merely to look transparent.

Raw chain-of-thought is especially unsuitable as a general user explanation. OpenAI does not show raw reasoning traces to users and notes that such traces may be unfit for direct display; useful ideas can instead be represented in the answer or a generated summary (OpenAI, 2024). Research on agent governance also cautions that chain-of-thought may not faithfully represent the process that produced a decision (Shavit et al., 2023).

Reviewable evidence is more concrete: the sources consulted, the passages used, the assumptions applied, the tools called, the external changes proposed and the failures encountered. These artefacts support verification without asking a reviewer to treat model-generated reasoning as ground truth.

Connect evidence to the claims it supports

A bibliography at the end of a brief is useful, but it leaves the reviewer to reconstruct which source supports each statement. Place references beside material claims. Preserve the source title, publisher, date or version, and the relevant passage or location where practical.

Test the relationship, not just the link. A source may mention the same topic without supporting the conclusion. A citation checker can confirm that a URL resolves and that quoted text appears; a reviewer may still need to judge whether the evidence is authoritative, current and appropriately interpreted.

Consider the Research Agent asked to compare two operating models. It finds three sources favouring one approach and one credible source describing a significant constraint. A reviewable brief should connect each major finding to its evidence, surface the disagreement and distinguish sourced facts from the agent’s synthesis. Quietly omitting the constraint produces a cleaner answer but a poorer decision record.

Microsoft’s guidance describes traceability as making agent actions and decisions reviewable and transparency as enabling users to verify information sources (Microsoft, 2026a). This is a design requirement, not simply a logging feature.

Show the assumptions that change the answer

Not every implementation detail deserves attention. Surface assumptions that could change the interpretation or next action: the date range searched, excluded data, definitions chosen, unavailable systems, authority thresholds or the meaning of “complete”.

For the Research Agent, “public sources published in the last 24 months” is decision-relevant. So is a decision to treat two company announcements as independent evidence when they repeat the same underlying study. Conversely, the order of harmless search queries is usually better kept in the operational trace.

Provide assumptions in a short, structured block near the answer, with an option to inspect details. If a reviewer changes a material assumption, treat that as a new run or version. The original output should remain attributable to the conditions under which it was produced.

Communicate uncertainty as a decision input

Uncertainty is not a disclaimer to append to every result. It should identify what is unknown, why it is unknown and what that means for use. “This may be inaccurate” transfers little practical information. “Two primary sources disagree on the 2025 figure; the recommendation does not depend on the difference” is more useful.

Avoid unsupported numerical confidence. Unless a score has been calibrated for the task, a percentage can create false precision. Use specific status language instead: evidence sufficient for the stated conclusion; evidence incomplete but adequate for a provisional view; material conflict requiring review; or insufficient evidence to answer.

NIST recommends documenting system knowledge limits and how outputs should be used and overseen by people (NIST, 2023b). In practice, that means connecting uncertainty to a boundary: proceed, proceed with a condition, seek more evidence or escalate.

Place review at meaningful action boundaries

Human oversight is effective only when the person has time, authority and enough context to change the outcome. Requiring approval after every harmless step creates noise. Asking for approval after an external action has already occurred is merely notification.

Place checkpoints before consequential or difficult-to-reverse actions. Show exactly what will happen: the recipient and final message before sending; the record and changed fields before updating; the selected evidence and recommendation before a decision brief is distributed. Provide clear choices such as approve, edit, reject or escalate, and record the decision.

Google Cloud’s human-in-the-loop pattern pauses an agent at a defined checkpoint so a person can approve, correct or provide input before execution continues (Google Cloud, 2026). Microsoft similarly recommends human approval for actions affecting people, money or compliance (Microsoft, 2026b).

The consequence should determine the checkpoint. Reviewing a low-risk internal draft may be enough after completion. Changing a customer record may require review before commitment. Some actions should remain outside the agent’s authority altogether.

Make failure and escalation legible

An agent that cannot complete its job should not fill the expected format with a plausible substitute. Distinguish at least four states: missing input, insufficient evidence, tool or system failure, and a request outside the authorised scope.

Each state needs a useful next step. The Research Agent might say that one required database was inaccessible, list the public evidence it did review, identify which conclusions remain unsupported and offer to resume when access is restored. If sources materially conflict, it should present the conflict and request domain judgement rather than silently choosing.

A good escalation package includes the original request, work completed, evidence gathered, unresolved issue, consequence of proceeding and the decision required from a person. This preserves momentum without disguising partial completion as success.

Keep the right records for each audience

One screen cannot serve every accountability need. Separate the concise decision record from the detailed operational record, while linking them through a stable run identifier and version.

Retain records proportionately. Logs can contain sensitive data, so visibility and retention should follow the same access, privacy and governance rules as the workflow itself. More logging is not automatically better accountability.

A practical reviewable-output record

The following structure is deliberately compact. Adapt its depth to the consequence of the task.

This record should be generated from system events and retained artefacts where possible, not reconstructed from the final prose alone. The purpose is to support a real review, not to certify the agent by its own description of what it did.

Evaluate the quality of human review

Adding a person to the workflow does not automatically create control. Reviewers may approve too quickly, lack domain knowledge or be overwhelmed by irrelevant detail. Test the combined human–agent system.

Measure whether reviewers notice seeded citation problems, unsupported conclusions and changed assumptions. Observe the time required, the corrections made, disagreement between reviewers and the frequency of override or escalation. Ask whether the interface directs attention to the highest-consequence information.

NIST recommends defining and documenting human-oversight processes and involving relevant domain experts in assessment (NIST, 2023b). Oversight is therefore a designed and evaluated capability—not a label attached to an approval button.

Article 01 asked whether a workflow needs an agent. Article 02 asked what evidence justifies release. This final question is operational: once an agent participates in work, can the people responsible understand the result, exercise meaningful control and recover when conditions exceed its limits?

Do not ask an agent to appear trustworthy. Design the work so an informed person can decide when trust is warranted.