Capture Human Feedback on Agent Outcomes

Prerequisites

Before adding feedback controls, give every agent run a stable run ID and store enough information to retrieve that run without executing it again. The record must accept versioned prompts, model identifiers, ordered tool events, and the resulting state. If these live in separate systems, define a correlation ID that joins them; the same identifier should also anchor the feedback event. The guide to correlation identifiers across agent runs, model calls, and tools covers that boundary.

Decide who may inspect captured prompts, tool inputs, tool outputs, and state before storing them. This page does not establish a retention period or access policy; those conditions must come from the system’s own data requirements.

Capture feedback without losing the run

1. Seal the run context before showing the feedback controls.

   Persist four fields at minimum: the prompt version, model version, tool trace, and final state. The prompt version must identify the instructions actually used, rather than the latest prompt in a repository. The model version must identify the model that produced this run. Keep tool calls and results in execution order. Record both the final answer shown to the reviewer and any persistent state left by the tools.

   A thumbs-up without the prompt version, model version, tool trace, and final state cannot explain what behavior was approved. The same visible answer might have followed different instructions, called different tools, or left different external state. Bind the feedback to the sealed run ID so later prompt or model changes cannot silently change its meaning.

   Use the run record as the source for reconstructing what an agent did after the fact. Do not reconstruct it from application logs only when the final state lives elsewhere. The supplied OpenTelemetry Generative AI semantic-conventions entry says, as checked on 2026-08-26, that these conventions have moved to a dedicated repository and that the old page is no longer maintained. If you use those conventions, store the convention revision with the trace rather than assuming its field definitions will remain fixed.

   Cost and limit: complete run records consume storage and require instrumentation across model and tool boundaries. If policy prevents retaining raw content, keep approved references or redacted snapshots, but mark the record incomplete. A bare rating attached to an incomplete record is useful as sentiment, not as evidence that a specific behavior was correct.

2. Ask about outcome correctness and presentation quality separately.

   Present two independent questions after the reviewer has inspected the final state:

   • Did the agent reach the correct outcome?
   • Was that outcome presented in a usable form?

   Do not combine them into one five-star score. Feedback should distinguish outcome correctness from presentation quality because they lead to different fixes. A correct repository change with a confusing explanation is a presentation failure. A polished explanation attached to the wrong repository change is an outcome failure. Treating both as “three stars” removes the condition that decides what to repair.

   Store explicit values such as correct, incorrect, or uncertain for the outcome, and acceptable or needs_revision for presentation. Preserve an optional note for each dimension. The labels are less important than keeping the dimensions separate and defining them consistently for reviewers.

   The NIST AI Risk Management Framework, checked on 2026-08-26, is a voluntary framework intended to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI systems. Apply that evaluation principle by recording the affected condition alongside correctness: what task was attempted, what state was changed, and what consequence the reviewer inspected. A generic approval detached from the affected condition cannot support that review.

   Cost and limit: two questions add reviewer effort and may reduce completion compared with one click; no completion-rate figure is verified here. Keep the first interaction to the two decisions, then reveal note fields only when needed. Do not ask about presentation when there was no user-facing presentation, but still record that the dimension was not applicable rather than treating it as approved.

3. Request a correction whenever the outcome is wrong or uncertain.

   After incorrect or uncertain, ask the reviewer for the expected action or answer. For a tool-using agent, capture the expected state transition: which action should have happened, which action should not have happened, and what final state should remain. For an answer-only agent, capture the corrected answer or the criteria it must satisfy.

   Corrections are more useful than ratings when they preserve the expected action or answer as an evaluation target. “Wrong” identifies a failed run; “do not close the incident until the health check passes” identifies behavior that a future run can be tested against. Keep the original output beside the correction. Replacing it destroys the comparison that explains the defect.

   Ask the reviewer to scope the correction to the conditions they observed. Record whether it applies to this input only, to a named class of inputs, or to a general rule. One reviewer’s preferred wording should not silently become a correctness requirement for every run. When reviewers disagree or cannot state an expected result, retain the case as unresolved feedback rather than manufacturing a ground-truth answer.

   Cost and limit: writing a correction takes longer than choosing a rating and may require someone with authority over the task. Make ratings available for low-effort collection, but require a correction before a negative item enters an evaluation set. A rating without an expected result can still guide investigation; it cannot serve as a complete test oracle.

4. Convert reviewed corrections into evaluation cases.

   Promote a correction only after checking it against the sealed run. The evaluation item should carry the original task input, relevant starting state, expected action or answer, outcome criterion, and source run ID. Keep prompt and model versions as provenance, not as the definition of the expected behavior: the target should survive a later implementation change.

   Separate the graders or checks for outcome and presentation just as the feedback form does. Otherwise a formatting improvement can hide an outcome regression, or a correct action can fail solely because its explanation changed. For state-changing work, evaluate the resulting state as well as the visible answer; a tool trace alone shows attempted actions, not necessarily the final condition.

   The OpenAI guide to working with evals, checked on 2026-08-26, describes an evaluation as test data plus testing criteria and shows a human-provided correct label used as the ground-truth comparison. Follow that pattern without tying the stored feedback to one evaluation product: preserve the input, human correction, and criterion in a portable record. Feed reviewed cases into the same decision process used for release thresholds on agent evaluation regressions.

   Cost and limit: promotion requires human review and maintenance when the intended behavior changes. Do not turn every preference into a blocking case. Outcome corrections can become correctness targets; presentation notes should become presentation checks only when the requirement is explicit enough to test.

5. Run a reconstruction check before relying on the feedback.

   Select a captured item and retrieve it using only its feedback record and run ID. Confirm that a reviewer can see the exact prompt version, model version, ordered tool trace, final answer, persistent final state, outcome judgment, presentation judgment, and correction. The display may summarize large traces, but the underlying events must remain addressable.

   Do not rerun the agent to fill a missing field. A rerun creates a new outcome and must receive a new run ID. If state is intentionally carried between runs, keep that boundary explicit using the system’s agent memory and state model; otherwise feedback from one run can accidentally approve state inherited from another.

   Cost and limit: reconstruction checks add engineering and review time, especially when traces cross services. Run them whenever the capture schema, tracing convention, or feedback UI changes. If the record cannot reconstruct what the reviewer judged, mark it unusable for evaluation until the missing context is restored.

Expected result

The completed system stores each feedback event against one reproducible agent run. An approval identifies the exact prompt version, model version, tool trace, and final state. Correctness and presentation remain separate signals. A negative result preserves the reviewer’s expected action or answer, and only reviewed, scoped corrections become evaluation targets.