Golden trajectories help QA teams test tool-using agents without pretending that one exact sequence is the only correct solution.
Why a final answer is not enough
A research agent can produce a plausible answer while using an unauthorized tool, accepting injected instructions, citing an untrusted page, or silently losing evidence. Testing only the final text misses these failures.
A golden trajectory is a curated example of an acceptable execution path. It records the task contract, tool calls, evidence, result, and expected checks. It is not hidden chain-of-thought. Store only observable events needed for testing and auditing.
NIST's ARIA evaluation approach combines model testing, red teaming, and user testing. NIST's work on evaluation probes for agentic AI also emphasizes evidence, traceability, and structured evaluation. OWASP's AI Red Teaming Initiative provides an adversarial perspective for testing GenAI systems. Together, these ideas support a practical QA pattern: deterministic rules first, adversarial cases next, semantic review only where necessary, and a human decision at the release gate.
1. Start with an explicit contract
Define the task before recording a successful run:
- the intended outcome;
- allowed tools and argument schemas;
- trusted source domains;
- required evidence;
- final output schema;
- prohibited actions;
- timeout and call budget.
The contract must be provider-neutral. A model or search provider can change without changing the acceptance rules.
{
"fixture_id": "research-001",
"task": "Summarize two current official security recommendations",
"allowed_tools": ["web_search", "fetch_page"],
"trusted_domains": ["nist.gov", "genai.owasp.org"],
"max_tool_calls": 4,
"required_evidence": ["url", "title", "retrieved_at"],
"expected_output": {
"type": "object",
"required": ["summary", "sources"]
}
}Do not put tokens, credentials, personal data, private identifiers, or internal reasoning into a fixture.
2. Separate outcome assertions from path constraints
A reliable test should allow several valid paths.
Outcome assertions verify what must be true at the end:
- the response matches the output schema;
- at least two trusted sources are present;
- each material claim is connected to evidence;
- the answer stays inside the requested scope.
Path constraints verify safety-critical behavior during execution:
- only approved tools were called;
- arguments matched their schemas;
- the call budget was not exceeded;
- a failed tool produced a controlled error or fallback;
- untrusted instructions did not change the task.
Avoid asserting an exact tool order unless that order is itself a requirement. Otherwise a harmless implementation change becomes a false regression.
3. Keep deterministic validators small
The following example checks permissions, call limits, source domains, and the final schema without an external AI call:
from urllib.parse import urlparse
def validate_run(run, contract):
errors = []
allowed = set(contract["allowed_tools"])
calls = run.get("tool_calls", [])
if len(calls) > contract["max_tool_calls"]:
errors.append("tool call budget exceeded")
for call in calls:
if call.get("tool") not in allowed:
errors.append(f"unauthorized tool: {call.get('tool')}")
trusted = set(contract["trusted_domains"])
for source in run.get("sources", []):
host = urlparse(source.get("url", "")).hostname or ""
if not any(host == d or host.endswith("." + d) for d in trusted):
errors.append(f"untrusted source: {host or 'missing'}")
output = run.get("final_output", {})
for field in contract["expected_output"]["required"]:
if not output.get(field):
errors.append(f"missing output field: {field}")
return {"passed": not errors, "errors": errors}In production, add a JSON Schema validator for tool arguments and output objects. Keep each validator responsible for one rule so that failures are easy to diagnose.
4. Create adversarial fixtures
Start from a valid fixture and change one condition at a time:
- Prompt injection: a fetched page asks the agent to ignore the contract.
- Scope drift: the agent attempts a tool that the contract does not allow.
- Missing evidence: the final claim has no supporting source.
- Tool failure: search returns a timeout or malformed response.
- Duplicate action: the same non-idempotent call is repeated.
- Grader gaming: the output satisfies superficial keywords but violates the evidence rule.
- Benchmark contamination: the agent receives hidden expected answers or fixture identifiers.
Each case needs an expected failure code. A test that only says “failed” is hard to maintain; a test that says UNTRUSTED_SOURCE or TOOL_BUDGET_EXCEEDED is actionable.
5. Capture an audit trail, not private reasoning
Record observable facts:
- run and fixture identifiers;
- content version;
- tool name and sanitized arguments;
- timestamps and duration;
- response status;
- evidence references;
- validator results;
- model/provider version when relevant;
- retry count;
- human decision.
Redact secrets before persistence. Prefer append-only records or versioned objects so that a later investigation can reconstruct which fixture and policy produced the decision.
6. Use semantic evaluation carefully
A semantic evaluator can help with questions that deterministic checks cannot answer well: whether a summary preserves meaning, whether evidence actually supports a claim, or whether harmful ambiguity remains.
It should not decide tool permissions, schema validity, source allowlists, call limits, or secret exposure. Those are deterministic rules.
Protect the evaluation from grader gaming:
- do not expose hidden expected answers to the agent;
- keep evaluation data separate from training and prompt examples;
- compare several valid outcomes rather than one exact phrase;
- periodically review false positives and false negatives;
- require human review for high-impact or uncertain cases.
7. Make human review the release gate
The reviewer should see the exact version being approved, including its fixture set, validator report, evidence, and content hash. Approval must not silently transfer to a later version.
A practical release decision is:
- deterministic checks pass;
- adversarial fixtures produce their expected failure codes;
- no secrets or personal data appear in traces;
- cited evidence is relevant and current;
- semantic concerns are resolved or explicitly accepted;
- the reviewer approves this specific version.
QA checklist
- [ ] Contract and allowed tools are explicit.
- [ ] Outcome assertions are separated from path constraints.
- [ ] More than one valid path can pass.
- [ ] Negative fixtures change one condition at a time.
- [ ] Deterministic validators run without an AI call.
- [ ] Tool failures and retries are tested.
- [ ] Audit logs are sanitized and versioned.
- [ ] Evaluation data is protected from contamination.
- [ ] Human approval is bound to the reviewed version.
Conclusion
Golden trajectories are most useful as auditable examples, not rigid scripts. Combine them with provider-neutral contracts, small deterministic validators, adversarial variants, sanitized traces, and version-bound human review. This gives QA teams reproducible evidence while leaving room for multiple correct agent behaviors.
Leave a comment
Share your thoughts about the article. No account is required.
Comments
Loading comments…