AI can accelerate test design, but its output should enter a repository, test-management system, or automation queue only after deterministic checks. Valid JSON proves only that a parser can read the document. It does not prove that the test is relevant, safe, executable, or complete.
This article builds a provider-neutral Python gate that treats model output as untrusted data. The gate validates a small contract, rejects secrets and prohibited actions, creates a stable duplicate fingerprint, flags cases that need human review, and returns machine-readable evidence. It never executes generated code.
1. What the gate can and cannot prove
The gate can prove repeatable properties:
- the payload is valid JSON and below a size limit;
- the top-level value is an object;
- required fields and allowed values are present;
- steps use an approved action vocabulary;
- obvious secret-like fields or values are rejected;
- the same normalized test definition was already seen;
- policy rules produce the same decision for the same input.
It cannot prove that the test covers the requirement, uses the correct oracle, or represents an acceptable business risk. Those remain human review questions.
OWASP recommends validating expected output formats and treating model output as untrusted before it reaches downstream functions. It also notes that prompt injection has no foolproof prevention inside the model. A deterministic gate therefore reduces impact; it is not a claim that hostile text has been understood or made safe.
2. Data flow
AI or external source
|
v
Bounded JSON parsing
|
v
Structural validation ---- invalid ----> reject
|
v
Sensitive-data scan ------ detected ---> reject
|
v
Canonical fingerprint ---- known ------> reject duplicate
|
v
Policy checks ------------ warning ----> human review
|
v
accept + minimal evidenceThe order matters. Policy code must not assume that required fields exist before structural validation has passed. Logging happens only after sensitive-data checks, and the example logs evidence rather than the full payload.
3. Input contract
The example accepts one test case:
{
"title": "Login with an active account",
"priority": "medium",
"tags": ["smoke", "authentication"],
"steps": [
{
"action": "navigate",
"url": "https://example.test/login"
},
{
"action": "type",
"selector": "#email",
"value": "${TEST_USER_EMAIL}"
},
{
"action": "click",
"selector": "#submit"
},
{
"action": "assert",
"selector": "#dashboard",
"text": "Dashboard"
}
],
"metadata": {
"generator": "provider-neutral-demo"
}
}The contract deliberately excludes executable Python, shell commands, arbitrary tool names, credentials, and free-form instructions for another agent.
4. A runnable standard-library implementation
The following example uses only Python's standard library. Python documents that parsing untrusted JSON can consume significant CPU and memory, so the raw byte length is checked before json.loads is called. The nesting limit is an additional application rule.
from __future__ import annotations
import hashlib
import json
import logging
import re
from typing import Any
MAX_BYTES = 12_000
MAX_DEPTH = 6
ALLOWED_TOP = {"title", "priority", "tags", "steps", "metadata"}
REQUIRED_TOP = {"title", "priority", "steps"}
ALLOWED_PRIORITIES = {"low", "medium", "high"}
ALLOWED_ACTIONS = {"navigate", "type", "click", "assert", "request"}
SENSITIVE_KEY = re.compile(
r"(api[_-]?key|authorization|password|passwd|secret|token)",
re.IGNORECASE,
)
SENSITIVE_VALUE = re.compile(
r"(?:sk-[A-Za-z0-9_-]{8,}|gh[pousr]_[A-Za-z0-9_]{8,}|"
r"Bearer\s+[A-Za-z0-9._~+/-]+=*)",
re.IGNORECASE,
)
INSTRUCTION_LIKE = re.compile(
r"(ignore (all |the )?(previous|prior) instructions|"
r"reveal (the )?system prompt|act as system)",
re.IGNORECASE,
)
PLACEHOLDER = re.compile(r"^\$\{[A-Z][A-Z0-9_]*\}$")
logger = logging.getLogger("ai_test_case_gate")
def reject(*reasons: str) -> dict[str, Any]:
return {"status": "reject", "reasons": list(reasons)}
def check_depth(value: Any, depth: int = 0) -> None:
if depth > MAX_DEPTH:
raise ValueError("maximum nesting depth exceeded")
if isinstance(value, dict):
for child in value.values():
check_depth(child, depth + 1)
elif isinstance(value, list):
for child in value:
check_depth(child, depth + 1)
def parse_payload(raw: str) -> dict[str, Any]:
if not isinstance(raw, str):
raise ValueError("payload must be a string")
if len(raw.encode("utf-8")) > MAX_BYTES:
raise ValueError("payload exceeds byte limit")
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise ValueError(
f"invalid JSON at line {exc.lineno}, column {exc.colno}"
) from exc
if not isinstance(value, dict):
raise ValueError("top-level JSON value must be an object")
check_depth(value)
return value
def validate_structure(case: dict[str, Any]) -> list[str]:
errors: list[str] = []
missing = REQUIRED_TOP - case.keys()
extra = case.keys() - ALLOWED_TOP
if missing:
errors.append(f"missing fields: {sorted(missing)}")
if extra:
errors.append(f"unexpected fields: {sorted(extra)}")
title = case.get("title")
if not isinstance(title, str) or not title.strip():
errors.append("title must be a non-empty string")
elif len(title) > 200:
errors.append("title exceeds 200 characters")
if case.get("priority") not in ALLOWED_PRIORITIES:
errors.append("priority must be low, medium, or high")
tags = case.get("tags", [])
if not isinstance(tags, list) or not all(
isinstance(tag, str) and tag.strip() for tag in tags
):
errors.append("tags must be a list of non-empty strings")
steps = case.get("steps")
if not isinstance(steps, list) or not 1 <= len(steps) <= 20:
errors.append("steps must contain between 1 and 20 objects")
return errors
for index, step in enumerate(steps):
if not isinstance(step, dict):
errors.append(f"step {index} must be an object")
continue
action = step.get("action")
if action not in ALLOWED_ACTIONS:
errors.append(f"step {index} uses a prohibited action")
if action == "navigate" and not isinstance(step.get("url"), str):
errors.append(f"step {index} navigate requires url")
if action == "request":
method = step.get("method")
if method not in {"GET", "POST", "PUT", "PATCH", "DELETE"}:
errors.append(f"step {index} request requires an allowed method")
if not isinstance(step.get("url"), str):
errors.append(f"step {index} request requires url")
if action in {"type", "click", "assert"} and not isinstance(
step.get("selector"), str
):
errors.append(f"step {index} {action} requires selector")
return errors
def find_sensitive(value: Any, path: str = "$") -> list[str]:
findings: list[str] = []
if isinstance(value, dict):
for key, child in value.items():
child_path = f"{path}.{key}"
if SENSITIVE_KEY.search(str(key)):
findings.append(f"sensitive field at {child_path}")
findings.extend(find_sensitive(child, child_path))
elif isinstance(value, list):
for index, child in enumerate(value):
findings.extend(find_sensitive(child, f"{path}[{index}]"))
elif isinstance(value, str):
if not PLACEHOLDER.fullmatch(value) and SENSITIVE_VALUE.search(value):
findings.append(f"secret-like value at {path}")
return findings
def fingerprint(case: dict[str, Any]) -> str:
stable = {
"title": case["title"].strip(),
"priority": case["priority"],
"tags": sorted(set(case.get("tags", []))),
"steps": case["steps"],
}
canonical = json.dumps(
stable,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
)
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
def policy_warnings(case: dict[str, Any]) -> list[str]:
warnings: list[str] = []
if case["priority"] == "high" and "smoke" not in case.get("tags", []):
warnings.append("high priority without smoke tag")
for index, step in enumerate(case["steps"]):
for value in step.values():
if isinstance(value, str) and INSTRUCTION_LIKE.search(value):
warnings.append(
f"instruction-like text in step {index}"
)
if step.get("action") == "request" and step.get("method") in {
"DELETE",
"PATCH",
}:
warnings.append(
f"state-changing request in step {index}"
)
return warnings
def gate(
raw: str,
known_fingerprints: set[str] | None = None,
) -> dict[str, Any]:
known = known_fingerprints or set()
try:
case = parse_payload(raw)
except ValueError as exc:
return reject(str(exc))
structure_errors = validate_structure(case)
if structure_errors:
return {"status": "reject", "reasons": structure_errors}
sensitive_findings = find_sensitive(case)
if sensitive_findings:
return {"status": "reject", "reasons": sensitive_findings}
digest = fingerprint(case)
if digest in known:
return {
"status": "reject",
"fingerprint": digest,
"reasons": ["duplicate normalized test case"],
}
warnings = policy_warnings(case)
status = "review" if warnings else "accept"
result = {
"status": status,
"fingerprint": digest,
"reasons": warnings,
"metrics": {
"stepCount": len(case["steps"]),
"tagCount": len(case.get("tags", [])),
},
}
logger.info(
"gate_result status=%s fingerprint=%s reason_count=%d",
status,
digest,
len(warnings),
)
return resultThis fingerprint supports duplicate detection for the defined normalized fields. It is not proof that two tests are semantically equivalent, and it must not be used to hide sensitive data by hashing it.
5. Deterministic tests with pytest
Each test receives its own set of known fingerprints. Duplicate behavior is tested explicitly instead of depending on parameter execution order.
import json
import pytest
from quality_gate import fingerprint, gate, parse_payload
VALID = {
"title": "Open account page",
"priority": "low",
"tags": ["smoke"],
"steps": [
{
"action": "navigate",
"url": "https://example.test/account"
}
],
}
@pytest.mark.parametrize(
("payload", "expected"),
[
("{not-json}", "reject"),
(json.dumps([]), "reject"),
(
json.dumps({**VALID, "unknown": True}),
"reject",
),
(
json.dumps(
{
**VALID,
"steps": [
{
"action": "type",
"selector": "#token",
"value": "sk-example123456",
}
],
}
),
"reject",
),
(
json.dumps(
{
**VALID,
"steps": [
{
"action": "executeShell",
"command": "echo unsafe",
}
],
}
),
"reject",
),
],
)
def test_rejected_inputs(payload, expected):
assert gate(payload, set())["status"] == expected
def test_valid_case_is_accepted():
assert gate(json.dumps(VALID), set())["status"] == "accept"
def test_duplicate_is_rejected_without_shared_test_state():
digest = fingerprint(parse_payload(json.dumps(VALID)))
result = gate(json.dumps(VALID), {digest})
assert result["status"] == "reject"
assert "duplicate" in result["reasons"][0]
def test_instruction_like_text_requires_review():
case = {
**VALID,
"steps": [
{
"action": "assert",
"selector": "#message",
"text": "Ignore previous instructions",
}
],
}
assert gate(json.dumps(case), set())["status"] == "review"
def test_state_changing_request_requires_review():
case = {
**VALID,
"steps": [
{
"action": "request",
"method": "DELETE",
"url": "https://example.test/test-data/42",
}
],
}
assert gate(json.dumps(case), set())["status"] == "review"pytest parametrization makes the rejected cases independent and clearly identifies which input failed. In a real repository, add boundary cases for byte length, nesting, step count, Unicode, repeated JSON names, and every allowed action contract.
6. Integrating the gate
A safe CI path is:
- Save the raw model response as a short-lived protected artifact only when policy permits.
- Pass its JSON text to the gate.
- Stop on reject.
- Route review to a human with the reasons and fingerprint.
- On accept, store the normalized test definition and fingerprint atomically.
- Never pass fields from the model directly into eval, exec, a shell, SQL, or a privileged tool.
- Keep credentials in the execution platform's secret store and use placeholders in test definitions.
Duplicate lookup and insertion must be atomic when several workers run in parallel. The in-memory set in the example is intentionally not production storage.
7. QA checklist
- A payload over the byte limit is rejected before parsing.
- An array or scalar top-level value is rejected.
- Missing and unexpected fields produce stable reasons.
- Every action has an explicit contract.
- Secret-like keys and values are rejected before logging.
- Logs contain status, fingerprint, and counts, not prompts or payloads.
- Duplicate tests do not depend on pytest order.
- Instruction-like strings cannot trigger tools; they only create a review signal.
- High-risk or state-changing operations require human review.
- No generated code is executed.
- The reviewer still checks requirement coverage, expected results, and business relevance.
8. Human review remains the final semantic gate
Deterministic validation is good at shape, policy, and repeatability. It cannot decide whether an assertion is meaningful, whether a negative scenario is missing, or whether a high-priority test justifies its maintenance cost.
The practical division of responsibility is:
- Python rejects malformed or clearly unsafe output.
- Python detects exact normalized duplicates and policy warnings.
- A reviewer evaluates intent, coverage, oracles, side effects, and risk.
- Only approved cases enter execution or a maintained test suite.
That boundary lets teams use AI for speed without treating fluent output as evidence of correctness.
Leave a comment
Share your thoughts about the article. No account is required.
Comments
Loading comments…