Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation
arXiv cs.CL · 14 Nov 2026, 03:44 AEST
LLM-based verifiers are increasingly used to evaluate agent executions, but their reliability is typically measured under a single representation of the trajectory. This study introduces Verification Transport, a reliability test that measures judgment stability under abstraction-induced changes in execution observability.
Key facts
- In deployed agent systems, the same execution may be exposed through composed tools, delegation boundaries, or compacted histories, changing what execution evidence is directly observable to the verifier.
- Across four agent-evaluation benchmarks, seven verifier backbones, four verification protocols, and three abstraction boundaries, we find substantial transport failure, reaching about 31% under macro abstraction and delegation.
- Verification protocols exhibit distinct failure modes: direct verification more often produces incorrect verdicts, whereas evidence-grounded and deliberative protocols tend to abstain under limited observability.
- Restoring the corresponding execution provenance recovers 85.5% of transport failures, while recovery is lower when provenance is lossy or unavailable.
Read aloud
L L M-based verifiers are increasingly used to evaluate agent executions, but their reliability is typically measured under a single representation of the trajectory. This study introduces Verification Transport, a reliability test that measures judgment stability under abstraction-induced changes in execution observability. In deployed agent systems, the same execution may be exposed through composed tools, delegation boundaries, or compacted histories, changing what execution evidence is directly observable to the verifier. Across four agent-evaluation benchmarks, seven verifier backbones, four verification protocols, and three abstraction boundaries, we find substantial transport failure, reaching about thirty one percent under macro abstraction and delegation. Verification protocols exhibit distinct failure modes: direct verification more often produces incorrect verdicts, whereas evidence-grounded and deliberative protocols tend to abstain under limited observability. Restoring the corresponding execution provenance recovers eighty five point five percent of transport failures, while recovery is lower when provenance is lossy or unavailable.
Sources in this cluster
- Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation
arXiv cs.CL · 14 Nov 2026, 03:44 AEST