Towards Verifiable LLM Infra: Agent-Driven Proof of Agent-Implemented Inference
Hacker News · 12 Nov 2026, 06:58 AEST
A new formal verification workflow called vstack has been introduced for coding agents in LLM infrastructure. This aims to address the verification challenges that arise as AI agents build complex inference engines.
Key facts
- We introduce vstack, a 5-stage formal verification workflow for coding agents, and veriserve, a prototype verifiable LLM serving engine with PD disaggregation built around a formally verified KV-page lifecycle design model.
- Applying vstack to production Prefill-Decode serving uncovered three classes of real RDMA and cache concurrency bugs across SGLang and vLLM, resulting in 6 proposed upstream pull requests.
- Veriserve backs its PdLedger design model with a 226-module, 516,973-obligation TLAPS safety proof while adding only 0.18% mean p50 TPOT overhead on NVIDIA GB200.
- Over the past three years, inference engines like vLLM and SGLang have evolved from single-node executors into asynchronous distributed systems, layering paged KV memory, shared prefix caching, and disaggregated prefill-decode RDMA transfers.
Read aloud
A new formal verification workflow called vstack has been introduced for coding agents in L L M infrastructure. This aims to address the verification challenges that arise as A I agents build complex inference engines. We introduce vstack, a five-stage formal verification workflow for coding agents, and veriserve, a prototype verifiable L L M serving engine with PD disaggregation built around a formally verified KV-page lifecycle design model. Applying vstack to production Prefill-Decode serving uncovered three classes of real RDMA and cache concurrency bugs across SGLang and vLLM, resulting in six proposed upstream pull requests. Veriserve backs its PdLedger design model with a two hundred twenty six-module, five hundred sixteen thousand nine hundred seventy three-obligation TLAPS safety proof while adding only zero point one eight percent mean p50 TPOT overhead on NVIDIA GB200. Over the past three years, inference engines like vLLM and SGLang have evolved from single-node executors into asynchronous distributed systems, layering paged KV memory, shared prefix caching, and disaggregated prefill-decode RDMA transfers.
Sources in this cluster
- Towards Verifiable LLM Infra: Agent-Driven Proof of Agent-Implemented Inference
Hacker News · 12 Nov 2026, 06:58 AEST