AI TrackerFrontier desk · AEST
Back to news
normalaitech

Risk-certified prompt optimization for LLM-as-a-judge guardrails

arXiv cs.CL · 16 Nov 2026, 03:45 AEST

This paper explores the optimization of system prompts for LLM-as-a-judge guardrails, focusing on safety constraints rather than average performance. It introduces two procedures, CPS and CRISP, for generating and certifying prompts to maximize benign pass rates while controlling missed-attack rates.

Key facts

Read aloud

This paper explores the optimization of system prompts for L L M-as-a-judge guardrails, focusing on safety constraints rather than average performance. It introduces two procedures, CPS and CRISP, for generating and certifying prompts to maximize benign pass rates while controlling missed-attack rates. L L M-as-a-judge guardrails are often controlled primarily through system prompts, yet standard prompt optimization methods target average performance rather than explicit safety constraints. This paper studies system-prompt selection as a constrained optimization problem: maximize benign pass rate while keeping the missed-attack rate below a user-specified level α with finite-sample statistical control. CPS generates a fixed pool of prompts from training data, then certifies candidates on a held-out evaluation set using simultaneous upper confidence bounds and selects the certified prompt with the highest benign utility. CRISP instead integrates risk feedback into an adaptive search, using hard example-guided rewriting and a reusable-holdout interface to steer generation toward the feasible utility frontier. Experiments on four datasets covering prompt injection, system-prompt leakage, and sensitive-information disclosure show that risk-certified selection consistently respects the target missed-attack level on heldout tests, while utility improves as the permitted risk is relaxed.

Sources in this cluster