SAUCE: Summary Analysis Using Conversation Entailment
arXiv cs.CL · 5 Nov 2026, 03:25 AEST
SAUCE is a reference-free, fact-based evaluation pipeline designed for cross-lingual conversational speech summarization. It measures summary accuracy and fact coverage through entailment between conversation and text, providing an explainable alternative to traditional LLM evaluators.
Key facts
- With the growing need for evaluating Large Language Models (LLMs) and their applications to speech, challenges persist in summarizing and evaluating conversations that lack a clear end goal.
- We introduce SAUCE -- a reference-free, fact-based evaluation pipeline for cross-lingual conversational speech summarization.
- Crucially, unlike black-box LLM evaluators or dense embedding metrics, SAUCE is inherently explainable: it maps summary scores to discrete, verifiable facts, allowing users to pinpoint exact hallucinations or omissions.
- Preliminary investigations show SAUCE strongly align with human judgment.
Read aloud
SAUCE is a reference-free, fact-based evaluation pipeline designed for cross-lingual conversational speech summarization. It measures summary accuracy and fact coverage through entailment between conversation and text, providing an explainable alternative to traditional L L M evaluators. With the growing need for evaluating Large Language Models L L M s and their applications to speech, challenges persist in summarizing and evaluating conversations that lack a clear end goal. We introduce SAUCE -- a reference-free, fact-based evaluation pipeline for cross-lingual conversational speech summarization. Crucially, unlike black-box L L M evaluators or dense embedding metrics, SAUCE is inherently explainable: it maps summary scores to discrete, verifiable facts, allowing users to pinpoint exact hallucinations or omissions. Preliminary investigations show SAUCE strongly align with human judgment.
Sources in this cluster
- SAUCE: Summary Analysis Using Conversation Entailment
arXiv cs.CL · 5 Nov 2026, 03:25 AEST