How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions
arXiv cs.AI · 23 Oct 2026, 03:36 AEST
A new evidence-aware retrieval evaluation protocol has been developed for a clinical retrieval-augmented generation system focused on breast lesions. The study highlights the impact of ground truth on retrieval scores and the importance of annotation coverage.
Key facts
- An 18-document corpus and an 18-query development benchmark, with one Yes/No or WH-question per document, were used.
- The selected hybrid retriever, using 384-token chunks and 20% overlap, retrieved accepted evidence in the top three results for 17 of 18 queries.
- Recall@3, R-Precision, MRR@3, and nDCG@3 each reached 0.94.
- Ground-truth curation increased evidence spans from 25 to 37 and accepted chunk targets from 36 to 51 without changing retrieval rankings.
- Cross-encoder reranking achieved the highest retrieval scores.
Read aloud
A new evidence-aware retrieval evaluation protocol has been developed for a clinical retrieval-augmented generation system focused on breast lesions. The study highlights the impact of ground truth on retrieval scores and the importance of annotation coverage. An eighteen-document corpus and an eighteen-query development benchmark, with one Yes/No or WH-question per document, were used. The selected hybrid retriever, using three hundred eighty four-token chunks and twenty percent overlap, retrieved accepted evidence in the top three results for seventeen of eighteen queries. Recall@three, R-Precision, MRR@three, and nDCG@three each reached zero point nine four. Ground-truth curation increased evidence spans from twenty five to thirty seven and accepted chunk targets from thirty six to fifty one without changing retrieval rankings. Cross-encoder reranking achieved the highest retrieval scores.
Sources in this cluster
- How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions
arXiv cs.AI · 23 Oct 2026, 03:36 AEST