Patient-specific rubric-based evaluation of large language models across longitudinal cancer care
arXiv cs.AI · 27 Oct 2026, 03:56 AEST
Large language models (LLMs) show promise in specialty medicine, yet existing evaluations largely focus on isolated clinical encounters. The introduction of NPC-RubricBench highlights the need for phase-adaptive safeguards in multistage medical AI.
Key facts
- Here we introduce NPC-RubricBench, a phase-aware benchmark for evaluating multistage clinical reasoning across the cancer care continuum, using nasopharyngeal carcinoma as a defined disease context.
- Evaluating 30 general-purpose and medical LLMs revealed that strong diagnostic performance does not carry over to downstream care.
- Explicit decision errors dominated anatomical staging at 48.2%, whereas decision omissions dominated initial work-up and post-treatment follow-up at 33.4% and 30.4%, respectively.
- Moreover, varying query granularity exposes a critical specification trade-off: broad open-ended requests degrade Decision Quality across all 30 models, while more detailed, checklist-like prompts raise Decision Quality but reduce Evidence Faithfulness and Safety Adherence.
Read aloud
Large language models L L M s show promise in specialty medicine, yet existing evaluations largely focus on isolated clinical encounters. The introduction of NPC-RubricBench highlights the need for phase-adaptive safeguards in multistage medical A I. Here we introduce NPC-RubricBench, a phase-aware benchmark for evaluating multistage clinical reasoning across the cancer care continuum, using nasopharyngeal carcinoma as a defined disease context. Evaluating thirty general-purpose and medical L L M s revealed that strong diagnostic performance does not carry over to downstream care. Explicit decision errors dominated anatomical staging at forty eight point two percent, whereas decision omissions dominated initial work-up and post-treatment follow-up at thirty three point four percent and thirty point four percent, respectively. Moreover, varying query granularity exposes a critical specification trade-off: broad open-ended requests degrade Decision Quality across all thirty models, while more detailed, checklist-like prompts raise Decision Quality but reduce Evidence Faithfulness and Safety Adherence.
Sources in this cluster
- Patient-specific rubric-based evaluation of large language models across longitudinal cancer care
arXiv cs.AI · 27 Oct 2026, 03:56 AEST