Can JEV Support Clinical Trial Criteria Assessment?
arXiv cs.CL · 27 Oct 2026, 02:41 AEST
A study compares JEV, a non-generative decision model, with Qwen Flash in clinical trial prescreening. JEV demonstrated superior performance in certain datasets, particularly in identifying conditions with insufficient information.
Key facts
- Given patient records, criterion questions, and predefined answer options, we retrospectively compared JEV, a non-generative decision model returning native option probabilities, with schema-constrained Qwen Flash across 2,048 test patient-condition pairs from TrialGPT and question-answer datasets derived from MIMIC-III and MIMIC-IV.
- In batch mode, JEV and Qwen achieved similar macro-F1 scores on TrialGPT (0.652 vs. 0.660) and MIMIC-IV (0.807 vs. 0.814), whereas JEV substantially outperformed Qwen on MIMIC-III (0.768 vs. 0.534; $Δ$ = +0.234, 95% CI: 0.187-0.281).
- This advantage largely reflected higher recall for conditions labeled as having insufficient information (56.3% vs. 9.9%).
- Across all three datasets, JEV also had lower observed median job latency (1.18-1.40 vs. 2.80-8.90 s) and 53%-74% lower estimated cost per condition.
- JEV's observed errors in interpreting clinical evidence underscore the need to verify its outputs against patient records before they inform eligibility decisions.
Read aloud
A study compares JEV, a non-generative decision model, with Qwen Flash in clinical trial prescreening. JEV demonstrated superior performance in certain datasets, particularly in identifying conditions with insufficient information. Given patient records, criterion questions, and predefined answer options, we retrospectively compared JEV, a non-generative decision model returning native option probabilities, with schema-constrained Qwen Flash across two thousand forty eight test patient-condition pairs from TrialGPT and question-answer datasets derived from MIMIC-III and MIMIC-IV. In batch mode, JEV and Qwen achieved similar macro-F1 scores on TrialGPT zero point six five two vs. zero point six six zero and MIMIC-IV zero point eight zero seven vs. zero point eight one four, whereas JEV substantially outperformed Qwen on MIMIC-III zero point seven six eight vs. zero point five three four; $Δ$ = +zero point two three four, ninety five percent CI: zero point one eight seven-zero point two eight one. This advantage largely reflected higher recall for conditions labeled as having insufficient information fifty six point three percent vs. nine point nine percent. Across all three datasets, JEV also had lower observed median job latency one point one eight-one point four zero vs. two point eight zero-eight point nine zero s and fifty three percent-seventy four percent lower estimated cost per condition. JEV's observed errors in interpreting clinical evidence underscore the need to verify its outputs against patient records before they inform eligibility decisions.
Sources in this cluster
- Can JEV Support Clinical Trial Criteria Assessment?
arXiv cs.CL · 27 Oct 2026, 02:41 AEST