Can AI Judge You? An Empirical Audit of Deployment Readiness in LLM-assisted Hiring Systems
arXiv cs.AI · 16 Nov 2026, 03:16 AEST
A study examines the deployment readiness of large language models (LLMs) in hiring systems. It highlights the importance of operational and compliance readiness in distinguishing candidates based on qualifications.
Key facts
- Delegating resume screening to large language models (LLMs) requires organizations to establish whether they can distinguish candidates by relevant qualifications while allowing only authorized users to change screening rules or approve exceptions.
- We assess its operational readiness on four LLMs using a controlled experiment on 210 resumes across 14 job roles and six decision interfaces.
- We find that operational performance depends on the choice of evaluation interface: scalar scores generally distinguish qualification changes better than simple pass/fail judgments, while comparative judgments are sensitive to presentation order.
- Explicit permission rules and separation of resume content from instructions reduce unauthorized compliance, to a varying degree of effectiveness across LLMs and usage contexts.
Read aloud
A study examines the deployment readiness of large language models L L M s in hiring systems. It highlights the importance of operational and compliance readiness in distinguishing candidates based on qualifications. Delegating resume screening to large language models L L M s requires organizations to establish whether they can distinguish candidates by relevant qualifications while allowing only authorized users to change screening rules or approve exceptions. We assess its operational readiness on four L L M s using a controlled experiment on two hundred ten resumes across fourteen job roles and six decision interfaces. We find that operational performance depends on the choice of evaluation interface: scalar scores generally distinguish qualification changes better than simple pass/fail judgments, while comparative judgments are sensitive to presentation order. Explicit permission rules and separation of resume content from instructions reduce unauthorized compliance, to a varying degree of effectiveness across L L M s and usage contexts.
Sources in this cluster
- Can AI Judge You? An Empirical Audit of Deployment Readiness in LLM-assisted Hiring Systems
arXiv cs.AI · 16 Nov 2026, 03:16 AEST