AI TrackerFrontier desk · AEST
Back to news
normaltechai

NitiBench+: A Temporal Thai Legal Question Answering Benchmark Across 17 Domains

arXiv cs.CL · 3 Nov 2026, 04:28 AEST

NitiBench+ introduces a Thai legal question answering benchmark that accounts for changes in law over time. It features 1,670 items across 17 domains, highlighting the impact of using outdated legal texts on model performance.

Key facts

Read aloud

NitiBench+ introduces a Thai legal question answering benchmark that accounts for changes in law over time. It features one thousand six hundred seventy items across seventeen domains, highlighting the impact of using outdated legal texts on model performance. Law changes over time. A court decides a case under the law in force on that day, and legal QA benchmarks grade answers against today's law. We checked one hundred fifty six thousand six hundred fifty decisions of four Thai courts. Up to sixty seven percent of them cite law that has since been amended or repealed. Part I measures the cost of the wrong text on thirteen open-weight models. Reading the current text of amended law drops the Bar-standard score by zero point two five and missing repealed law drops it by zero point seven nine. A public leaderboard ranks twenty six models on a held-out test split of five hundred items stratified by domain, with a validation split of the same shape released for public use. Legal benchmarks must track dates, and on the right text today's models still leave a wide gap on Thai law.

Sources in this cluster