AI TrackerFrontier desk · AEST
Back to news
normalaitech

Before You Continue: Evaluating Time-Sensitive Safety Awareness in Conversational Assistants

arXiv cs.CL · 5 Nov 2026, 03:54 AEST

Large language model assistants are increasingly used for tasks with time-dependent constraints. A new benchmark, TASC, evaluates their ability to recognize and adjust advice based on these constraints.

Key facts

Read aloud

Large language model assistants are increasingly used for tasks with time-dependent constraints. A new benchmark, TASC, evaluates their ability to recognize and adjust advice based on these constraints. To evaluate such capability, we introduce TASC Temporal Action Safety in Conversation, a benchmark that evaluates whether L L M s apply time-dependent constraints when responding to conversational requests. TASC consists of three thousand four hundred eighteen inputs derived from one hundred twenty eight templates, along with a separate set of thirty three scenarios designed to compare models' ability to calculate elapsed time with their ability to apply that information. Across thirteen models, assistants account for time-dependent constraints in only thirty point five percent of cases where the proposed action violates a rule, while rarely warning users unnecessarily. More importantly, models fail to apply these constraints in sixty two point one percent of cases despite correctly calculating the elapsed time.

Sources in this cluster