RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
arXiv cs.AI · 10 Oct 2026, 03:38 AEST
RATED is a new framework designed to improve the reliability of AI agent tool retrieval by combining semantic matching with dynamic reliability adaptation. It significantly enhances performance in real-world scenarios, reducing fault selection rates during API outages.
Key facts
- Existing methods optimize semantic relevance but largely ignore tool availability during execution.
- RATED employs a frozen dual-encoder with contrastive fine-tuning for semantic matching, and a lightweight contextual bandit module that re-ranks candidates based on real-time execution feedback.
- Experimental results demonstrate that RATED achieves a 52.2% relative MRR improvement over the ToolLLM API retriever on real-world queries.
- RATED reduces fault selection rate from 86.2% to 2.9% under severe API outage scenarios with a 30% failure rate, while incurring negligible latency overhead of less than 1 ms over encode-only inference.
Read aloud
RATED is a new framework designed to improve the reliability of A I agent tool retrieval by combining semantic matching with dynamic reliability adaptation. It significantly enhances performance in real-world scenarios, reducing fault selection rates during A P I outages. Existing methods optimize semantic relevance but largely ignore tool availability during execution. RATED employs a frozen dual-encoder with contrastive fine-tuning for semantic matching, and a lightweight contextual bandit module that re-ranks candidates based on real-time execution feedback. Experimental results demonstrate that RATED achieves a fifty two point two percent relative MRR improvement over the ToolLLM A P I retriever on real-world queries. RATED reduces fault selection rate from eighty six point two percent to two point nine percent under severe A P I outage scenarios with a thirty percent failure rate, while incurring negligible latency overhead of less than one ms over encode-only inference.
Sources in this cluster
- RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
arXiv cs.AI · 10 Oct 2026, 03:38 AEST