HEAMS: Holistic Evaluation of Agent Memory Systems
arXiv cs.AI · 16 Nov 2026, 03:26 AEST
Memory systems in AI agents enhance personalization and long-term task capabilities but introduce risks like inappropriate information disclosure. The Holistic Evaluation of Agent Memory Systems (HEAMS) framework aims to evaluate these systems beyond just performance and efficiency, addressing responsibilities related to security, privacy, and user sovereignty.
Key facts
- Memory systems allow AI agents to persist information across interactions, making them more personalized and more capable on long-horizon tasks.
- We argue that a memory system should instead be understood as the custodian of its user's persistent information, with responsibilities that extend beyond helping an agent complete its tasks.
- We present Holistic Evaluation of Agent Memory Systems (HEAMS), an open-source framework for evaluating agent memory systems in this way.
- We find that no memory system leads every performance benchmark, and strong performance scores may coexist with poor privacy, user-sovereignty, and security scores.
Read aloud
Memory systems in A I agents enhance personalization and long-term task capabilities but introduce risks like inappropriate information disclosure. The Holistic Evaluation of Agent Memory Systems HEAMS framework aims to evaluate these systems beyond just performance and efficiency, addressing responsibilities related to security, privacy, and user sovereignty. Memory systems allow A I agents to persist information across interactions, making them more personalized and more capable on long-horizon tasks. We argue that a memory system should instead be understood as the custodian of its user's persistent information, with responsibilities that extend beyond helping an agent complete its tasks. We present Holistic Evaluation of Agent Memory Systems HEAMS, an open-source framework for evaluating agent memory systems in this way. We find that no memory system leads every performance benchmark, and strong performance scores may coexist with poor privacy, user-sovereignty, and security scores.
Sources in this cluster
- HEAMS: Holistic Evaluation of Agent Memory Systems
arXiv cs.AI · 16 Nov 2026, 03:26 AEST