When to Assist and What to Suggest: Joint Gating and Memory-Grounded Hint Generation in One Compact Decoder
arXiv cs.AI · 30 Oct 2026, 03:54 AEST
Proactive assistants can provide memory-grounded hints during conversations by determining when to assist and what to suggest. A new compact decoder model shows improved efficiency and performance compared to traditional systems.
Key facts
- Current systems split this between a gate and a larger generator; we evaluate both together on 28,058 decision points.
- A gate pays off through precision and compute, whose saving falls from 4.39x with an 8B generator to 1.38x with a 1B one.
- The memory profile drives hint quality: removing it costs 24.7 exact-match points (rescaled BERTScore 0.57 to 0.35), while retrieval and a larger generator add nothing on exact match.
- At its final checkpoint it matches or exceeds the two-model pipelines on automatic end-to-end measures with 1.31-1.76x less wall-clock time than the matched 1B pipeline.
Read aloud
Proactive assistants can provide memory-grounded hints during conversations by determining when to assist and what to suggest. A new compact decoder model shows improved efficiency and performance compared to traditional systems. Current systems split this between a gate and a larger generator; we evaluate both together on twenty eight thousand fifty eight decision points. A gate pays off through precision and compute, whose saving falls from four.39x with an 8B generator to one.38x with a 1B one. The memory profile drives hint quality: removing it costs twenty four point seven exact-match points rescaled BERTScore zero point five seven to zero point three five, while retrieval and a larger generator add nothing on exact match. At its final checkpoint it matches or exceeds the two-model pipelines on automatic end-to-end measures with one point three one-one.76x less wall-clock time than the matched 1B pipeline.
Sources in this cluster
- When to Assist and What to Suggest: Joint Gating and Memory-Grounded Hint Generation in One Compact Decoder
arXiv cs.AI · 30 Oct 2026, 03:54 AEST