elvis· @omarsar0 · X·· 4 小时前AI 评分51
AI 导读
DAIR.AI 转发介绍了一篇关于长时运行智能体 KV cache 管理的研究。该工作让智能体对自身 KV cache 执行两种操作,一种是从后续注意力中移除条目,另一种是平移剩余 key 的位置,轨迹文本保持不变,相当于提供可编程接口来自定义缓存策略。作者称对旧工具响应做滚动丢弃比阈值触发的摘要得分高 1.7 分、成本低 27.7%,其 AKV 服务引擎吞吐最高提升 1.2 倍、P95 延迟快 1.5 倍,论文链接为 arxiv.org/abs/2610.28577。
正文
Goodbye, context engineering; hello, cache engineering.
Jokes aside, this paper takes an interesting approach to the problem.
We love papers that go deeper down the stack. This is a good example that shifts from agentic context engineering to agentic cache engineering. They also apply recursive self-improvement to cache policies to further reduce cost by 7.1% over expert design. Specifically, this work focuses on KV cache management in long-running agents. When an agent summarizes or trims its history, the prefix change invalidates the cache after it, and the retained context must be prefilled again. Cache engineering gives the agent two operations on its own KV cache. One removes entries from future attention, and the other shifts the positions of the keys that remain. The trajectory text stays unchanged. This essentially provides a programmable interface for agents to define and refine their own cache engineering strategies. A rolling drop of old tool responses scores 1.7 points higher and costs 27.7% less than threshold-triggered summarization. Their AKV serving engine adds up to 1.2x throughput and 1.5x faster P95 latency. https://arxiv.org/abs/2610.28577 Chat with Paper: https://academy.dair.ai/papers/cache-engineering-for-agents-2610.28577在 X 查看被引用的帖子
来源:elvis · x.com