// Latent Semantic Memory for LLMs //

This paper presents some really interesting findings about agent memory.

They find that sixteen memory tokens beat full-context inference on FactKG while using 108x fewer tokens for the source documents.

Approach:

Documents are encoded once into reusable embeddings. For each query, a learned module gathers the relevant information from the whole collection and writes it into 16 tokens in the input space of a frozen LLM.

Results:

It beats Mem0, MemoRAG, and other baselines on five of eight benchmarks. On FactKG, it scores 84.33% against 80.43% with the full context.

It also works as agent memory. In TextWorld, using the agent's own history as the collection raises success from 18.6% to 24.2%, and the agent revisits fewer rooms.

It also seems that it could help frozen LLMs accumulate and use knowledge effectively across long-context interactions. This is applicable to building continually learning AI systems.

https://arxiv.org/abs/2610.21375

Chat with Paper: https://academy.dair.ai/papers/memorilla-latent-semantic-memory-for-llms-2610.21375