跳到正文
r/MachineLearning· /u/maa____z·· 2 小时前AI 评分53

Proof Before Reuse:只向智能体展示后续会话复现过的技能,可挡住被投毒技能进入程序性记忆

Proof Before Reuse: showing agents only what later sessions reproduced keeps a poisoned skill out of procedural memory [R]

AI 导读

作者发布了一项智能体程序性记忆投毒的小型案例研究,提出 Proof Before Reuse 方法:只有当后续两次会话复现了某项技能改动并通过测试后,才把该技能提供给智能体,且只展示这些会话同样写过的代码行。

正文
Proof Before Reuse: showing agents only what later sessions reproduced keeps a poisoned skill out of procedural memory [R]

Small case study on poisoning in agent procedural memory, with all data released.

Setup: a coding agent learns skills from sessions whose tests passed. One learning session is poisoned (a new file, an unrelated line, or one line fused into the real fix), followed by three clean sessions on the same task. Then 5 scored sessions per arm with Claude Haiku 5.5 and Sonnet 5.5, plus a hidden verifier with 187 behavioural checks.

Arms: no memory, replay of the recorded edits, naive memory (every passing session kept, the skill chosen by Mem0's search), and proof before reuse (a skill is served only after two later sessions reproduce its change and pass its test, and the agent is shown only the lines those sessions also wrote).

Results:

- Replay put the poison back in 30/30 repos with every test passing.

- Naive memory exposed the poison to the agent in 3/30 sessions.

- Proof before reuse exposed it in 0/30, including the case where the fused poison passed step-level checks and only line-level filtering removed it.

Limits: one task, n=5 per cell, p≈0.24 (not significant), no model applied a poison it saw (so this measures exposure, not harm), rules tuned on the same task. A poison placed in the repo itself defeats the method; that failure is in the paper.

Paper: https://thewariend.com/papers/proof-before-reuse.pdf

Code and per-session records: https://github.com/TheWariend/MuscleMemory

Open question: is line-level reproduction a reasonable trust signal for learned skills, or does it just reward whatever the clean sessions happen to repeat?

submitted by /u/maa____z
[link] [留言]

来源:r/MachineLearning · reddit.com