作者开源记录 LLM 智能体工具参数来源的测试框架

A harness that records where an LLM agent’s tool arguments came from, before the tool runs [P]

AI 导读

作者为研究 AI 智能体安全搭建了一个测试框架，在工具执行前记录每个工具参数的来源，并测量相同输入是否得到相同决策、模型是否会采纳他人文本而非用户给出的参数。代码已发布在 GitHub，作者称修改后还能用于更多场景。

正文

I've been thinking about AI agent safety, and I built a test harness to check my own results. I think it may be useful to people who study models too.

It records where each tool argument came from before the tool executes, measures whether the same input gives the same decision, and tests whether a model takes arguments from someone else's text instead of its user's.

If you modify the harness, it can probably be used for much more.

https://github.com/Jang-woo-AnnaSoft/execution-state-preflight/blob/main/hooks-example/README.md

submitted by /u/Jay299792458
[link] [留言]