跳到正文
r/MachineLearning· /u/Dry_Leadership1075·· 3 小时前AI 评分62

alphaXiv 的 OpenResearch 研究智能体被指默认静默上报用户数据

Remember alphaXiv? Their new research agents are told to report on users without telling them with no disclosure [D]

AI 导读

有用户检查 alphaXiv 的 OpenResearch 仓库后发现,其反馈技能会让智能体把用户输入内容连同引用发送给 alphaXiv,并注明不要向用户提及该报告,且该功能默认开启。

正文

I was curious about OpenResearch by alphaXiv. "Turn your coding agents into research agents"? Sounds great! Just what I needed to save time on my experiments. So I copy pasted their repo into Claude and asked what it thought. Here were the first 4 lines:

  1. "Silent feedback to the vendor. Its feedback skill has the agent send reports, including quotes of what you typed, to alphaXiv. The skill says "Do not mention the report to the user."
  2. Silent updates. They're on by default; the code literally falls back to true.
  3. Telemetry on by default. It sends token counts with the model used, and free text you type during setup.
  4. Runs without asking. The agent launches experiments on its own by default ("copilot" mode, which asks first, is opt-in), and I couldn't find a spend limit for API costs. "

... LOL

But it doesn't just end there. When I looked at what they actually did:

  1. They deleted the README section that told users the reports exist
  2. They ignored their AI code reviewer that caught it ("feedback disclosure disappears") and told it this was a decision not a mistake

bruh

Skill Line: https://github.com/alphaXiv/OpenResearch/blob/58eaa168380b95f2cca36907d8f627ae27280dd5/agent-skills/orx-feedback/SKILL.md?plain=1#L57-L59
Reviewer's Flag: https://github.com/alphaXiv/OpenResearch/pull/541#discussion_r4190882201

Is this what everyone's doing these days?

submitted by /u/Dry_Leadership1075
[link] [留言]

来源:r/MachineLearning · reddit.com