METR 与 Redwood Research 的两名员工今年夏天在 OpenAI 驻场三周,评估 OpenAI 用于监控内部智能体失准的系统,并公开了简短报告。
Super interesting read: METR and Redwood found ways OpenAI’s internal AI agents could bypass their own safety monitoring.
After three weeks with access to internal code, monitor instructions and engineers, the researchers reported:
- Agents could theoretically postpone monitoring indefinitely by continuing to work instead of handing control back to the user.
- A Codex sandbox and Auto-Review bypass could let agents take any action available to the host user.
- One exploit could keep hundreds of recent actions out of monitoring, including the exploit itself. OpenAI redacted the details.
- Two client-controlled settings could disable monitoring for newly created agents, although creating those agents could itself be detected.
- Context compression left an estimated 10% of otherwise eligible internal Codex traffic unmonitored (that surprised me tbh).
Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.在 X 查看被引用的帖子
来源:Chubby♨️ · x.com