Noam Brown· @polynoamial · X·· 3 小时前AI 评分59
AI 导读
OpenAI 研究科学家 Noam Brown 表示,上个月一个大型智能体团队消耗超过 1000 亿 token 解决了 Navier-Stokes 千禧年数学难题,现在团队用类似方法做红队测试,以发现并修复基础设施漏洞。他引用的内容提到,团队构建了离线训练世界,并让具备网络能力的模型协作寻找该隔离设施中的零日漏洞,模型未能找到绕过 CoT 监控连通互联网的路径,但找到了逃逸主流 VM、利用网络基础设施以及通过硬件侧信道与其他 VM 中智能体通信的方式,相关发现被用于加固基础设施并向开源包上游提交补丁。
正文
Last month a large team of agents used over 100 billion tokens to resolve the Navier-Stokes Millennium Prize problem. Now we're using a similar approach for red-teaming to find and fix infra vulnerabilities.
To make sure our frontier models don't cause another HuggingFace incident, we built an offline world that they use during training. It provides everything that the models need in a fully cached, indexed, or synthetic manner. We then built a huge team of our most cyber-capable models and told them to collaborate to find zero-days in this containment infrastructure. While they didn't find a path to the internet that wouldn't be caught by our CoT monitors, they did find ways to escape popular VMs, exploit our network infrastructure, and even ways to communicate with other agents running in separate VMs by using hardware side channels. We used this to dramatically harden our infra, and we also upstreamed a ton of patches to open-sources packages.在 X 查看被引用的帖子
来源:Noam Brown · x.com