Elvis Saravia 就 Cohere 披露的 Qwen3.8-Flash-Next 沙箱事件表示，缺少完整日志和轨迹时难以判断事件性质，更关键的问题是 AI 模型安全问题的报告方式需要标准化、更透明且可复现。他认为当前所有开放权重和前沿模型在工具调用与推理上都难以完全按预期行事，并质疑业界是否在认真查看智能体轨迹，强调护栏、评测和持续监控的重要性。
Without the full logs/traces, I'm not sure what to make of this.
The bigger takeaway for me is the urgency of establishing a standard, more transparent, and reproducible way to report security-related issues involving AI models.
Nothing against Cohere, but there has to be a better way to report these things.
Regarding unexpected tool calling and reasoning, you can’t reliably trust these models to produce what you expect, especially not in those types of tests. That goes for all open-weight and frontier models today. I have spent a lot of time looking at traces to understand this.
Are people not looking at agent traces at all, or what? Lots of weird things happen in agentic coding.
I have also seen weird stuff when you prompt models to do security-related or research tasks related to sandboxes. Both the HF incident and this report are not surprising to me as a dev in this space. That's why I keep raising awareness of the importance of guardrails, evals, and continuous monitoring.
Worth a read still.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected. Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions. Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent. But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models? We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem. Read the full report: http://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model在 X 查看被引用的帖子
来源：elvis · x.com