在 LLM 智能体写入动作前加 conformal 风险门控，tau-bench 实测边界成立但未胜过全量确认
I put a conformal risk control gate between an LLM agent and its write actions on tau-bench. The bound held. The gate didn't beat asking every time [R]
作者在 LLM 智能体与写入动作之间加入基于 conformal risk control 的风险门控，在 3,340 条 tau-bench 零售对话上以 glm-5.3-flash 作为智能体和用户进行实测，最严格设置下 1.7% 的对话执行了错误写入而未询问，目标为 2%，但置信区间达到 3.7%。
![I put a conformal risk control gate between an LLM agent and its write actions on tau-bench. The bound held. The gate didn't beat asking every time [R]](/api/img-proxy?u=https%3A%2F%2Fpreview.redd.it%2Fyto3hlnsxlvh1.png%3Fwidth%3D140%26height%3D100%26auto%3Dwebp%26s%3D68f4fc97524ac99eaffec70a46cfebc0b63f3137&mode=full&exp=1792281600&sig=953fadf004652696 "I put a conformal risk control gate between an LLM agent and its write actions on tau-bench. The bound held. The gate didn't beat asking every time [R]") | My agent said it was 100% sure. The user asked it to change two items in an order. It changed one. So I built a gate that stops an agent before every write and asks a human when it looks risky. Confidence signals, calibration, conformal risk control for the threshold. The ToolChain-CRC and CORA idea (arXiv 2606.18467, 2604.09155), built open and actually measured. 3,340 tau-bench retail conversations, glm-5.3-flash as agent and user. The risk bound held. 1.7% of conversations executed a wrong write without asking at the strictest setting, target was 2%. CI reaches 3.7%, so grain of salt. tradeoff curve, wrong writes executed vs approvals requested That was the only win. At looser settings the gate never asks, so it's a fixed 0.7 cutoff with extra steps. At the strict one it asks constantly and avoids 0.058 wrong writes per approval, against 0.062 for just confirming everything. Same within noise. Confirming everything gets you to zero mistakes. The gate can't, at any setting. The signals are garbage. The model said 90%+ confidence on 548 of 549 write steps, all 43 wrong ones included. Its confidence cannot tell right from wrong, so the calibrator used the only feature that could, how far into the conversation the step is. Step position did all the work. Code, data, negative results. https://github.com/Gjusev/gate-calibrated submitted by /u/Gjuesev[link] [留言] |
来源：r/MachineLearning · reddit.com