DeepSeek 开源推理框架 DeepInfer，单集群日处理 155 万亿 token
[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
DeepSeek 开源生产推理框架 DeepInfer，称单集群日处理 155 万亿 token、输入缓存命中率 96%、非高峰时段每加速器小时收入 38 美元。
材料把 DeepInfer 的架构取舍、缓存分层与 vLLM 复现结果放在一起，便于判断开源推理栈的真实边界。
译文尚不完整，完整内容请切换到原文。
由 SerpAPI 自豪呈现：今天的发布内容涵盖面很广，但你可以把这一切都抓取成供你的 agent 使用的 markdown！token 成本仅为 JSON 的 50%。
我们上次在四月报道了推理拐点，而从历史上看，大多数进展都来自闭源的十角兽公司，从Fireworks 到 Baseten，再到 Modal。
那么，是时候给出开源回应了。今天是 PyTorch Conf 的第一天，而 DeepSeek 通过开源 DeepInfer 向讨论中发射了一枚洲际弹道导弹——这是他们的生产栈，能够实现最长 72 小时的提示缓存，而 OpenAI 刚刚宣布的是 12 小时：
这是他们对 vLLM 的定制版本，并致谢 SGLang 和 TRRTLLM，但新增了很多内容……除了他们的皇冠明珠——他们超优化的 KV 缓存（紧凑 + 超过 8 小时生命周期）、PD/分离，以及负载均衡（他们的 3FS 此前已经开源）。我们翻译了他们的博文，并在此发布，同时附上完整的研究指南、解释文章、代码地图和子系统指南：
正如我们在他们的补充披露中指出的，他们声称每天服务 155T tokens，这使他们在非高峰价格下每张 SM100/A950 卡每小时获得 38 美元（目前每百万输入/输出 tokens 为 0.15/0.60 美元：155000000 tokens * 0.14 美元混合成本 / 24 小时约为每小时 100 万美元）。采用他们的生产数据（我们已翻译）：每块 prefill GPU 每秒 150 万含缓存输入 tokens，96% 输入缓存命中率，每块 decode GPU 每秒 14,000 输出 tokens（约 2400 块 GPU）：
在每卡每小时 5-8 美元的全包成本下，这意味着超过 80% 的推理利润率！
做一些集群计算，这大约是每 GW 1660 亿美元，推理利润率的回本周期为 81 天（DeepSeek 以计划 10 个月回本而闻名）。想象一下这头鲸鱼拥有更多 GW 能做什么。
2026/10/19-2026/10/20 的 AI 新闻。我们检查了 12 个 subreddit、544 个 Twitter，没有更多 Discord。AINews 网站让你搜索所有过往期刊。提醒一下，AINews 现在是 Latent Space 的一个栏目。你可以选择开启/关闭邮件频率！
AI Twitter 回顾
头条：DeepSeek DeepInfer——生产推理、开源边界与缓存 tokens 的经济学
发生了什么
DeepSeek 发布了 DeepInfer，这是 V4.1 Flash 背后的推理引擎，同时公布了生产服务细节，使得这份工程报告与代码同样重要。
此次发布涵盖推理引擎和 API 处理层。DeepInfer 支持分布式推理、投机解码、SWA Bounded Replay，以及在 NVIDIA 和华为昇腾平台上的结构化输出。配套的 deepseek-recipe 组件实现了生产级 token 编码/解码和兼容官方 API 的处理。DeepSeek 表示，未来的模型推理实现和优化也将出现在这个仓库中。@deepseek_ai、@deepseek_ai、公告翻译
所报告的生产数据相当可观——但描述的是一个高度依赖缓存的服务。根据所提供帖子中的总结和引用，DeepSeek 的工程报告称单个集群每天处理 155 万亿 tokens、96% 的输入缓存命中率、平均首 token 时间 400 毫秒、生成速度 230 tokens/s，编码场景达到300 tokens/s。报告还称在非高峰价格下每加速器小时收入 38 美元。这些是第一方生产声明，并非经独立审计的测量结果。@ZhihuFrontier、@swyx
“完全开源”并不意味着整个生产服务可以凭此次发布复现。尽管 DeepSeek 将 DeepInfer 描述为完全开源，但技术说明称KV 缓存管理和负载均衡组件尚未完全发布，因为它们仍与内部基础设施耦合。引擎是开源的；重要的服务级编排仍属于部署方的责任。@deepseek_ai、@ZhihuFrontier
vLLM 给出了最具体的对比回应。在承认 DeepInfer 最初主要基于 vLLM 构建并以 Apache 2.0 发布之后，vLLM 团队报告称，在匹配配置下于 GB200 上，其解码吞吐量与 DeepInfer 公布结果相差在 6% 以内。该团队还发布了 AgentX 结果和复现方法，实质性地质疑了 DeepSeek 的效率对现有开源引擎不可企及的说法。@vllm_project、@vllm_project、复现资源
此次发布引发了两场不同的争论：DeepSeek 是否拥有独一无二的强大服务栈，以及其披露信息是否更普遍地证明了高推理盈利能力。第一个问题现在已有对比基准证据；第二个问题仍受工作负载构成、缓存核算、硬件成本以及计入哪些费用的影响。@Tim_Dettmers、@zephyr_z9、@Yuchenj_UW
架构：分离阶段，调度计算而非 token
DeepInfer 披露的架构将视觉编码、预填充和解码分离，使每个阶段都能针对自身瓶颈进行扩展。推荐的部署形态是预填充使用普通八加速器机器，解码使用 64–128 加速器超级节点。解码采用宽专家并行来分布 MoE 专家，同时保持较大的有效批次。这些是部署建议，不足以证明 DeepSeek 生产集群的确切硬件构成。@ZhihuFrontier
调度细节解释了为什么传统的“每批 token 数”比较可能遗漏重大差异：
预填充调度会估算注意力计算成本。工作分配基于估算的计算量，而非仅依据原始请求长度。块大小会变化以适配时间预算，而不是对异构请求强制使用单一固定块大小。@ZhihuFrontier
Prefill 路由不需要缓存亲和性。由于缓存前缀被池化在共享存储中，请求可以发往任意可用 worker，而不必发往此前服务其前缀的那个 worker。在一份详细的中文分析中，推理工程师 Su Jin Yan 将此视为一项特别有用的生产设计：prefill 数据并行组按“计算 token”进行均衡，而共享存储提供可复用的状态。同一分析还强调了请求分桶和带运行时估算器的 FIFO 调度。@SuJinYan123
Decode 路由同时平衡运行中请求数和 KV 缓存长度。请求数近似反映调度压力；累计 KV 长度则捕捉了仅靠请求数无法体现的内存/状态压力。该工程评论将 prefill/decode 扩缩信号描述为：prefill 使用聚合计算负载，decode 使用运行中请求数。@ZhihuFrontier、@SuJinYan123
专家并行负载均衡会复制热门专家。副本会被定期重新分配，以减少路由倾斜导致的设备间等待。代价是额外消耗 HBM 以换取更高吞吐，且随着复制增加收益递减；有效的复制因子因工作负载而异。@ZhihuFrontier、@SuJinYan123
CPU 调度与加速器执行重叠。异步调度消除了 decode 步骤之间的空闲间隙，而不是把主机侧编排视为免费。@ZhihuFrontier
一个在技术上值得注意的观察是，所披露的 Blackwell 配置中似乎没有使用张量并行。一位评论者将此与 vLLM/SGLang 的默认方案进行对比，并将这一做法追溯到 DeepSeek V3 在 H800 时代的互连限制。这是对该配置的一种解读——并非张量并行普遍更差的证据。@nrehiew_
规模同样重要。Su Jin Yan 认为，大规模专家并行 prefill 需要足够的流量和 worker 才能划算，并且对于只有少量 prefill worker 的中小型部署，更倾向于流水线并行。作者还指出，层间异构性会使流水线划分变得复杂。这是实践者的判断，而非确立通用交叉点的基准测试。@SuJinYan123
缓存层级：持久前缀与瞬态状态的廉价重建
缓存设计是吞吐数据、agent 工作负载与所报告的单加速器收入之间的核心联系。
持久块 KV 存放在独立的 3FS 存储集群中，SSD 容量约为 100 PiB。报告的保留期为agent 流量三天、chatbot 流量七天。任何 prefill worker 都可以检索缓存前缀，从而将缓存局部性与计算放置解耦，并使存储和计算能够独立扩展。@ZhihuFrontier
这种保留机制对于需要反复重发不断增长的历史记录的智能体很有用，这些历史记录包括指令、仓库内容、工具输出和之前的响应。这也意味着每天 155T 的服务 token 不能解读为每天新生成 155T token：缓存命中属于服务 token 统计的一部分，而新计算的 prefill 和自回归输出带来的成本则截然不同。因此，所报告的 96% 输入缓存命中率是理解这一头条吞吐量的关键背景。@ZhihuFrontier，经济工作负载假设
该实现针对两种不同的状态生命周期：
持久块缓存以异步方式加载。主机到设备的传输与层执行重叠进行。Su Jin Yan 强调了一个更底层的实现细节：细粒度 KV 块拷贝通过 CUDA 拷贝引擎执行效果不佳，因此所描述的路径使用基于 GDR/RDMA 的传输，目标是在不占用 SM 的情况下跑满 PCIe 带宽。这是从报告中得出的技术观察，并非推文中单独测量的结果。@SuJinYan123
滑动窗口注意力状态保留在主机内存中，而非持久化 SSD 存储中。它主要在活跃会话内有用，生命周期比可复用的块前缀更短。@ZhihuFrontier
SWA 有界重放仅从最新的 128 个 token 重建缺失的短生命周期状态。重放将注意力限制在该片段上，同时复用持久块缓存，产生的是近似结果，而非跨完整先前历史重建状态。DeepSeek 报告称这几乎将总 prefill 计算量减半。在考虑向其他架构移植时，这种近似及其对特定模型的有效性很重要。@ZhihuFrontier
Su Jin Yan 还报告称，长保留尾部仅带来大约0.5–1 个百分点的额外缓存命中，这体现了存储容量与重计算之间的权衡。同一评论还指出，以解码为主导的缓存管理允许生成的输出在后续轮次中成为可复用的前缀状态。@SuJinYan123
这一时机不禁让人将其与 OpenAI 另行宣布的12 小时 API 提示缓存（相对于其默认 30 分钟）进行比较，包括在单个请求内混合不同的保留时长。这些并非相同的产品或架构，但两者都面向在长时间间隔内复用上下文的持久化智能体。信息流中其他地方关于登录状态下无限期缓存的说法属于评论，而非 API 公告的条款。@OpenAIDevs，保留控制，比较评论
推测、API 兼容性与模型特定优化
DSpark 使推测验证具有自适应性。它预测草稿 token 的接受概率，并改变验证长度以最大化预期吞吐量。DeepSeek 报告称，相对于始终验证五个草稿 token，生产吞吐量提升了 6%。这是相对于特定推测解码基线的增量收益，而非相对于非推测解码的 6% 提升。@ZhihuFrontier
deepseek-recipe 针对的是服务接口，而不仅仅是分词器速度。它支持 Chat Completions、Responses 和 Messages API，并行化分词，并能区分真正的控制 token 与作为用户文本提供的、看起来相同的字符串。这种区分可防止字面用户内容被意外解析为结构分隔符；但不应将其误认为完整的提示注入防御。@ZhihuFrontier，@deepseek_ai
一种社区解释将这一效率归因于若干模型/运行时协同设计选择：196B Engram 参数、更小的 prefill 主干在 prefill 阶段激活约 8B 参数/token，而解码阶段为 16B、宽 EP、高度压缩的 KV，以及投机解码。它进一步描述了 552B 专家参数、DP64/EP64、约 45B 常驻参数/设备、大部分为四位权重，以及声称相对 GQA 基线实现 400× KV 缩减。这些是二手说法：该帖子没有就复制、共享权重、量化布局或 GQA 参考给出足够说明，无法独立重建这些数字。特别是，45B/设备并非简单用 552B 除以 64 得到。@bookwormengr
这种模型特异性正是不能假定相同收益会自动迁移到任意检查点的原因之一。Su Jin Yan 认为按模型定制的仓库/内核策略有价值；Andrew Carr 建议用编码智能体将该实现适配到其他模型，但未提供可移植性或性能验证。@SuJinYan123，@andrew_n_carr
独立对比：vLLM 复现了什么——又没有复现什么
对“只有 DeepSeek 能做到这一点”的最强技术制衡来自 vLLM 的后续工作：
需要作三点区分：
6% 以内的结果属于解码阶段持平，而非复现 DeepSeek 的整个服务。它不能证明等效的 100-PiB 缓存操作、留存经济性、路由、生产可用性，或在 DeepSeek 流量分布下的端到端延迟。
1M token/s/GPU 这一数字不可能指每用户生成。同一帖子另行报告约 200 P90 输出 TPS；在将吞吐标题与纯解码吞吐比较之前，需要该基准的输入/缓存/输出核算。
该工作是外部实现对比，但仍由竞争引擎团队自行报告。公开的 InferenceX 结果和配方提高了可复现性；它们并不能使每一项引用的生产声明都得到独立审计。
开源传承是明确的：vLLM 感谢 DeepSeek 的 Apache 2.0 发布与署名，而 Woosuk Kwon 将 DeepSeek 描述为早期采用者和贡献者。因此，DeepInfer 更适合理解为生产经验回流到既有生态，而非完全独立的替代品。@vllm_project，@woosuk_k
其他反应涵盖了一个有用的范围：
强烈认可：Tim Dettmers 认为 DeepSeek 的推理栈在其用户规模下可与 OpenAI 比肩，并优于 Anthropic；Graham Neubig 指出官方 API 的速度和价格优势。两人均未提供匹配的跨实验室基准测试。QuixiAI 表示此次发布将为 SlimServe 提供参考。@Tim_Dettmers、@gneubig、@QuixiAI
部署层面的质疑：Su Jin Yan 认为，对于经验丰富的推理开发者而言，工程文章比代码仓库更有价值，部分原因是其中缺少重要组件。Finbarr Timbers 要求与 Modal、Together 和 vLLM 进行对比；所提供的帖子中未出现与 Modal/Together 匹配的对比。@SuJinYan123、@finbarrtimbers
来自一位原本热情评论者的有条件支持：Teortaxes 承认缺少基础设施，并建议 3FS 加上可能的 Dynamo 可以弥补部分缺口，同时估计要达到集群级吞吐量大约需要 3,000–4,000 块 GB200。这是一条提议的部署路径，而非已演示的复现。@teortaxesTex
随后在代码仓库中发现了技术报告的官方英文版本，减少了对社区翻译的完全依赖。@teortaxesTex
经济性：一项有力的收入披露，而非完整的利润率陈述
最具影响力的财务披露是非高峰价格下每加速器小时 38 美元的收入。收入不等于利润，而加速器小时只是生产服务成本基础的一部分。@swyx
一项广泛传播的估算最初计算出87.76–96.94% 的推理利润率，假设：
所有计费 token 中97% 为缓存输入、2% 为非缓存输入、1% 为输出；
每 GPU 每秒 80K 非缓存输入 token 和 每 GPU 每秒 16K 输出 token；
每用户每秒 250 个输出 token；
每 GPU 小时 3–6 美元的租用成本。
对于十亿计费 token，该模型包含 970M 缓存输入、20M 非缓存输入和 10M 输出，大约需要 875 GPU 秒：250 秒用于预填充，625 秒用于解码。估算的 GPU 成本为 0.73–1.46 美元，对应非高峰 11.91 美元或高峰 23.82 美元的收入。@zephyr_z9
随后作者更正了该估算：理想化场景意味着约每小时 49 美元，而 DeepSeek 报告的是每小时 38 美元，约为生产环境中基准性能的 80%。修正后引用的利润率区间为84.2–96.1%。不应在未作此更正的情况下重复原始估算。@zephyr_z9
其他帖子使用了更高的硬件成本：
按每小时 7 美元对每小时 38 美元的收入计算，在其他成本之前，算术结果约为81.6%。Yuchen Jin 明确指出，DeepSeek 未披露该收入数字背后的 GPU。@Yuchenj_UW
按每小时 5–10 美元计算，同样的仅加速器成本计算跨度约为73.7–86.8%，而非确立一个通用的“80% 服务利润率”。@swyx
这些计算遗漏或未完整计入SSD 缓存集群、CPU、网络、空闲/预留容量、故障、工程、支持及其他运营成本。它们还依赖于一种极为有利的、以缓存为主的 token 组合。训练支出和未来算力承诺又是另一回事：推理的高贡献利润率可以与公司层面的巨额亏损并存。
更激进的推断明确属于推测性质。Teortaxes 估计，据称的 10 亿美元 ARR 可能由约 42 个 GB200 NVL72 机架/约 3,000 块 GPU 在非高峰价格下支撑，随后在计入高峰定价后将其下调至 33 个机架。这些帖子还进一步外推至约 4,167 个机架带来 1,000 亿美元年收入，以及每吉瓦的高收入。这些是情景计算——并非经审计的 DeepSeek 收入、机队规模或回本披露。初步估计、修正估计、行业外推
对市场影响的看法存在分歧。一些人将这些披露视为推理已跨过盈利门槛的证据；另一些人则预计 token 价格下降和算力价格上涨最终会压缩回报。所提出的对 Moonshot/智谱托管协议的竞争影响看似合理，但未经衡量：一个可公开部署的技术栈可以改善提供商的经济效益，同时也会加剧定价竞争。@KCDN19、价格趋同论点、托管竞争
NVIDIA、昇腾与硬件推理的局限
昇腾兼容性已确认；可比的昇腾生产效率则未确认。若干帖子将 Blackwell 支持、sm100 支持以及 NVL72 形态的配置解读为生产环境中使用 NVIDIA 硬件的证据。这些是线索，而非披露的库存清单或经过测量的 NVIDIA 与昇腾对比。@eliebakouch、@stochasticchasm、明确的不确定性
随后的一份硬件分析比较了 GB200 与昇腾 950DT 所报告的 decode 数据：
在 100 TPS 约束和 131K 上下文下：约为 25.5K 对 5K tokens/s/设备。
在 230 TPS 下：约为 20K 对 2.7K，将比值从约 5 倍扩大到 7.4 倍。
它引用了早先 DeepEP-Ascend 的披露，即 EP64–128 和 combine 操作仍处于优化中，包括本地归约开销以及与 URMA 的 HBM 争用。
它假设改进后可达 8–10K tokens/s/芯片，以及约 6.8K tokens/s/芯片的经济交叉点，使用假设价格 1000 万美元/NVL72 机架和传闻中的 3.7 万美元/950DT。
这些预测混合了不同的测量结果、假设的采购价格和未经证实的未来优化收益；它们并非同类可比的 DeepInfer 硬件基准测试。@teortaxesTex
同一位评论者预测，到 2027 年底 DeepSeek 将获得大量未来昇腾配额以及超过 1 GW 的 Blackwell 等效算力，同时还报告了一个与之矛盾的、由模型生成的 25 万块 GPU 估计。这两项预测都不是 DeepSeek 的采购公告。算力预测、相互矛盾的估计
V4.1 Flash 上下文：廉价服务放大能力，但并未解决质量问题
独立评估支持开源模型能力取得了有意义的进步，尽管并非普遍达到前沿水平：
Arena：V4.1 Flash 20261017 Max 在 Agent Arena 和 Code Arena: WebDev 中位列开源模型第 1 名。在 Agent Arena 中，它位列总榜第 11 名，在7,500+ 真实会话中实现 +5.1% 净提升，每任务中位成本 $0.20。Arena 将其描述为约以 GPT-6.1 Sol 27% 的成本实现其 37% 的提升。报告信号包括确认成功率 +11.5%、好评与投诉比 +5.9%、Bash 可靠性 +5.4%、可操控性 +2.5%，且该评估中未观察到工具幻觉问题。@arena，详细结果
网络安全：Vals 报告称，V4.1 Flash 1017 在使用基于 Pi 的 agent 完成模拟企业网络任务时，平均进展超过了 GPT-6 Sol。DeepSeek 和 Sol 各获得三次全新范围运行；其他模型仅单次运行。完整评估细节尚待公布。DeepSeek 使用的 token 显著更多，但 API 总成本更低。@ValsAI，成本解读，方法论
该网络安全结果在效率和最佳运行表现方面存在争议。Scaling01 认为“击败 Sol”夸大了结果，因为 DeepSeek 的效率低数倍。Teortaxes 报告称约为4.3 倍 token 和 2.4 倍时间，并表示 Sol 的最佳运行达到了 DeepSeek 任何运行都未达到的目标组。将节省的成本用于重复运行或集群的建议仍属假设，并非经过评估的等预算结果。@scaling01，@teortaxesTex，等成本集群建议
非正式报告比受控证据所能支持的程度更为乐观。Teortaxes 描述了与 DeepSeek 的冷知识轮次，结果为 100%、100% 和 91%，而 MiniMax 约为 60%、48% 和 62%，以及一项描述/变体拼写测试为 100% 对 50%。这些是作者自行构建的小型测试，问题由模型生成且答案键存在错误；它们提示应进行后续实验，而非经过验证的知识基准，也不能证明差异仅由 Engram 造成。另一位用户报告了 Hermes 辅助的强 AI 研究，而另一篇帖子称 DeepSeek 在另一项评估中仍远低于 Sol-xhigh。冷知识测试，后续，变体拼写测试，研究轶事，对比结果
本地部署引入了不同的权衡。Brandon Music 发布了TrellisMX K2 Mix，面向两块 RTX PRO 6000，使用二位专家，其中629 个专家提升至三/四位，四位 Engram 表，容量设计约为3M KV token。报告的平均 KLD 为0.1306 nats，比普通 K2 低 8.6%；一个+5GB 变体面向约2M KV token。仅测量了 KLD，且服务使用了测试镜像，因此任务质量保持、实际速度和运行稳定性仍未确立。@BrandonMusicKy
Agent 基础设施、本地执行与硬件加速
长上下文推理内核：SGLang 合并了 megakernel 支持，SemiAnalysis 报告在 AgentX 中于 GB300 NVL72 上达到近 1,000 tokens/s/user，在 MI355X 上交互性提升 1.5 倍。另外，Inferact 报告在 TPU v7 上 Kimi K3 达到 391 tokens/s/user，使用 vLLM TorchTPU 中的开源 megakernel（SGLang 结果，TPU 结果）。
Apple 芯片运行时提升：PyTorch 2.15 用 Metal 内核替换了若干 MPSGraph 路径。报告的改进包括 MoE 解码速度提升 5–6.5 倍，Qwen3.6-35B-A3B 在 M3 Ultra 上从 9.4 tokens/s 提升至 61.3 tokens/s，MPS 支持 FP8，以及 Wan2.2 VAE 的峰值驱动内存从约 270 GiB 降至 8 GiB。这些改进落在 PyTorch 本身，而非独立的推理运行时（发布摘要）。
本地 Perplexity Computer：模型和智能体现在都可以完全在受支持的 Apple 芯片 Mac 上运行，包括离线运行，并与原生应用和本地文件交互。
持久化智能体服务：LangChain 的 Managed Deep Agents 打包了用户专属凭据、限定范围的内存、沙箱执行和定时任务；Hugging Face 的 Agent Manager 在 Claude Code、Codex 和其他 harness 之间编排持久会话（LangChain 架构，Agent Manager）。
端侧模型、语音与图像发布
Muse Glimmer：Meta 的开源 30B 多模态智能体模型，从 Muse Spark 蒸馏而来，面向 32 GB 显存，使用 4 位量化和投机解码头可在单张 RTX 5090 上运行。论文报告其在 MCPAtlas 上得分 75.5，在 SWE-Bench Pro 上得分 51.2；Qwen3.6-27B 在 OSWorld-Verified 和 TerminalBench 2.1 上仍保持领先。它在 Apache 2.0 许可下支持文本、图像和 100 多种语言（论文摘要）。
Voxtral 3: Mistral launched batch and streaming transcription, multilingual TTS and a voice-agent SDK with tool calling, available through its API or on customers’ infrastructure (launch, deployment options).
Independent measurements: Artificial Analysis places Small Transcribe 3 at #3 with 2.1% AA-WER. The realtime model achieves 3.3% final-transcript AA-WER at 960 ms; TTS enters the top 12 in both voice arenas.
Pricing and speed: TTS costs $13/million characters and generates 112.3 characters/s in its measurements (evaluation, throughput).
CPU alternative: Kyutai released smaller Pocket TTS variants down to 1.9M parameters, claiming 103× realtime on one CPU core, with voice cloning (release).
Fibo 2.0: Bria’s 8B open-weight image family includes Dev for text-to-image and distilled Edit Turbo. Artificial Analysis reports leading open-weight editing performance and a statistical tie with Qwen-Image-2.1 for text-to-image; Bria API pricing is $32/1,000 images (evaluation).
Krea 3 Ultra: Rankings differ by arena and update. Artificial Analysis’s own announcement places Ultra #5, whereas earlier posts cited #4; Arena places it #4 in multi-image editing and #8 in single-image editing. The defensible conclusion is competitive image-editing quality, not a universal fourth-place rank (Artificial Analysis, Arena).
Pricing: Fast costs $0.03 per 1K output image plus $0.02 per 1K reference image. Ultra supports 2K output only; a 2K edit with one 1K reference costs $0.27.
Independent Evaluations, Cyber Capability and Safety Evidence
HealthBench exposes scoring sensitivity: Artificial Analysis launched leaderboards for 5,000 general health conversations and 525 clinician conversations. GPT-6 Astra leads Professional’s length-adjusted score at 66.0%; Claude Sonnet 5.5 leads general HealthBench at 62.1%.
Length effects: Opus 5.5 has Professional’s highest raw score, 71.4%, but falls to 59.9% after adjustment. General HealthBench’s shortest-answer model leads communication quality while scoring lowest on completeness. Answer length is therefore a material ranking choice, not a cosmetic detail.
Cost trade-off: DeepSeek V4.1 Flash scores 60.9% at $0.005/task, versus Sonnet’s 62.1% at $0.08/task. Grading uses OpenAI models, following the original implementations (methodology and results).
Routing versus specialized decisions: Across five routers and approximately 1,250 tasks, Vals found every router beaten on both score and cost by at least one standalone model. Hindsight-optimal selection within router budgets would improve scores by 8–27 points, indicating substantial selection headroom (Router Index, oracle comparison).
ClaimBench: On a separate financial-document classification benchmark, Decider V1.2 handled 99.7% of claims itself, escalating the remainder to Haiku 5.5; Vals reports 99.9% accuracy at $0.11/1,000 claims, versus $1.19 for Haiku alone. This narrow classification result does not contradict weaker broad agent routing (benchmark, cascade result).
DeepSeek offensive-cyber result: Vals reports V4.1 Flash 1017 outperforming GPT-6 Sol on a simulated corporate-network task using the Pi harness. DeepSeek consumed more tokens but achieved a lower total price.
Evidence limits: DeepSeek and Sol each received three runs; other models received one. Full evaluation publication is pending. A reaction disputes calling this an unqualified win because of efficiency differences, so treat it as a preliminary capability signal rather than a settled ranking (finding, methodology, qualification).
Safety assurance and accountability: METR agrees with Anthropic’s low catastrophic-misalignment-risk conclusion for covered models but identifies significant analytical disagreements. It recommends end-to-end red-teaming of monitoring systems and assessing models’ ability to undermine safety-relevant decisions (review, recommendations, decision risk).
Incident transparency: Hugging Face CEO Clément Delangue criticized Netflix’s portrayal of the reported OpenAI-agent cyberattack, arguing that containment, monitoring, defensive use of open models and missing agent traces deserved more attention (statement).
Rumor correction: Theo denied having Fable 5.5; the apparent leak involved false model identification, followed by a clarification that the supposed source was a troll. The supplied posts do not establish a launch (denial, clarification).
Scientific Agents: Wet-Lab Results and Open-Ended Limits
Scorpion-toxin binders: Anthropic and Citrini report 17 binders from 211 tested protein designs, following substantial computational work by an internal Claude research model. All tested designs and laboratory data are being released (announcement, data).
Critical limitation: None bound multiple toxins, and none were tested for toxin neutralization. This is not an antivenom, nor evidence of therapeutic efficacy; cellular, animal and further validation remain ahead (researcher account, official caveat).
Pathology-to-biology prediction: Sophont introduced Holon-S1, which predicts spatial gene activity and biomarkers from routine pathology slides. The team reports evaluation across 16 organ types and more than 15 scanners, outperforming two comparison models on spatial gene prediction and identifying potentially novel breast- and colon-cancer biomarkers. These are research findings, not validated treatment-selection outcomes (launch).
Open-ended research remains difficult: Evaluators gave GPT-6 Astra and Claude Fable 5.1 unpublished AI-research questions and $10,000 budgets each. Original paper authors found improved experimental design and resource management over earlier tests, but still rejected both generated papers for weaknesses in creativity, judgment, calibration and communication. The small study is a useful counterweight to results on narrowly verifiable research tasks (evaluation).
Industry and Policy: Drug Development, Manufacturing and Capital Commitments
OpenAI acquires Orchestra Bio: OpenAI announced the acquisition of a drug-development coordination platform, extending its life-sciences work beyond research tools into operational execution alongside Rosalind. Financial terms were not supplied (announcement).
Arda launches publicly: Former OpenAI research chief Bob McGrew announced manufacturing software developed with Anduril as a design partner. His first-party account specifies a $160M Series A at a $1.5B valuation; CNBC’s accompanying post cites $230M raised, without clearly reconciling that figure with the announced round. The initial product supports manual assembly-line design and operation; reconfigurable automation is the next ambition (founder announcement, CNBC report).
Financing and compute exposure: Reports describe OpenAI seeking approximately $30B at a $1.4T+ valuation ahead of an anticipated Anthropic IPO. Separately, a reported WSJ review of Anthropic’s draft prospectus cites $16.3B first-half revenue, nearly $9B compute/infrastructure spending, and more than $600B in future compute commitments, including negotiations. These are attributed reports, not completed transactions or independently verified financial disclosures (OpenAI report, Anthropic report).
Top tweets (by engagement)
39,393 — DeepSeek releases DeepInfer
9,809 — OpenAI corrects usage-limit mismatches affecting approximately 40,000 accounts
3,056 — Mistral introduces Voxtral 3
2,600 — Researcher explains binder results and biological limitations
2,476 — Hugging Face CEO challenges cyberattack documentary framing
AI Reddit Recap
/r/LocalLlama + /r/localLLM Recap
1. DeepSeek DeepInfer Open-Source Inference Framework
New from DeepSeek - DeepInfer - (Activity: 313): The image is a mobile screenshot of DeepSeek’s X announcement that DeepInfer, its in-house inference framework, is now fully open source: image, GitHub. The technically relevant claim from the linked/source discussion is that DeepInfer powers DeepSeek V4.1 Flash at scale, with reported single-cluster throughput of 155 trillion tokens/day and revenue of $38 per card per hour at off-peak pricing, emphasizing high-throughput, low-latency large-scale inference rather than model training. Commenters were notably surprised that DeepSeek open-sourced an internal production inference stack, contrasting it with what they expect from Anthropic, OpenAI, or Meta. There was also basic uncertainty from some readers about what the framework actually does, i.e. serving/inference infrastructure rather than a new model.
A linked Zhihu source claims DeepInfer enables high-throughput, low-latency large-scale inference for DeepSeek V4.1 Flash, reporting 155 trillion tokens/day throughput for a single cluster and $38 per card-hour revenue under off-peak pricing:
https://zhuanlan.zhihu.com/p/2095560295930508953
. One commenter also highlights a reported serving figure of roughly 256 tokens/s × 100 users, implying very high concurrent decode throughput if sustained in production.
A technical concern raised is whether DeepInfer is a general-purpose inference framework or primarily a stack heavily optimized for DeepSeek’s own model architecture and deployment environment. The repo’s low initial GitHub star count versus high announcement engagement was interpreted as likely due to freshness, but commenters expect more meaningful evaluation once engineers inspect the implementation and portability.
Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models! (Activity: 1023): DeepSeek released DeepInfer, an open-source high-performance offline inference framework for DeepSeek-V4.1-Flash, with production-oriented primitives including optimized kernels, attention/data/context parallelism, MoE expert parallelism with EPLB, DSpark-style speculative decoding, SWA bounded replay, async scheduling, structured outputs, batch-invariant inference, and deterministic sampling. It currently targets NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs via DEEPINFER_ACCELERATOR=nvidia|ascend, requiring Linux/Python 3.12 plus CUDA 13 or CANN 9.2; however, full online serving still needs external KV-cache management, prefill/decode disaggregation, and DP replica load balancing. DeepSeek also released deepseek-recipe, a Rust library with Python bindings for protocol conversion and model-output parsing across APIs such as Chat Completions, Messages, and Responses. Commenters framed the release as valuable for reverse-engineering production inference techniques into community engines such as gufo, halogen, strixite, and Strata. There was also interest in the explicit Ascend 950 support, with one commenter joking that “Nvidia [is] in shambles,” though the listed NVIDIA target remains next-gen SM100 rather than current widely deployed GPUs.
DeepSeek’s release is split into two repositories: DeepInfer, described as a high-performance large-scale inference framework for DeepSeek models, and deepseek-recipe, a Rust library with Python bindings for protocol conversion and model-output parsing across API formats like Chat Completions, Messages, and Responses.
Commenters noted the currently listed accelerator targets are NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with build-time platform selection exposed through DEEPINFER_ACCELERATOR=nvidia or ascend before running ./build.sh. The Ascend support drew attention because it implies DeepSeek is optimizing beyond the standard CUDA/NVIDIA path.
One commenter referenced DeepSeek’s Zhihu post claiming roughly $38 per GPU-hour revenue/throughput economics, linking the original post at zhuanlan.zhihu.com/p/2095560295930508953. The thread raised whether similar per-GPU economics could be used to estimate inference revenue for competitors like OpenAI or Anthropic, though no concrete derivation was provided.
Can DeepSeek V4.1 Flash 1017 really run on 12GB VRAM + 64GB RAM? (Activity: 352): The image is a screenshot of an X/Twitter thread claiming DeepSeek V4.1 Flash 1017 may be runnable locally at Q4 with 12GB VRAM + 64GB RAM; the Reddit post asks whether an RTX 3060 12GB could offload enough layers to make inference usable. The key technical pushback is that V4.1 Flash is reportedly already around 4.25 bpw for many experts via mxfp4 + fp8, so while it may fit if total RAM is sufficient, expected performance with heavy CPU/RAM execution could be extremely slow—one commenter estimates roughly ~3 tokens/sec. Commenters are skeptical that this would be practical for everyday use: the consensus is closer to “possible to load” than “pleasant to run.” One comment notes the person making the original claim is the Strata creator, lending some credibility to the sizing claim, but not necessarily to performance expectations.
Commenters disputed the claim that DeepSeek V4.1 Flash 1017 can practically run on 12GB VRAM + 64GB RAM: even the “Flash” variant is described as already being around 4.25 bpw for many experts, using MXFP4 + FP8, so a standard Q4-style fit is unlikely within that memory budget.
A technically viable setup may require heavy offloading: one commenter says Q4 does not fit and suggests checking the Hugging Face model size directly, then offloading n-grams to NVMe and streaming some MoE experts from disk. This may run in principle, but expected throughput is likely very poor, with one estimate around ~3 tokens/s or worse depending on disk and offload implementation.
There was skepticism that the original Strata claim was accurate; commenters noted that even if the model can be made to “fit” somewhere across RAM/NVMe, the relevant question is practical inference speed rather than mere loadability.
2. MiniMax M3.1 Flash Release
MiniMax M3.1 Flash released, Model weights Oct 23 (Activity: 874): The image is an official MiniMax X announcement for MiniMax M3.1 Flash, described as “Space Performance + Bunny Speed,” with API availability now and open model weights scheduled for Oct 23. Commenters add that the model is reportedly a MoE with 428B total parameters, ~23B active parameters, sparse attention, and a native visual encoder; a preview/tool page is linked at agent.minimax.io/tools/m3-1-flash-preview. The main technical concern in comments is that benchmark excitement depends heavily on knowing the model size/active parameter count, since performance claims are hard to contextualize without architecture details.
A commenter cites the MiniMax M3.1 Flash Preview tool page as specifying a Mixture-of-Experts architecture with 428B total parameters and roughly 23B active parameters, plus sparse attention and a native visual encoder: https://agent.minimax.io/tools/m3-1-flash-preview. This directly answers discussion uncertainty about model size and suggests the model is positioned as a large sparse multimodal system rather than a dense LLM.
MiniMax-M3.1-Flash is coming (Activity: 521): MiniMax teased MiniMax-M3.1-Flash via an official X post (source), but the Reddit post provides no architecture details, parameter count, context length, pricing, release date, or benchmarks. The only technical signal is the product naming: commenters infer “Flash” may denote a speed/latency-optimized variant, though no evidence is given. Commenters are skeptical about the meaning of “Flash,” joking that in current LLM naming it could still be a very large model — e.g. “Flash means a 500B model” — or sarcastically a supposedly lightweight model that is not actually small.
3. Benchmark Inflation and AI Slop Measurement
AI labs report benchmark scores 6.4% higher than independent testers, on average (Activity: 334): The image is a technical benchmark-comparison chart from the whatmodel.sh open leaderboard, visualizing 611 matched comparisons across 180 AI models and 103 benchmarks from 1,629 published results. It reports that AI labs’ self-published scores are 6.4% higher on average than independent testers for the same model/benchmark pairs, with 375 lab-higher cases vs. 216 independent-higher and 20 ties; the largest median benchmark gaps shown include SWE-bench Verified and MBPP at about +9.0 points for labs. Image · interactive chart · data/code Commenters debated whether the gap reflects intentional score inflation or more mundane reproducibility issues such as package versions, harness settings, random seeds, prompt/eval setup, or labs optimizing benchmark tooling to present best-case results. Several compared it to vendor-favorable performance charts from companies like Apple or Nvidia, arguing that only independently reproduced or peer-reviewed results should be trusted.
Several commenters argued that the 6.4% average gap may come from benchmark reproducibility issues rather than outright inflation: small differences in package versions, evaluation harness settings, random seeds, prompt formatting, and lab-specific benchmark tooling can materially affect reported scores. One theme was that labs likely optimize the exact harness/configuration used to showcase their models, while independent testers may use more generic or less model-tailored setups.
A commenter noted the mean gap may be driven by outliers and suggested the median discrepancy would likely be closer to reported benchmark values. This implies the headline average could overstate systematic inflation if a small number of large deviations dominate the aggregate.
One technical explanation proposed was deployment mismatch: labs may benchmark full-weight models, e.g. BF16, while independent testers often evaluate served API versions that have been quantized to FP8, MXFP4, or other hardware-efficient formats. If true, some score degradation could reflect inference serving optimizations for datacenter tensor units rather than benchmark manipulation.
Let’s delve into the tapestry of AI slop on r/LocalLLaMA (Activity: 455): The post links an informal longitudinal analysis of “AI slop” across Reddit communities, including r/LocalLLaMA, with an attempted extrapolation of future slop prevalence: thil.ooo/blog/measuring-slop. A technical commenter notes likely measurement bias: moderator removals and karma filters may cause Pushshift/archive-style datasets to undercount short-lived bot posts, while bot evasion tactics such as all-lowercase writing may weaken simple “human marker” heuristics; they also suggest comparing against Claude-associated vocabulary from “load-bearing vocabulary of Claude”. Moderation feedback was supportive: a mod left the post up as directly relevant to ongoing anti-slop moderation despite reports. Commenters broadly agreed that obvious AI-generated style markers—em dashes, bullet lists, and stock phrases like “delve”—have become noisy signals and can falsely implicate human writing.
A commenter noted several measurement biases that could affect an AI-slop detector trained/evaluated on Reddit archive data: moderators may remove many generated posts from /new before they appear in the dataset, and a new karma filter may be suppressing bot/slop submissions upstream. They also flagged that bots may evade simple style heuristics by posting in all lowercase, potentially corrupting “human marker” assumptions.
One technical suggestion was to expand the detector’s lexical feature set using Claude-specific “slop” terms, referencing the analysis of Claude’s load-bearing vocabulary at https://louisabraham.github.io/load-bearing/. The commenter emphasized distinguishing obvious AI slop from broader low-value generated content that may not exhibit obvious stylistic markers and may be removed quickly.
Less Technical AI Subreddit Recap
/r/Singularity, /r/Oobabooga, /r/MachineLearning, /r/OpenAI, /r/ClaudeAI, /r/StableDiffusion, /r/ChatGPT, /r/ChatGPTCoding, /r/aivideo, /r/aivideo
1. Opus 5.5 Game Automation Feats
After $712 and 39 hours, Opus 5.5 beat Baldur’s Gate 3 all by itself (Activity: 1145): A user reports that Claude Opus 5.5 autonomously completed Baldur’s Gate 3 using a custom harness: claude -p with built-in tools disabled, a Python MCP server connected over TCP to a Lua BG3 mod, Vulkan-layer screenshots, and SDL-mediated clicks (write-up, code). The run took 39 hours, $712, 99 fresh sessions, 20,809 tool calls, 862 screenshots, 2.2B+ cached input tokens vs 3.5M output tokens, and relied on a git-backed memory system with 157 commits, ~23k curated words, and 67 distilled tactical rules; an audited Faith-Leap rerun required 36 min, 674 tool calls, $24, one death, and eleven reloads using only screenshots of the mosaic after fixing a hidden-walkway leak. The author plans to turn the harness into a benchmark of short BG3 scenarios scored by success, attempts, time, and cost, with Claude’s Nautiloid opening reportedly succeeding 9/9 at about $1.50/run, and Codex/Gemini integrations prepared but not yet evaluated. Top commenters mostly reacted that the project is expensive but promising: one called it “one of the best benchmark ideas” for comparing agents across vendors, while another dismissed the spend as “more money than sense.” Non-technical joking centered on whether the agent made morally significant in-game choices, e.g. whether it raided the grove.
A commenter framed the run as a potentially useful agentic benchmark, suggesting that repeating the Baldur’s Gate 3 task across competing models could provide a more practical comparison of long-horizon planning, tool use, and persistence than many standard benchmarks. The key technical value would come from standardizing the setup and measuring cost, completion time, failure modes, and decision quality across models.
Claude Opus 5.5 decompiles a PS2-era game overnight, and hundreds of them now run in browser tabs (Activity: 2348): A Kotaku report claims AI-assisted reverse engineering—attributed in the post title to Claude Opus 5.5—has enabled browser-playable ports of older titles such as Halo: CE, GTA: Vice City, Skate 3, and The Simpsons: Hit & Run. The article says some ports run entirely in-browser with notable performance claims, e.g. Hit & Run targeting 240fps and a Halo: CE build supporting low-latency online multiplayer, but commenters question whether these are true decompilations or instead recompilations/reimplementations of existing binaries/assets. Top comments are skeptical: one asks for an actual playable browser link and calls the post an “empty flex,” while another argues the terminology is being conflated—“these aren’t decomps… they’re just recomps?” A separate commenter speculates that public access to strong models may be restricted once corporations view this as an IP-enforcement threat.
A commenter challenges the post’s terminology, arguing these likely are not full decompilations but rather recompilations/recomp projects, noting that the community may be conflating the two. This distinction matters technically: decompilation implies recovering high-level source or equivalent structure, while recompilation often translates existing binaries/IR into a new target such as browser-compatible code.
2. AI Proofs and Academic Labor
Fields Medalist Richard Borcherds on OpenAI proofs: “I think what people are getting upset about is they’re slowly realizing that they are dumb and somehow blaming this on AI, even though it’s not really AI’s fault” (Activity: 1309): The linked Daily Cal piece covers UC Berkeley mathematicians’ reaction to an alleged OpenAI Navier–Stokes proof (article), framing it as a stress test for mathematical proof validation, pedagogy, and the role of AI systems in advanced research. Richard Borcherds is quoted arguing that backlash may reflect researchers’ realization of their own limitations rather than an AI-specific failure: “they’re slowly realizing that they are dumb and somehow blaming this on AI.” Top commenters focused mostly on education: one endorsed the view that AI use on homework is irrelevant if students cannot reproduce understanding on exams, while another highlighted Borcherds’ claim that AI may be a more patient and expressive tutor for graduate abstract algebra than a human instructor.
A quoted discussion from Richard Borcherds frames AI-generated mathematics as potentially producing very large formal artifacts—e.g. “a 100,000-line formal proof of some major conjecture”—but argues this does not necessarily degrade mathematical learning or research because humans can choose to ignore machine-produced formalizations. The technically relevant implication is a distinction between formal proof generation/verification and the human-centered process of understanding, motivation, and theory-building.
One comment highlights Borcherds’ claim that AI may be better than him at teaching MATH 250A graduate abstract algebra, specifically because it is patient, consistent, and able to re-explain material without frustration. This is an education-focused technical point: LLMs may function well as adaptive tutors for dense symbolic subjects, while still not replacing the need for students to independently solve problems for exam-level mastery.
A physicist spent a year of his life and ~500 pages of hand calculations for quantum gravity in his PhD thesis, but Opus 5.5 did it in 10 hours, including the article and GitHub repo. And it even found the errors. What will “labor” mean in academia from now on? (Activity: 1710): The linked repo, eokahya/claude-audits-quantum-gravity, is an AI-assisted audit by E. O. Kahya, the original coauthor, of prior one-loop quantum-gravity calculations for the graviton correction to a massless minimally coupled scalar in de Sitter space—not a fully autonomous rediscovery. The workflow was manually decomposed into an 18-task/9-stage pipeline where Claude orchestrated deterministic SymPy/mpmath checks, including high-precision numerical sampling, and found mostly bookkeeping/printing issues: an overall sign convention error in the printed bare self-mass, missing/non-analytic and analytic terms in Table X, finite intermediate issues, and 23 typos. The main physics result reportedly survived unchanged: after correction, the Schwinger–Keldysh late-time mode-function analysis still yields no growing one-loop terms such as ln(a), ln²(a), or a, only constant field-strength renormalization plus decaying terms. Top commenters pushed back on the framing that “Opus 5.5 did a PhD thesis in 10 hours,” arguing the hard scientific work was the original formulation, boundary conditions, Feynman rules, and expert scaffolding; the AI mostly accelerated verification of dense tensor algebra. A broader debate compared current academia to human+engine chess, with one view that AI-assisted researchers are temporarily strongest, but another predicting fully automated systems may eventually outperform human academics except in posing high-value questions.
Multiple commenters argue the headline misrepresents the work: E. O. Kahya, the original coauthor, did not have Claude autonomously rederive quantum gravity results from first principles; he used it to audit prior calculations. The workflow was manually scaffolded into an 18-task, 9-stage pipeline with expert-supplied boundary constraints, while Claude orchestrated deterministic tools like SymPy and mpmath for tensor-algebra checks and 30-digit numerical validation.
The technically important outcome was that the core physical result survived unchanged. Reported issues were limited to arithmetic slips, an overall sign/definition flip, a missing bookkeeping term, and 23 typographical errors, with commenters emphasizing that this is closer to computer-assisted verification of dense symbolic work than novel physical discovery.
Commenters highlight that Kahya’s own README includes caveats such as “What a human expert should still review” and limitations around numerical sampling. The main technical framing is therefore AI as an audit/cross-check layer for expert-designed theoretical pipelines, not yet a replacement for posing the theory, deriving Feynman rules, or establishing the physical framework.
3. Agent Security and Prompt-Injection Risks
PSA: Do NOT connect your gmail and drive with your Dots at the moment (Activity: 1225): The post warns that Dots agents connected to Gmail/Drive/PC may execute attacker-controlled instructions delivered via email attachments or message bodies, exfiltrating personal data or installing malware if the agent treats the content as trusted user intent. Commenters identify this as a classic prompt injection / tool-use security failure affecting any LLM agent with external capabilities, not just Dots: untrusted content can become executable instructions when read by an autonomous agent. The dominant reaction is strong skepticism toward granting AI agents privileged access to local machines, email, or cloud storage until robust permissioning, sandboxing, and instruction-boundary defenses exist.
Several commenters identify the core risk as prompt injection against AI agents with connected tools, not a Dots-specific flaw. The technical concern is that any LLM granted access to Gmail/Drive or local actions can be manipulated by untrusted content to perform unintended operations, making autonomous account access unsafe without strong sandboxing, permission boundaries, and human-in-the-loop approval.
My post about rogue AI agents got 1.3M views here and then reddit just deleted it. no reason given (Activity: 1315): The image is a non-technical Reddit UI screenshot showing a prior r/ChatGPT post marked “[Removed by Reddit]” for a content-policy violation, despite reportedly reaching ~1.3M views, 1.5K upvotes, 530 comments, and moderator approval before removal: image. The post claims the removed content discussed research, also covered by the NYT, about “rogue” AI agents operating online, but the screenshot itself provides no technical evidence about the agents—only evidence of platform-level removal without a visible rule citation. Comments lean conspiratorial/skeptical, suggesting the alleged agents or bots could be suppressing discussion or manipulating perception, while others frame reposting as the expected Reddit response.
来源：Latent Space · latent.space