跳到正文
Lisan al Gaib· @scaling01 · X·· 4 小时前AI 评分37
AI 导读

EpochAI 发布 GPT-6 训练算力与参数量估算,其预训练算力为 RL 算力的 5.35 倍,预训练时长中位数 90% 区间为 42–135 天。

正文

EpochAI also released some estimates for GPT-6 compute and params

I think total training compute estimate is fine, but 5.35x more pre-training compute than RL compute seems silly

the median for the pretraining duration is also very high with a 90% interval of 42–135 days

the most obvious flaw is that for GPT-5.5 (released on April 23) we know that it was trained on Stargate Abilene, and other models like GPT-5.3-Codex, GPT-5.4, and the 5.6 family were probably trained there too, which means that the pre-training of Astra couldn't have used all 100k GPUs on average in all scenarios they model

OR more simply, pre-training just happened later than they modeled after GPT-5.5 release (I put it at early May - late July)

for the model size estimate they also don't really consider trends in sparsity or looping

lets not talk about the param estimates. they are obviously way too high:
- "Across the named scenarios: active 0.1T-4.8T; total 1T-143T (corners: p5 FLOP, 400T, x10 -> p95 FLOP, 50T, x30). Central case (p50 FLOP, 100T tokens, sparsity 20): active 1.2T, total 25T."

引用Epoch AI@EpochAIResearch
The New York Times visualizes the staggering amount of math needed to train AI models, using data from Epoch's research teams 📊 https://www.nytimes.com/interactive/2026/10/15/upshot/ai-data-center-math.html?unlocked_article_code=1.I1E.njvN.w-SoXEYFfzKU&smid=url-share Our training compute estimates: https://epoch.ai/data/ai-models The Epoch Capabilities Index (ECI): https://epoch.ai/eci
在 X 查看被引用的帖子

来源:Lisan al Gaib · x.com