假设的 GPT-6 级模型理论参数容量估算：19.2T 只是 FP8 权重内存上限

Estimating the Theoretical Parameter Capacity of a Hypothetical GPT-6-Class Model

I've been exploring a hypothetical question: How large could a model's parameter count be if a major AI provider allocated a substantial GPU cluster to inference while targeting 66 output tokens per second per active user?

This is a rough hardware-based thought experiment, not a claim about OpenAI's actual infrastructure or the architecture of GPT-6 Astra.

Assumptions

• GPU resources: 800,000 H100-equivalent GPUs available across the broader infrastructure.
• Allocation scenario: 10–30% allocated to the hypothetical model's inference workload.
• Per-user generation speed: 66 tokens/second.
• Concurrent users: 10,000–100,000.
• GPU memory: 80 GB per H100, giving 64 PB across 800,000 GPUs before allocation.
• Weight format: FP8, approximately one byte per parameter.
• H100 peak performance: Approximately 1.98 PFLOP/s dense FP8 per GPU, under the relevant hardware configuration.

A rough memory-based estimate

If 30% of the assumed GPU pool were allocated to inference:

800,000 × 0.30 = 240,000 H100-equivalent GPUs.

At 80 GB of memory per GPU, this allocation would provide approximately 19.2 PB of aggregate GPU memory. At roughly one byte per FP8 parameter, that corresponds to a theoretical weight-storage capacity of approximately 19.2 trillion parameters.

However, this is only a memory-based upper bound under these assumptions. Actual usable capacity would be lower because of KV cache, runtime overhead, communication buffers, redundancy, and other memory requirements.

What about inference throughput?

The required output throughput would be:

• 10,000 concurrent users × 66 tokens/s = 660,000 tokens/s.
• 100,000 concurrent users × 66 tokens/s = 6.6 million tokens/s.

Whether a model of a given size could sustain those rates depends on its architecture, batching strategy, memory bandwidth, parallelism, workload, and actual hardware utilization. Peak FLOPS alone cannot establish that it can.

Important limitations

1. The 800,000-GPU figure is a hypothetical resource pool, not a verified count of GPUs owned or operated by OpenAI.
2. The 10–30% allocation is an illustrative assumption, not a known allocation for GPT-6 Astra.
3. No verified parameter count or architecture for GPT-6 Astra is established by this calculation.
4. Parameter count, model architecture, and inference throughput cannot be inferred reliably from GPU count alone.

Conclusion: Under this hypothetical scenario, 19.2T parameters is a rough FP8 weight-memory ceiling before overhead—not a prediction of GPT-6 Astra's actual size. A credible throughput estimate would require additional assumptions about architecture and measured serving efficiency.

I'd be interested in feedback from people experienced with large-scale inference: what assumptions would you change first, and how would you estimate sustainable throughput at this scale?

submitted by /u/Gokbagbasi
[link] [留言]