有人做过 Chinchilla 缩放定律的小规模实验吗？[R]

Experiments with Chinchilla scaling laws (following DeepMind) [R]

AI 导读

一位用户向群里的 PhD 和研究工程师询问是否有人用 PyTorch 或 JAX 做过 Chinchilla 缩放定律的小规模实验，并希望进一步交流。帖子附上了 DeepMind 官网的 Chinchilla 官方论文《Training Compute-Optimal Large Language Models》链接。

正文

Question to all the PhDs & research engineers in this group.
Have any of you done small-scale experiments with the Chinchilla scaling laws (using PyTorch or JAX)? If so, we'd be keen to hear more (or have a chat). Thanks!

Here's the official Chinchilla paper, from the DeepMind website ("Training Compute-Optimal Large Language Models").
https://deepmind.google/blog/an-empirical-analysis-of-compute-optimal-large-language-model-training/
https://proceedings.neurips.cc/paper/2022/file/c1e2faff6f588870935f114ebe04a3e5-Paper-Conference.pdf

submitted by /u/turing-math-labs
[link] [留言]