用 PyTorch 从零训练 8M 参数小 LLM 并复现论文实验，适合分享到 r/MachineLearning 吗？

Is it okay to share my experiments implementing research papers on my own LLM here? [P]

AI 导读

一位开发者用 PyTorch 从零构建并训练了约 8M 参数的小型 LLM，作为复现不同研究论文想法的实验平台，已训练 60,000 步。下一步计划实现 KV caching，之后是 PagedAttention、Orca 等。他以博客记录实验过程与踩坑，询问在 r/MachineLearning 分享这些非推广性内容是否受欢迎。

正文

Hey everyone!

I've built and trained a small LLM (~8M parameters) from scratch using PyTorch, mainly as an experimental lab where I can implement and test ideas from different research papers.

I've been documenting the whole journey through casual blogs, including my mistakes, experiments, and things I've learned.

So far:

Next up is KV caching, followed by PagedAttention, Orca, and whatever else I find interesting.

Just wanted to ask if sharing these experiments and blogs here would be welcome. Nothing promotional, just me experimenting, learning, and hopefully getting feedback from people who know more than I do.

submitted by /u/the-random-guy-2002
[link] [留言]