MacroStories：19,969 参数的最小故事生成语言模型
Trained a ~20K LM (probably smallest) that can still write stories
开发者训练出 MacroStories，一个仅 19,969 参数、81 KB FP32 的故事生成语言模型，比 1M 参数的 TinyStories 模型小约 50 倍，比 AlexNet 小约 3,000 倍。
I’ve been pushing TinyStories-style models downward in size, and this is the smallest one so far:
MacroStories — 19,969 parameters, 81 KB FP32
https://huggingface.co/raincandy-u/MacroStories
For scale:
→ ~50× smaller than the 1M TinyStories model
→ ~3,000× smaller than AlexNet
→ 32-dim hidden state
→ 378-token vocabulary
→ one decoder block, recurrently applied 4 times with shared weights
It’s obviously not a general-purpose LM, but within its constrained story distribution it can maintain a 100–300 word narrative with a goal, problem, relevant actions, and resolution.
It also runs extremely fast on CPU and needs no GPU.
I’m mostly interested in how far the lower bound for coherent narrative generation can be pushed.
Would be curious to see how people manage to break it.☺️
submitted by /u/x_Raincandy_x
[link] [留言]
来源：r/LocalLLaMA · reddit.com