Taco：跨平台 LLM 压缩工具，更换 GPU 或 CPU 后仍保持一致

Taco: cross-platform LLM compression that doesn't break when you change your GPU or CPU. [P]

AI 导读

Taco 是一个跨平台 LLM 压缩项目，目标是让压缩器在解码时算出与编码器完全相同的概率。作者指出，浮点数在不同硬件上存在差异，在 Mac 上编码的 float GPT-2 到 x86 上解码时。

正文

The problem: an LLM compressor needs the decoder to compute exactly the same probabilities as the encoder. Floats differ across hardware; float GPT-2 encoded on a Mac decoded 0 of 4 windows on x86.
The approach:
- integer-only inference (int8 weights, fixed rounding, lookup tables);
- an exact float32 → integer converter (mantissa × 2^exp);
- conformance vectors.
The proof: one hash across 7 backends (Mac CPU/Metal, Linux CPU/CUDA, Windows CPU/CUDA/Vulkan