Anthropic 的研究重新审视 MLP 神经元在可解释性中的作用，发现后层中许多活跃的 transcoder 特征与单个 MLP 神经元高度接近。作者在 13 个开源 transcoder 上测量了这一现象，并指出只有与神经元对齐的特征与底层 MLP 保持稳定联系，其余特征在多次训练之间变化更大、与 MLP 的 Jacobian 不匹配，也更偏离随机初始化。
Interesting work from Anthropic.
They reframe MLP neurons as a stronger basis for interpretability than commonly assumed.
It is standard practice to study circuits with transcoders, because individual MLP neurons are considered too polysemantic to interpret.
They find that many active transcoder features in late layers are close copies of single MLP neurons. They measure this across 13 open-source transcoders.
These neuron-aligned features are also the only ones with a stable link to the underlying MLP. The other features change more between training runs, don't match the MLP's Jacobian, and move less from their random initialization.
That makes MLP neurons a stronger starting point for interpretability than usually assumed. It also means the non-neuron transcoder features need better validation before they are used to explain model behavior.
https://arxiv.org/abs/2610.24521
Chat with Paper: https://academy.dair.ai/papers/comparing-sparsified-mlps-and-transcoders-many-transcoder-features-are-neurons-i-2610.24521
来源：elvis · x.com