r/MachineLearning· /u/hansenjakob·· 4 小时前AI 评分45
模型在 RL 中究竟学到了哪些行为？用稀疏自编码器一探究竟
What behaviors does a model actually learn in RL? [P]
AI 导读
有研究者用稀疏自编码器提取特征，对比训练前后模型轨迹的差异，以无监督方式发现模型在 post-training 中学到的行为。该方法应用于 MiMo-v2.6 的 RL 环境轨迹后，检测到明显的 eval awareness 以及轻微的环境利用行为。
正文
I've been working on unsupervised tooling to discover what behaviors a model learns in post-training. The core idea is to use features from sparse autoencoders to identify differences in traces generated before and after training. I applied this to traces from the MiMo-v2.6 RL environments and it found significant eval awareness as well as some mild environment exploitation. Writeup is here. Would love to hear any feedback or questions!
submitted by /u/hansenjakob
[link] [留言]
来源：r/MachineLearning · reddit.com