Lisan al Gaib· @scaling01 · X·· 3 小时前AI 评分30
AI 导读
ECI 推出一周年了 在此之前我有自己的 Index，是基于 Elo 的。 但 ECI 发布后，我认可了 IRT 方法论的优势，便停止了自己 Index 的工作 虽然我当时考虑过采用这套方法论，但没这么做，因为不够新颖
正文
The ECI was introduced one year ago
Before that I had my own Index which was based on Elo.
But when the ECI launched I recognized the supremacy of the IRT methodology and stopped working on my own Index
Although I was thinking about adopting the methodology, I didn't do it because it wasn't novel enough
I'm back and Gemini 2.5 Pro is still the king (no glaze) I did some more manual data cleaning and scrapped the shitty "average scaled score" and replaced it with Glicko-2 rating system with params: INITIAL_RATING = 1500 INITIAL_RD = 350 INITIAL_VOL = 0.06 TAU (τ) = 0.5 Furthermore I increased the minimum number of appearances from 5 to 10 benchmarks to make it more stable. The labels show the lower 95% ratings (a conservative lower skill estimate) and in brackets the number of benchmarks the model appeared in. Below this post I attached the full table with mu, sigma, lower 95% ratings and number of appearances.在 X 查看被引用的帖子
来源：Lisan al Gaib · x.com