Agent Memory Leaderboard 八个系统并列 52.67% 榜首，结果表该怎么呈现？

When eight systems share a benchmark top score, what should the table show?[D]

AI 导读

Agent Memory Leaderboard 8 月公开榜的 Cycle 1 编码榜上，43 个开源方法在 150 个任务上的解决率介于 46.00% 至 52.67%，其中 8 个系统并列 52.67% 的最高分。由于单个任务价值 0.67 个百分点，整个头部集群的差距仅为一个任务。作者由此讨论原始计数、分档分组与区间估计等表格呈现方式，并询问读者在论文或内部评测中更认可哪种。

正文

I have been thinking about how result tables get read when the interesting part is how close everyone is.

The Agent Memory Leaderboard's public board from August is a nice case. On the Cycle 1 coding board, Open-source Methods, 43 listed systems run from 46.00% to 52.67% solve rate over 150 tasks, meaning 69 to 79 tasks solved, and eight systems sit together at 52.67%. Since one task there is worth 0.67 percentage points, the whole top cluster moves by a task at a time.

Honestly I like seeing this. My read is that a lot of different designs land in the same neighbourhood on these coding tasks, and convergence like that is informative in itself. It also means the table format carries real weight. Options I have seen or used: raw counts beside percentages, tier groupings instead of strict ranks, and interval estimates across tasks.

If you have had to report scores this tight in a paper or internal eval, which of these did your readers actually engage with, and what did they take away from it?

submitted by /u/_sRhubarb_
[link] [留言]