跳到正文
elvis· @omarsar0 · X·· 3 小时前AI 评分59
AI 导读

Google Cloud AI Research 提出 OO-Harness,在同一个 Gemini 3.5 Flash executor 下把 AutomationBench 从 61.6% 提升到 77.4%,并在所测三个基准上均优于 Meta-Harness。

正文

Build specialized harnesses, folks.

This is a banger paper from Google Cloud AI Research on self-evolving agent harnesses.

(bookmark it)

They introduce OO-Harness, which raises AutomationBench from 61.6% to 77.4% with the same Gemini 3.5 Flash executor.

It also beats Meta-Harness on all three benchmarks they test.

More details below.

Instead of evolving one global harness for every task, an evolution agent builds a class hierarchy of specialized harnesses from execution records.

The design has three parts.

Subclasses inherit the base harness and override only the methods a task cluster needs
Design lessons record which changes helped and which ones failed
A router reads the task and the base run, then picks a subclass or keeps the base result

On Workspace-Bench, the score goes from 25.5% to 39.5%, and on Toolathlon-GYM strict pass goes from 31.2% to 37.5%. Spending extra inference on best-of-K sampling or self-refinement does not reliably help.

Inheritance also improves results. On Workspace-Bench, harnesses built as subclasses score 4.96 points higher than the same evolved code copied into flat classes.

https://arxiv.org/abs/2610.28380

Chat with Paper: https://academy.dair.ai/papers/object-oriented-harness-evolving-agent-harnesses-through-object-oriented-design-2610.28380

来源:elvis · x.com