跳到正文
r/MachineLearning· /u/Pale-Training-6944·· 2 小时前AI 评分61

Gear:面向 RSI 的开源元学习框架,提供 GEPA 变体与基准对比

I built a scikit-learn for RSI [D]

AI 导读

作者发布开源框架 Gear,用元智能体根据任务智能体的执行轨迹和评估反馈改进提示词、工具、工作流与上下文管理,目标是把 GEPA、RRSI、Harness² 等方法做成类似 scikit-learn 的统一实现与比较平台。

正文
I built a scikit-learn for RSI [D]

After reading a bunch of RSI papers, I keep coming back to familiar problems from RL and evolutionary search:

  1. Sampling: How do we find promising changes efficiently?
  2. Evaluation: How do we know a new variant actually improves the agent?
  3. Diversity: How do we avoid converging too early?

That motivated us to build Gear, an open-source, meta-learning-inspired framework. A task agent produces execution traces; a meta-agent uses those traces and evaluation feedback to improve prompts, tools, workflows, and context management.

Gear separates search decisions from execution and evaluation. Our goal is a scikit-learn for RSI, making approaches such as GEPA, RRSI, and Harness² easier to implement and compare. We currently have a working GEPA variant with shared failure diagnosis and staged evaluation.

With GPT 5.6 Luna held at medium reasoning effort, we observed these task pass rates:

Benchmark Base DSH Gear-optimized DSH
AutomationBench Marketing 27.0% 40.0%
AutomationBench Support 17.0% 25.0%
Terminal-Bench 2.1 52.8% 62.9%

At max effort on Marketing, Luna with the optimized harness reached 88.88% objective completion, versus 84.08% for Astra + Codex. Strict task pass rates were 53% versus 57%, so the advantage was in partial completion.

Separately, with both models using the same optimized DSH at max effort, Luna’s estimated API cost averaged $0.0568 per task versus $1.1858 for Astra—95.21% lower, excluding optimization and infrastructure costs.

We’d welcome feedback on candidate evaluation and diversity preservation under limited budgets.

Code · Paper

submitted by /u/Pale-Training-6944
[link] [留言]

来源:r/MachineLearning · reddit.com