elvis· @omarsar0 · X·· 2 小时前AI 评分51
AI 导读
Elvis Saravia 认为 LLM 智能体偏好结构化数据，context graphs 在复杂代码库中效果明显，Harvey 的法律知识任务报告显示其应用范围更广。他引用的 Harvey 数据称，智能体在无评测任务和评分标准的情况下为文档构建 context graphs 后，Firm Knowledge 任务从 65% 提升到 80%，Precedent lists 从 67% 到 80%，Matter counting 从 53% 到 83%，同时成本降低 48%。他表示 DAIR.AI 正将这一思路用于研究类任务。
正文
LLM-based agents love structured data.
I've seen this work really well, but mostly in complex codebases. But I think context graphs have broader applications, as shown in this Harvey report on legal knowledge tasks.
We are working on this at @dair_ai for research-related tasks. More to share soon.
We built context graphs to help agents work through long-horizon legal tasks. Starting with Legal Agent Bench tasks in 11 categories like M&A diligence and contract negotiation, we had agents build context graphs of the underlying documents (without access to the eval tasks or grading rubrics). Context graphs dramatically improved agent performance on LAB tasks: - Firm Knowledge tasks: 65% → 80% - Precedent lists tasks: 67% → 80% - Matter counting tasks: 53% → 83% They also boost agent efficiency: we found a 48% cost reduction for agents given access to a context graph to assist their work. Full video below - more to come soon.在 X 查看被引用的帖子
来源：elvis · x.com