生产级 AI 智能体的四层上下文工程：Deep Insight 在可运行代码中的实践

When production AI agents handle tasks such as data analysis and report generation, context management can become a major bottleneck. A 200K-token window sounds generous until a real-world task burns through most of it in minutes. Data analysis and report generation leave the agent failing mid-execution, losing accuracy, or driving up costs. A body of published guidance already addresses this, but the individual techniques are easier to read about than to apply. Applied together, they make a measurable difference: in the reference implementation this post walks through, a single write-and-execute tool cuts the tokens each script adds to context by about 95 percent, and loading skills on demand shrinks the system prompt by about 97 percent.

This series organizes that guidance into practices you can apply directly. Anthropic’s engineering posts, together with features built into the Strands Agents SDK, converge on four layers of context engineering: isolate context at the architecture level, control it through prompts, externalize it to the filesystem, and verify it with independent agents and safeguards. In this post (Part 2), you will learn how each layer works and what it looks like in working code. We applied these techniques in Deep Insight, an open source multi-agent data analysis solution built on the Strands Agents SDK and Amazon Bedrock AgentCore, which serves as the reference implementation throughout.

Post series

• Part 1: 5 production challenges in multi-agent systems and how Deep Insight solved them.
• Part 2: Four layers of context engineering for production AI agents: How Deep Insight applies them in working code.
• Part 3: Harness engineering in practice: moving Deep Insight from dev to production. (Coming soon!)

The first post defined five core challenges that production multi-agent systems must solve. It introduced how Strands Agents SDK, Amazon Bedrock and Bedrock AgentCore, and AWS Fargate each address different layers of the problem, with code examples. This second post takes the four layers of context engineering practice that sit on top of that architecture and shows each one implemented. The third post guides you through deploying and operating agents in a secure private network on Amazon Virtual Private Cloud (Amazon VPC), covering everything needed for real-world operations.

What is context engineering?

Context engineering is the discipline of optimizing the information in a large language model (LLM)’s limited context window so agents perform at their best. Anthropic Claude Sonnet 4.5, available on Amazon Bedrock, offers a 200K context window, but complex production tasks quickly exceed it. For example, a seemingly straightforward request like “Analyze sales data and create a DOCX report” runs through several stages: data loading and exploration (5 min), category-based sales analysis (3 min), generating five charts for time-series analysis (10 min), statistical validation (3 min), and final report generation (5 min). Together they consume over 150K tokens across 15 minutes.

Processing such tasks in a single context causes problems: the window overflows mid-task, stale context degrades performance, and errors require re-running the whole pipeline. As context grows, attention to important information drops, and inefficiency raises cost under per-token pricing.

“As the number of tokens in the context window increases, the model’s ability to accurately recall information from that context decreases (Context rot).”

— Anthropic, “Effective Context Engineering for AI Agents” (2025.09)

Context engineering solves exactly these problems, and it is not only a collection of techniques. Spanning architecture design, prompt writing, tool implementation, and verification, it’s a layered discipline where each layer compensates for what the previous ones miss. This post details the four-layer approach you can apply to manage context efficiently.

How to handle tasks that exceed the context window

Anthropic has published several techniques for handling long-horizon tasks. Deep Insight groups these techniques into four layers, with each layer addressing the limitations of the others. The following table sets out the four layers.

The sections that follow take each layer in order, stating what the layer prescribes, why the preceding layers alone are insufficient, and how the practice is implemented in Deep Insight.

1. Isolating context through multi-agent architecture

Give each specialized agent its own context window and pass only compressed results to the orchestrator. Deep Insight does this with a hierarchy of Coordinator, Planner, Plan Reviewer, Supervisor, and Tool Agents (Coder, Validator, Reporter, Tracker, Auditor).

With a single agent, all information, from user requests through data exploration, analysis, chart generation, validation, and reporting, accumulates in one context and easily exceeds the window. Giving each agent its own context keeps even the Coder, which uses the most, under 25K tokens total. Each agent receives only what it needs and signals others with a completion notice and brief summary.

The key to isolation is minimizing shared information. Deep Insight uses a Python dictionary, shared_state, that stores only four things:

• messages: The previous agent’s final message (not the full conversation history).
• clues: Compressed summaries collected from each agent.
• full_plan: A checklist generated by Planner and updated by Tracker.
• history: A brief log of agent names and messages.

The clues use a fixed CLUES_FORMAT: instead of each agent’s full conversation (tens of thousands of tokens), only compressed core results are passed.

# coder_agent_tool.py -- context passing format
CLUES_FORMAT = "Here is clues from {}:\n\n<clues>\n{}\n</clues>\n\n"
RESPONSE_FORMAT = "Response from {}:\n\n<response>\n{}\n</response>\n\n*Please execute the next step.*"

# After Coder completes: add only compressed result to clues
clues += CLUES_FORMAT.format("coder", response["text"])

This pattern is the one recommended in Anthropic’s “How We Built Our Multi-Agent Research System” post, which describes an orchestrator-worker pattern in which a lead agent delegates to specialized subagents, with each agent working only within its own context. You do not need to build it yourself: the agents-as-tools pattern in the Strands Agents SDK provides this isolation at the framework level.

2. Controlling context inflow through prompts

Layer 1 isolation alone is not sufficient: even with independent contexts, an agent that generates long responses inflates the Supervisor’s context. Layer 2 limits each agent’s output through prompts.

For Deep Insight, we use prompts to limit how much information each agent returns to the Supervisor. Three rules do most of the work, as Deep Insight’s prompts show.

“Start with the strongest model and a simple prompt. Then incrementally add specific instructions based on actually observed failure cases.”

— Anthropic, “Effective Context Engineering for AI Agents” (2025.09)

Implementation detail 2-1: Explicit per-agent output token budgets

Each agent’s prompt specifies a maximum response token count, preventing long responses that pollute the Supervisor’s context.

Anthropic’s ” Writing Effective Tools for Agents” post suggests giving tools a CONCISE or DETAILED response option for context efficiency. Apply the same idea one level up, in each agent’s system prompt. In Deep Insight, all agents share one standardized format:

## Status
[SUCCESS | PARTIAL_SUCCESS | ERROR]
## Completed Tasks
- [Task 1 from FULL_PLAN -- use exact wording so Tracker can match]
- [Task 2 -- be specific, not generic]
## Key Insights
- [Key finding 1 with numbers and ratios]
- [Key finding 2 with business implications]
## Generated Files
- ./artifacts/chart.png -- brief description
- ./artifacts/calculation_metadata.json -- N calculations

Implementation detail 2-2: Self-contained code philosophy

The Coder agent’s prompt explicitly states “variables do not persist between scripts.” Every script must include all necessary imports and load data directly. The prompt provides both incorrect and correct examples:

# WRONG -- Attempting to use a variable from Turn 1 in Turn 2
# Turn 1:
df = pd.read_csv('data.csv')
total = df['Amount'].sum()

# Turn 2:
print(f"Total: {total}")  # NameError: name 'total' is not defined

# CORRECT -- Every script is self-contained
import pandas as pd
df = pd.read_csv('./artifacts/cache/df_main.csv')  # Load from cache
total = df['Amount'].sum()
print(f"Total: {total}")

The reason this rule relates to context engineering is that referencing variables from previous scripts requires the agent to “remember” prior conversations, which means past code must remain in context. The self-contained principle means past code can be safely removed from context.

Implementation detail 2-3: Supervisor’s section completion rule

The Supervisor’s prompt includes the rule “Do not move to the next agent until all checklist items in the current agent’s section are marked [x].” We express this rule in the prompt because deciding whether the Coder’s work is complete requires the Supervisor to interpret the plan using information from the Tracker’s updates. This judgment is difficult to express in code alone.

3. “Only pointers in context, actual data in files”

Layer 1 isolates context between agents, and Layer 2 limits output within agents. But when the Coder agent generates 100 lines of Python code, that code itself enters the context. Even with “respond briefly” prompt instructions, some code can’t be written briefly. Layer 3 uses tool design to move heavy content outside the context, into the filesystem. Four techniques carry most of the benefit, as the following implementation details show.

Implementation detail 3-1: Moving code outside context with Write and Execute Tool

The Coder reads the analysis plan, writes Python, and executes it to generate visualizations. But when the code gets long, the conventional approach (separate file-write and execute tools) adds 100 lines directly to context per script, accumulating over 1,000 lines.

The fix is to collapse the two steps into a single tool, so the code never enters context as a message. Deep Insight’s Coder agent uses one write_and_execute_tool for this.

Here’s part of the write_and_execute_tool. After writing and executing the file, it returns only a summary: “Wrote 105 lines, execution successful, completed analysis of five categories.” The code stays in the file and about three lines of summary remain in context. Based on this example, that saves about 95 percent of the tokens.

# write_and_execute_tool.py
def _handle_write_and_execute_tool(file_path, content, timeout=300):
    """Write a Python script and immediately execute it."""
    # Step 1: Write file
    with open(file_path, 'w', encoding='utf-8') as f:
        f.write(content)
    # Step 2: Execute immediately
    result = subprocess.run(
        f"python {file_path}",
        shell=True,
        capture_output=True,
        timeout=timeout
    )
    # Return only summary
    return f"Written {len(content.split(chr(10)))} lines to {file_path}\n" \
        f"Execution successful\n" \
        f"Output: {result.stdout[:500]}"  # Only first 500 chars

“Server-side filtering: filtering data at the execution layer before it reaches context, 98.7% token reduction (150K to 2K).”

— Anthropic, “Code Execution with MCP” (2025.11)

Implementation detail 3-2: Reusing common logic as modules in the Coder agent

Common modules get reused across analyses. Rather than regenerating them each time, define frequently-used functions as a module once and import them, which prevents token waste and context duplication.

Without this, defining functions like to_python_type() or track_calculation() in every analysis produces 200–300 lines of duplicate code across 10 analyses. Deep Insight generates a module called coder_analysis_utils.py before the first task. It includes functions for converting numpy types, tracking calculation metadata, and saving to JSON. Subsequent analyses import it, replacing 20–30 regenerated lines with two import lines, which saves tokens and improves consistency and debugging.

# coder_analysis_utils.py -- common module generated before first task
import numpy as np
import json
from datetime import datetime

_calculations = []

def to_python_type(value):
    """Convert numpy/pandas types to Python native types"""
    if isinstance(value, (np.integer, np.int64)): return int(value)
    elif isinstance(value, (np.floating, np.float64)): return float(value)
    elif isinstance(value, np.ndarray): return value.tolist()
    return value

def track_calculation(calc_id, value, description, formula,
                      source_file, source_columns, importance):
    """Track calculation metadata -- Validator independently verifies later"""
    _calculations.append({
        "id": calc_id, "value": to_python_type(value),
        "description": description, "formula": formula,
        "source_file": source_file, "source_columns": source_columns,
        "importance": importance
    })

def save_calculation_metadata(path="./artifacts/calculation_metadata.json"):
    """Save all tracked calculations to JSON"""
    with open(path, 'w', encoding='utf-8') as f:
        json.dump({"generated_at": datetime.now().isoformat(),
                   "calculations": _calculations}, f, ensure_ascii=False, indent=2)

Implementation detail 3-3: Saving and sharing analysis results through structured note-taking

We use structured note-taking to share detailed analysis results without passing them through each agent’s context. Deep Insight stores these results in a file called all_results.txt, which is then read by agents as needed.

For example, the Coder agent appends results to all_results.txt after each analysis task:

with open('./artifacts/all_results.txt', 'a', encoding='utf-8') as f:
    f.write(f"""
## Category Sales Analysis (completed at: 14:32)
- Top revenue category: Fruit (417,166,008 KRW, 45% of total)
- Top 3 categories: Fruit, Vegetables, Meat (78% of total)
- Generated file: ./artifacts/category_sales_chart.png
""")

The Coder records detailed results in this file after each analysis, and the Reporter reads it when writing the final report. Other agents collaborate through all_results.txt the same way.

Without note-taking, the Coder passes detailed results to the Supervisor after every task, accumulating in its context. With note-taking, only an approximately 30-token message like “Category sales analysis complete, 3 key insights found, detailed results saved to all_results.txt” is passed, and the Reporter reads the file when needed.

The pattern generalizes beyond a single file. The filesystem itself can serve as the inter-agent communication bus, as it does in Deep Insight:

“The”just in time” approach, dynamically loading data into context at runtime rather than pre-processing all relevant data up front, keeps context fresh, relevant, and adaptive to the evolving state of the task.”

— Anthropic, “Effective Context Engineering for AI Agents” (2025.09)

Implementation detail 3-4: Loading only needed knowledge with Skills

Rather than putting all specialized knowledge into the system prompt, this approach loads it only when needed. Including every skill guide (data structure checking, README generation, skill creation, system prompt writing, tool creation) would add over 20K tokens, load unnecessary information into the Coder agent on every call, and force a redeploy to add a skill.

Deep Insight’s Skills system loads only a skill catalog of about 500 tokens at startup, with the skill list holding only names and brief descriptions. When a user requests “Create a README file,” the agent decides “I need the readme-generator skill” and calls skill_tool(skill_name="readme-generator"). The SkillLoader reads readme-generator/SKILL.md and returns a detailed guide of about 3K tokens.

# 1. Skill Discovery (at initialization)
from src.utils.skills.skill_utils import initialize_skills
available_skills, skill_prompt = initialize_skills(
    skill_dirs=["./skills"],
    verbose=True
)

# 2. Add to System Prompt (~500-token catalog)
system_prompt = base_prompt + skill_prompt

# 3. Runtime -- Agent loads skill when needed
# Agent decides: "This is README generation, I need the readme-generator skill"
# -> skill_tool(skill_name="readme-generator")
# -> Returns full SKILL.md content (~3K tokens)

Each skill is defined as a SKILL.md file composed of YAML frontmatter and detailed markdown guide. It includes metadata like name, description, license, and allowed-tools, along with Overview, When to Use This Skill, detailed instructions, and Best Practices. To add a new skill, create a new directory under skills/ and write a SKILL.md file, and it is automatically discovered.

At startup, the agent loads a skill catalog of about 500 tokens instead of wasting over 20,000 tokens on full guides (about 97 percent less). When needed, it can load an additional few thousand tokens’ worth of detailed instructions for skills.

4. Verifying quality with verification agents and safeguards

Even with Layers 1–3, one risk remains: isolated agents can’t see each other’s work, so if the Coder miscalculates, the Reporter carries that error into the report. Layer 4 is the final line of defense.

“Provide testing tools so the agent can self-verify features and fix bugs that aren’t obvious from the code alone. Have the model commit its progress to git with descriptive commit messages and write progress summaries in a progress file.”

— Anthropic, “Effective Harnesses for Long-Running Agents” (2025.11)

Implementation detail 4-1: Tracker, Validator, and Auditor agents: progress tracking and quality assurance

Tracker updates the plan’s checklist in real time. When the Coder completes data loading, the Supervisor calls Tracker, which updates the plan item from [ ] to [x] based on information like “Data load complete (1,000 rows, 5 columns).” This helps both users and the system know exactly where progress stands.

Validator independently re-executes the Coder’s calculations to verify accuracy. It reads calculation_metadata.json, re-performs each calculation from source data, and compares: matches within 0.01 pass, mismatches are flagged. Verified calculations go to citations.json for the Reporter to cite. After the Reporter finishes, the Auditor checks the final DOCX against citations.json and calculation_metadata.json without modifying them. It records citation defects in audit_findings.json and can request up to two targeted retries from the Reporter.

Implementation detail 4-2: Prioritizing calculations for verification

Verifying every calculation is inefficient, so scope the verification by importance. Deep Insight’s OptimizedValidator applies the following thresholds:

• Over 50 calculations: aggressive filtering, verify all high plus only 10 medium.
• Over 20 calculations: moderate filtering, verify all high plus 15 medium.
• 20 or fewer calculations: verify most, all high plus all medium plus 5 low.

It also applies type-safe comparison: to avoid numpy-versus-Python type issues, all values are converted to float() and checked for an absolute difference under 0.01:

# validator.md -- type-safe numerical comparison
try:
    match = abs(float(expected) - float(actual)) < 0.01
except:
    match = str(expected) == str(actual)

Implementation detail 4-3: SummarizingConversationManager: the last safety net

Context can still overflow in exceptional cases, so keep a safety net under the other layers. Deep Insight uses the SummarizingConversationManager in the Strands Agents SDK for this.

# strands_sdk_utils.py -- safety net configuration at agent creation
agent = Agent(
    model=llm,
    system_prompt=system_prompts,
    tools=tools,
    conversation_manager=SummarizingConversationManager(
        summary_ratio=0.5,  # summarize oldest 50% on overflow
        preserve_recent_messages=10,  # always preserve 10 most recent messages
        summarization_system_prompt=apply_prompt_template("summarization", {})
    )
)

On overflow, the LLM summarizes the oldest 50 percent of messages into one, while always preserving the 10 most recent to keep the current task’s context. The summarization prompt (summarization.md) requires clues, plans, and tracking status to be preserved verbatim.

Implementation detail 4-4: Selective prompt caching for cost optimization

Context engineering covers cost optimization, not only token savings. Enable prompt caching per agent, based on how often each is called with the same prompt. Deep Insight’s per-agent settings:

Agents with caching enabled insert cache points into system prompts and tool definitions, reducing input token costs by up to 90 percent on repeated calls. This is a feature natively supported by the Strands Agents SDK. Plan Reviewer does not appear in the table because it’s a human-in-the-loop step that collects user approval or feedback rather than calling a model.

Conclusion

This post has detailed the four-layer engineering approach you can use to efficiently manage context for complex tasks that exceed the context window.

• Layer 1 (architecture-level isolation): Each agent works in an independent context through multi-agent architecture, passing only compressed summaries.
• Layer 2 (prompt-based control): Output token budgets, standardized response formats, and self-contained code philosophy actively limit context inflow.
• Layer 3 (filesystem use): write_and_execute_tool keeps code outside context, the filesystem serves as an inter-agent communication bus, and knowledge is lazy-loaded only when needed.
• Layer 4 (verification and safeguards): Validator’s independent calculation re-verification, Auditor’s read-only citation audit of the final report, Tracker’s progress tracking, SummarizingConversationManager’s overflow prevention, and selective prompt caching manage both quality and cost.

Each layer forms a defense system that compensates for what the preceding layers miss. When architecture isolation is insufficient, prompts limit output. When prompts are insufficient, tools keep content external. And when problems still arise, verification catches them.

Each of these layers is documented in Anthropic’s engineering posts, “Effective Context Engineering for AI Agents”, “Code Execution with MCP”, and “How We Built Our Multi-Agent Research System”. Much of what they recommend already ships in the Strands Agents SDK rather than being something you build: SummarizingConversationManager, prompt caching, and the agents-as-tools pattern.

When building production agents, the LLM’s context window alone is rarely enough. Apply these context engineering techniques to manage context efficiently and scale your agents for production workloads.

Deep Insight, the reference implementation used here, is open source. Review the code and extend it for your domain.

• GitHub
• Workshop

About the authors

Yoonseo Kim

Yoonseo is a Solutions Architect who helps customers solve business challenges using AWS technologies. With a strong interest in AI/ML, she focuses on helping customers design and optimize agentic AI workloads, along with supporting infrastructure and evaluation environments.

Gonsoo Moon

Gonsoo is an AI/ML Specialist Solutions Architect at AWS. Drawing on experience across diverse AI/ML use cases and production deployments, he works closely with customers to solve their AI/ML challenges. He also shares AI/ML knowledge with data scientists, developers, and analysts, aiming to contribute to positive change in Korea and around the world.

Jiyun Park

Jiyun is a Solutions Architect who helps customers in the software and internet industries migrate their workloads to the cloud. She works with customers to design cloud solutions that help them achieve their business goals.

Chloe Kwak

Chloe is a Senior Specialist Solutions Architect for Amazon Connect Decisions in the Applied AI Solutions organization, supporting customers across APJ. She spent several years on the Korea Retail team, where she helped retail and e-commerce customers modernize on AWS and put AI/ML into production. Today she helps customers apply agentic AI to demand and supply planning, working hands-on from data preparation and forecast validation through to production rollout.

Jesam Kim

Jesam is a Solutions Architect who helps enterprise customers adopt cloud technologies and solve business challenges. With a focus on AI/ML, including recommendation systems and generative AI, he helps customers design architectures that meet their business needs.

Kyutae Park, Ph.D.

Kyutae is an AI Specialist Solutions Architect who helps customers across industries design and build applied AI workloads, drawing on extensive experience in machine learning, MLOps, and agentic AI. He focuses on intelligent automation and operational innovation through agentic AI, bringing autonomous decision-making and multi-agent collaboration architectures into real-world business environments. Through this work, he helps customers use AWS AI/ML services and applied generative AI to achieve tangible business outcomes and build a sustainable competitive advantage.