agent-token-optimization-patterns

The saved excerpt is a snapshot from review. The external source remains the complete and most current version.

---
name: agent-token-optimization-patterns
description: Use when designing or reviewing AI agent systems for cost and latency efficiency — covers context engineering (write/select/compress/isolate), prompt caching strategies, model routing by complexity, token budget management, context window techniques, prompt compression, and anti-patterns with TypeScript and Python examples
---

# Agent Token Optimization Patterns

## Overview

Token = cost + latency. Every token you can save without losing quality is pure value — lower bills, faster responses, and more headroom before context limits bite. At scale, unoptimized agents burn 3-10× more tokens than necessary, primarily through context stuffing, wrong-model routing, and cache thrashing.

Token optimization is not about cutting corners. It is about precision: sending exactly the information the model needs, in exactly the right form, to the right model, at the right time.

## Quick Reference

| Technique | Token Savings | Implementation Complexity | Risk |
|-----------|--------------|--------------------------|------|
| Prompt caching (stable content first) | 45-80% on cache hit | Low | Cache thrashing if content rotates |
| Model routing (haiku for simple tasks) | 60-90% cost reduction | Medium | Quality degradation on misrouted tasks |
| Structured data over prose | 20-40% | Low | Schema design overhead |
| Progressive summarization | 30-60% | Medium | Lossy compression of earlier context |
| System prompt deduplication | 10-30% | Low | Divergence if copies drift out of sync |
| Context slicing (role-based) | 20-50% | Medium | Missing context if slices are too narrow |
| Sliding window (drop oldest turns) | Variable | Low | Loss of early conversation context |
| Spawn new agent vs stuff context | High (resets window) | High | Coordination overhead, handoff cost |

## Context Engineering

Context engineering is the systematic practice of controlling what goes into an agent's context window. The framework has four operations: **Write, Select, Compress, Isolate**.

### Write — Craft Precise Prompts

Write prompts that express the task in minimum tokens without ambiguity. Prefer imperative verbs over explanatory prose. Replace "Could you please help me understand..." with "Explain:". Remove politeness markers, hedges, and meta-commentary — the model does not need them.

```typescript
// WRONG: verbose and hedging
const verbose = `
  I was wondering if you could help me take a look at the following code
  and maybe identify any potential issues that might be present in it.
  Please be thorough but also concise in your response if possible.
`;

// CORRECT: imperative, direct
const precise = `Review this code. List issues by severity (critical/high/medium/low). One line each.`;
```

**Rules:**
- One instruction per sentence
- State the output format explicitly (JSON, numbered list, one line per item)
- Omit background context the model already has from its training data
- Remove examples unless the task is genuinely ambiguous without them

### Select — Choose Relevant Context

Do not inject the entire codebase or conversation history. Select only what is directly relevant to the current task.

```python
def select_context(files: list[str], task_keywords: list[str]) -> list[str]:
    """Return only files that contain at least one task keyword."""
    relevant = []
    for path in files:
        content = read_file(path)
        if any(kw.lower() in content.lower() for kw in task_keywords):
            relevant.append(path)
    return relevant[:5]  # Hard cap: never inject more than 5 files per task
```

**Selection heuristics:**
- For code tasks: include the file being changed + direct imports only
- For Q&A tasks: include the most recent N turns, not the full history
- For multi-step workflows: pass only the output of the previous step, not all prior steps
- Use semantic similarity search to rank and trim candidate context chunks

### Compress — Reduce Token Count

When context can