Chapter 2 · What Exactly Is a Harness
Chapter goals: Master the precise definition of a harness — the five-subsystem model — understand the core principle of "the repo IS the spec," and learn to distinguish a harness from a simple prompt file.
2.1 An Overused Word
"Harness" is used constantly in the AI coding agent space, but most of the time what people call a "harness" is just a prompt file. A prompt file is not a harness.
This chapter gives harness a precise, actionable definition — not an academic abstraction, but a framework you can use today. A complete harness consists of five subsystems: instructions, tools, environment, state, and feedback. Each subsystem has a clear responsibility and evaluation criteria.
2.2 Starting from an Analogy
Imagine you're a new engineer thrown into a project with zero documentation. No README, no comments in the code, nobody tells you how to run tests, CI config is buried somewhere. Can you write good code? Maybe — if you're smart enough and patient enough. But you'll spend a lot of time "figuring out what this project is" instead of "solving the problem."
AI agents face the exact same dilemma, and it's even worse. At least you can ask colleagues. An agent can only see the files you put in front of it and the commands it can execute.
OpenAI expresses the core principle of harness engineering as "the repo IS the spec" — all necessary context should exist in the repo, delivered through structured instruction files, explicit verification commands, and a clear directory layout. Anthropic's long-running agent docs emphasize state persistence, explicit recovery paths, and structured progress tracking. The two companies focus on different aspects, but they're saying the same thing: the engineering infrastructure outside model weights determines how much of the model's capability can actually be realized.
Look at some tools you're already familiar with:
Claude Code embodies harness thinking. It reads CLAUDE.md from the repo, can run shell commands, executes in the local environment, maintains session history, and can run tests to check results. But if you don't tell it how to run tests, it can't verify it did the right thing.
Cursor follows similar logic. .cursorrules is its instruction source, the terminal is its toolset, it can read project structure and lint config. However Cursor's state management is relatively weak — close the IDE and reopen, and the previous context is gone.
Codex (OpenAI's coding agent) uses git worktrees to isolate each task's runtime environment, paired with a local observability stack (logs, metrics, traces), so every change is verified in an independent environment. It performs far better in repos with AGENTS.md and clear verification commands.
AutoGPT is the counter-example. Lack of structured state management leads to unbounded context accumulation during long tasks; lack of precise feedback mechanisms causes the agent to fall into loops. Many people say AutoGPT "doesn't work," but actually it's the harness that doesn't work.
2.3 The Five-Subsystem Model
┌─────────────────────────────────────────────────────┐
│ AI Agent │
└───────────────┬─────────────────────────────────────┘
│
┌───────────┼───────────┬───────────┬───────────┐
↓ ↓ ↓ ↓ ↓
┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐
│Instructions│ │ Tools │ │Environment│ │ State │ │ Feedback │
│Subsystem │ │Subsystem│ │Subsystem │ │Subsystem│ │Subsystem │
│ │ │ │ │ │ │ │ │ │
│AGENTS.md │ │shell │ │deps │ │PROGRESS│ │test │
│CLAUDE.md │ │files │ │versions│ │DECISIONS│ │lint │
│rules │ │tests │ │docker │ │commits │ │check │
└────────┘ └────────┘ └────────┘ └────────┘ └────────┘2.3.1 Instructions Subsystem (Instructions)
Create AGENTS.md (or CLAUDE.md), containing: project overview and goals, tech stack and versions, first-run commands, non-negotiable hard constraints, links to detailed docs.
# AGENTS.md
## Project Overview
User preference management system, FastAPI + PostgreSQL + Redis.
## Tech Stack
- Python 3.11+, FastAPI 0.100+, SQLAlchemy 2.0
- PostgreSQL 15, Redis 7
- pytest, mypy --strict, ruff
## First Run
```bash
./init.shHard Constraints
- All API endpoints must go through OAuth 2.0 auth
- Using legacy SQLAlchemy 1.x syntax is forbidden
- New code must pass pytest and mypy
### 2.3.2 Tools Subsystem (Tools)
Make sure the agent has sufficient tool access. Don't disable the shell for "security reasons" — if the agent can't even run `pip install`, how can it possibly complete the task? But don't open everything either — follow the principle of least privilege.
### 2.3.3 Environment Subsystem (Environment)
Make the environment self-describing. Use `pyproject.toml` or `package.json` to pin dependencies, `.nvmrc` or `.python-version` to specify runtime versions, Docker or devcontainers to make the environment reproducible.
### 2.3.4 State Subsystem (State)
Long tasks must have progress tracking. Use a simple `PROGRESS.md` file to record: done, in progress, blocked. Update before each session ends; read at the start of the next session.
### 2.3.5 Feedback Subsystem (Feedback)
This is the subsystem with the highest ROI. List verification commands explicitly in `AGENTS.md`:
```markdown
## Verification Commands
- Tests: pytest tests/ -x
- Types: mypy src/ --strict
- Lint: ruff check src/
- Full: make check (includes all of the above)If any of the five subsystems is missing, the harness is incomplete and the agent will always feel awkward to use.
2.4 Quantifying the Value of Harness Components
Want to know which component currently adds the most value? Use the "controlled-variable exclusion method": keep the model fixed, remove each of the five subsystems one at a time, and observe which removal causes the biggest performance drop. The component whose removal hurts the most has the highest marginal contribution for the current task and deserves priority investment.
Whether you decide to strengthen or defer, first check the failure logs and attribution: was the task unclear? Was context insufficient? Was the environment non-reproducible? Was verification feedback missing? Was state management broken? Component ablation results should only serve as supporting evidence.
2.5 A Real-World Case
A team used GPT-4o to develop a TypeScript + React frontend app (~20,000 lines of code). They went through four phases, each本质ually adding one harness component:
Phase 1: README had only a basic project description. 1 out of 5 runs succeeded (20%). Main failures: picked the wrong package manager (npm vs yarn), didn't follow component naming conventions, couldn't run tests.
Phase 2: Added AGENTS.md specifying tech stack versions, naming conventions, key architecture decisions. Success rate rose to 60%. Remaining failures mainly came from environment issues and missing verification.
Phase 3: Added init.sh to standardize startup, PROGRESS.md to track progress, explicit verification commands. Success rate rose to 85%.
Phase 4: Added DECISIONS.md to record design decisions, git worktrees to isolate tasks. Success rate stabilized at 95%+.
They didn't change the model. They changed the harness.
2.6 Chapter Summary
- Harness ≠ prompt file; it is all engineering infrastructure outside model weights
- Five subsystems: instructions, tools, environment, state, feedback
- "The repo IS the spec": information the agent can't see is equivalent to non-existent
AGENTS.mdshould be a table of contents, not an encyclopedia — about 100 lines- Constrain, don't micromanage: use executable rules to constrain the agent, don't enumerate instructions one by one
- Quantify value: remove subsystems one by one, observe performance changes
🧪 随堂测验
点击你认为正确的选项。答错时会展示正确答案与原因解析。
1. According to OpenAI's formulation, what does the core principle 'the repo IS the spec' mean?
2. Which tool is described as the counter-example of 'lack of structured state management leading to unbounded context accumulation'?
3. Among the five harness subsystems, which has the highest ROI?
4. What is the ideal length for an AGENTS.md?
🛠️ Hands-on Practice
- Analyze a tool you're familiar with (Claude Code / Cursor / Codex) and identify what each of its five subsystems corresponds to.
- Create a minimal harness for an empty repo: AGENTS.md + init.sh + verification commands.
- Run a controlled experiment: same task, with "README only" vs. "complete five-subsystem harness," record differences in success rate and context efficiency.