Chapter 12 · Durability: Persistence and Recovery

Chapter goals: Understand Flue's "accepted-work contract" — once an input is admitted, the runtime owes that session a persistent final state; master the recovery-replay mechanism and the @flue/postgres persistence adapter.

12.1 The Accepted-Work Contract

Durability's core promise is one sentence: every input that arrives at an agent — direct HTTP prompt, dispatch(...) call, channel delivery, scheduled trigger — once admitted, the runtime must deliver a persistent final-state result even after process crash, restart, or redeploy.

submission → admitted → executing……💥crash → restart → recovery replay → final state ✅

// dispatch returns "admitted", not the execution result
// This means the caller can return immediately; the runtime guarantees eventual completion
const submission = await dispatch(triage, [
  { role: 'user', content: 'Triage issue #42' },
]);
console.log('Admitted:', submission.id); // Will continue even after进程 restart

12.2 What Happens During Recovery

After an interruption, recovery replays the necessary work to bring the session to its final state:

// Pseudocode understanding of recovery semantics
// 1. Completed LLM turns: read from persisted state; no double-billing
// 2. In-progress turns: replay that turn from scratch
// 3. Durable tool calls: decide whether to replay based on the tool's idempotency declaration

type RecoveryPlan = {
  completedTurns: number;   // Turns that need no redo
  replayFrom: number;       // Replays from this turn onward
};

Two specialmember categories have different recovery behavior:

• Durable tools: tools declared as durable are not blindly re-executed on recovery;
• Delegated tasks (sub-agent tasks): sub-tasks are also "admitted work" with their own独立 recovery chain.

12.3 Where State Is Stored: Database and Postgres

Persisted state lives in the Database layer. Use the built-in default during development; switch to @flue/postgres for production:

npm install @flue/postgres pg

// src/db.ts
// Production-grade persistence: Postgres adapter
import { postgresAdapter } from '@flue/postgres';
import { Pool } from 'pg';

export const db = postgresAdapter({
  pool: new Pool({
    connectionString: process.env.DATABASE_URL!, // e.g. postgres://user:pass@host:5432/flue
    max: 10,
  }),
});

// src/server.ts
// Mount the adapter onto the runtime; all session state auto-persisted
import { createApp } from '@flue/runtime/node';
import { db } from './db.ts';

const app = createApp({
  database: db, // Sessions, admission records, recovery points all written to DB
});

app.listen(3000);

Why not SQLite?

Edge targets like Cloudflare Workers have no local filesystem — SQLite is naturally unavailable. Postgres is the most universal cross-target choice — the same adapter works for both local dev and production.

12.4 Recovery Behavior per Deployment Target

"How recovery happens after restart" depends on where you run:

Target	Crash Scenario	Recovery Trigger
Node.js long-lived process	Process killed / OOM	Scans unfinished admissions on restart and resumes
Cloudflare Workers	Isolate reclaimed	Resumes from DB on next request/cron wake
GitHub Actions job	Runner timeout or failure	Workflow retry policy + DB progress接力

12.5 Chapter Summary

• Durability = "accepted-work contract": once admitted, the runtime guarantees eventual completion even after crash/restart;
• Recovery replays only what's necessary: completed turns from state, in-progress turns from scratch;
• Durable tools are not blindly re-executed; sub-agent tasks have independent recovery chains;
• @flue/postgres is the production adapter; works across all deployment targets.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What does "admitted" mean in Flue's durability model?

2. During recovery, what happens to completed LLM turns?

3. Why is Postgres preferred over SQLite for production?

4. What is a "durable tool" in the recovery context?

🛠️ Hands-On Exercises

1. Set up a Postgres database; connect it via @flue/postgres; dispatch a long-running task; kill the process mid-execution; restart and verify recovery.
2. Write a durable tool (idempotent POST); verify it is not re-executed during recovery.
3. Test recovery on a simulated Cloudflare Worker environment (use the in-memory adapter first, then swap to Postgres).

Durability understood. Next, Observability: OpenTelemetry integration.