Chapter 10 · Memory System: Short-term, Long-term, and Entity Memory

Chapter goals: Understand CrewAI's unified memory system design (and how it evolved from the old short-term/long-term/entity three-class memory), master embedder third-party integration, learn memory write/retrieval timing and cleanup, and verify memory's impact on multi-turn task quality through experiments.

10.1 What Problem Does Memory Solve?

By default, every kickoff() is "amnesiac": conclusions learned in the last run, preferences the user shared — all lost. The memory system gives crews cross-task accumulation capability.

Version evolution note (important): Early CrewAI (0.x era) split memory into three categories — short-term memory (in-run context), long-term memory (cross-run persistent task experience), and entity memory (structured facts about people/things/organizations). The current version (1.x) has upgraded to a unified Memory class: a single API covering storage, retrieval, scope management, and deduplication/merging. The old category concepts are retained only in CLI cleanup commands and compatibility layers. This tutorial follows the latest unified model.

10.2 Quick Start: Enable Memory in a Crew

import os

from crewai import Agent, Crew, Process, Task, LLM, Memory

llm = LLM(
    model="openai/deepseek-chat",
    base_url="https://api.deepseek.com/v1",
    api_key=os.getenv("DEEPSEEK_API_KEY"),
    temperature=0.7,
)

# Option 1: one-line enable with default config
crew = Crew(
    agents=[...],
    tasks=[...],
    process=Process.sequential,
    memory=True,          # crew auto-creates a default Memory and applies it across all tasks
)

# Option 2: pass a tuned Memory instance
memory = Memory(
    recency_weight=0.4,            # weight for recency (newer = more important)
    semantic_weight=0.4,           # weight for semantic similarity
    importance_weight=0.2,         # weight for content importance
    recency_half_life_days=14,     # memory "half-life" in days
)
crew2 = Crew(agents=[...], tasks=[...], memory=memory)

Automatic behavior after enabling (per official docs):

• After each task: the framework uses the LLM to extract discrete facts from the task output and stores them in memory;
• Before each task: the agent automatically recalls memory semantically related to the current task and injects it into the task prompt.

In other words, both memory writing and retrieval are automatic — you don't need to write "please remember this" prompts by hand.

10.3 Unified Memory Core API

Memory can also be used standalone outside a crew (for scripts, knowledge bases, etc.):

from crewai import Memory

memory = Memory()

# Store: LLM auto-infers scope, category, and importance
memory.remember("The project has decided to use PostgreSQL as the primary database.")
memory.remember("API rate limit is 1000 requests per minute.", scope="/team/ops")   # or specify scope explicitly

# Retrieve: sorted by composite score of semantic similarity + recency + importance
for m in memory.recall("What database do we use?"):
    print(f"[{m.score:.2f}] {m.record.content}")

# Forget: clean by scope
memory.forget(scope="/team/ops")

# View memory tree: scopes grow like directories automatically
print(memory.tree())

Three mechanisms worth understanding:

1. Scope: memories are organized as a tree resembling a filesystem (/project/alpha, /agent/researcher). Retrieval searches only within the specified branch — better precision and performance;
2. Consolidation/dedup: when new memory similarity to existing records exceeds the threshold (default 0.85), the LLM decides to keep/update/delete, preventing redundant accumulation;
3. Non-blocking writes: batch saves run on background threads; recall() auto-waits for in-flight writes, so agents don't sit idle.

10.4 Embedder Configuration: Third-Party OpenAI-Compatible and Local Models

Memory's semantic retrieval depends on an embedding model. Default is OpenAI text-embedding-3-large (requires OPENAI_API_KEY). Per course conventions, switch to a third-party OpenAI-compatible endpoint or a local model:

from crewai import Crew, Memory

# Option A: configure embedder directly on Memory (third-party OpenAI-compatible endpoint)
memory = Memory(embedder={
    "provider": "openai",
    "config": {
        "model_name": "text-embedding-3-small",
        "api_key": os.getenv("DEEPSEEK_API_KEY"),   # your compatible endpoint key
        # some compatible endpoints also require base_url — check the provider's docs
    },
})

# Option B: local Ollama, data never leaves the network
memory_local = Memory(embedder={
    "provider": "ollama",
    "config": {
        "model_name": "mxbai-embed-large",
        "url": "http://localhost:11434/api/embeddings",
    },
})

# Option C: when crew enables memory=True, crew-level embedder config auto-propagates to memory
crew = Crew(
    agents=[...], tasks=[...],
    memory=True,
    embedder={"provider": "openai", "config": {"model_name": "text-embedding-3-small"}},
)

Dimension mismatch pitfall

text-embedding-3-large produces 3072-dimensional vectors; the older text-embedding-3-small / ada-002 produce 1536. Switching embedding models mid-project without cleaning the old vector store causes immediate dimension mismatch errors. The official remedy: run crewai reset-memories -m to clear old memory, or delete the local memory storage directory, or explicitly reconfigure the old model.

10.5 Memory Cleanup and Governance

The CLI provides fine-grained cleanup commands (the old category concepts are preserved here):

crewai reset-memories --all          # clear everything
crewai reset-memories --short        # clear short-term memory only
crewai reset-memories --long         # clear long-term memory only
crewai reset-memories --entities     # clear entity memory only
crewai reset-memories --knowledge    # clear knowledge store (covered in next chapter)

Production advice: memory is a "pollutable asset" — wrong memories get recalled repeatedly. Validate memory quality with a fixed test set before going live; if pollution is detected, clean by scope immediately rather than doing a full reset.

10.6 Experiment: Memory's Impact on Multi-Turn Task Quality

Design a reproducible A/B experiment:

"""memory_experiment.py —— Memory on/off controlled experiment
Round 1: tell the crew the user's preference; Round 2: produce content, verify preference is respected.
"""
import os

from crewai import Agent, Crew, Process, Task, LLM

llm = LLM(
    model="openai/deepseek-chat",
    base_url="https://api.deepseek.com/v1",
    api_key=os.getenv("DEEPSEEK_API_KEY"),
)

writer = Agent(
    role="Columnist",
    goal="Write according to user preferences",
    backstory="A writer skilled at adjusting style based on feedback.",
    llm=llm,
)


def run_round(use_memory: bool, description: str, expected: str) -> str:
    crew = Crew(
        agents=[writer],
        tasks=[Task(description=description, expected_output=expected, agent=writer)],
        process=Process.sequential,
        memory=use_memory,           # experiment variable
        verbose=False,
    )
    return crew.kickoff().raw


# Round 1: inject preference
run_round(True, "User preference: likes short punchy sentences, hates idioms and parallel structures. Please remember this preference.",
          "Confirm you understand the user's preference, in one sentence.")

# Round 2: verify memory takes effect (with use_memory=False, round 2 will be "amnesiac")
output = run_round(True, "Write a product tagline for this user.",
                   "A tagline matching the user's style preference.")
print(output)   # memory on: short sentences; memory off: random style

Observation points: does the round 2 output maintain the "short sentences, no idioms" constraint? Compare with the output when memory is disabled. This is the most直观 proof of memory's value.

Chapter Summary

• Current version uses a unified Memory class replacing the old short/long/entity three-way split; old concepts remain in CLI cleanup commands;
• Crew(memory=True) enables it: facts auto-extracted after each task, auto-recalled and injected before each task;
• Core API: remember() / recall() / forget() / tree(), scope tree grows automatically;
• Embedder defaults to OpenAI; configurable with third-party compatible endpoints or local Ollama; watch for dimension mismatch when switching embedding models;
• crewai reset-memories supports fine-grained cleanup; treat memory as a "pollutable asset" requiring governance.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What is the memory system architecture in the current CrewAI version?

2. After enabling Crew(memory=True), what automatic behavior does the framework exhibit?

3. After switching the embedding model from text-embedding-3-small to text-embedding-3-large, the local memory throws an error. Most likely cause?

4. What is the correct command to clear only long-term memory while keeping other memories?

🛠️ Hands-On Exercises

1. Run the controlled experiment from 10.6; record the on/off memory outputs and token consumption in a comparison table.
2. Build a "team decision memo" with standalone Memory(): store 5 decisions, verify semantic retrieval hit rate with recall(), then observe scope auto-growth with tree().
3. Configure a local Ollama embedder (or third-party compatible endpoint) to replace the default; repeat the experiment from exercise 1 and confirm behavior is consistent.

Memory is the accumulation of "experience"; knowledge sources are the injection of "textbooks." Next: Chapter 11 · Knowledge Sources.