Chapter 12 · RAG: Retrieval-Augmented Generation

Chapter goals: Master Mastra's RAG pipeline: document chunking, embedding, ingestion, retrieval, and reranking — inject私有 knowledge into Agent context.

12.1 RAG Five-Step Pipeline

The model doesn't know your internal documents. The standard RAG (Retrieval-Augmented Generation) flow:

1. Load — convert Markdown/PDF/HTML to MDocument;
2. Chunk — split by token count or semantic boundary;
3. Embed — use an embedding model to turn chunks into vectors and write to a vector store;
4. Retrieve — on query, embed the question too, fetch the top-K most similar;
5. Inject — concatenate retrieved results into the prompt for the model to answer.

12.2 Document Chunking and Embedding

// src/rag/ingest.ts
import { MDocument } from '@mastra/rag'
import { embedMany } from 'ai'
import { openai } from '@ai-sdk/openai'

// Create a document object from Markdown text (also supports fromPDF / fromHTML etc.)
const doc = MDocument.fromText(`
# Refund Policy
No-questions-asked refund within 7 days of ordering.
# Shipping Timeline
In-stock items ship within 48 hours.
`)

// Chunk by heading structure; each chunk ≤ 512 tokens, 50-token overlap
const chunks = await doc.chunk({
  strategy: 'markdown',
  size: 512,
  overlap: 50,
})

// Batch-generate vectors (embedding dimension determined by the model)
const { embeddings } = await embedMany({
  model: openai.embedding('text-embedding-3-small'),
  values: chunks.map((c) => c.text),
})

12.3 Writing to the Vector Store

Mastra provides多种 Vector adapters; here using LibSQL's vector extension:

// src/rag/store.ts
import { LibSQLVector } from '@mastra/libsql'

const vector = new LibSQLVector({ url: 'file:./vectors.db' })

// Create index: dimension must match the embedding model output
await vector.createIndex({
  indexName: 'kb_docs',
  dimension: 1536, // text-embedding-3-small outputs 1536 dimensions
})

// Upsert vectors with原文 metadata
await vector.upsert({
  indexName: 'kb_docs',
  vectors: embeddings,
  metadata: chunks.map((c) => ({ text: c.text })),
})

12.4 Retrieving and Injecting Context

// src/rag/query.ts
import { embed } from 'ai'
import { openai } from '@ai-sdk/openai'

export async function retrieve(question: string): Promise<string[]> {
  // Embed the question
  const { embedding } = await embed({
    model: openai.embedding('text-embedding-3-small'),
    value: question,
  })

  // Cosine similarity; take top 3
  const results = await vector.query({
    indexName: 'kb_docs',
    queryVector: embedding,
    topK: 3,
  })

  return results.map((r) => r.metadata?.text ?? '')
}

// Usage: pass retrieved results as context to the Agent
const context = (await retrieve('How long does a return take?')).join('\n---\n')
const answer = await agent.generate(
  `Answer the customer question参考 the following materials:\n${context}\n\nQuestion: How long does a return take?`,
)

12.5 Reranking and Advanced Techniques

• rerank() uses a cross-encoder to re-sort初筛 results, significantly improving precision;
• Metadata Filters支持按类别、时间等字段过滤;
• For large datasets, explore GraphRAG (knowledge-graph-enhanced retrieval).

Rerank Example

Vector初筛 is fast but粗; use a cross-encoder to精排 the top-20 then取前 3:

// src/rag/rerank.ts
import { rerank } from '@mastra/rag'

const candidates = await retrieveAll(question, 20) // Expandrecall for初筛
const reranked = await rerank({
  query: question,
  results: candidates,
  topK: 3, // Keep only the top 3 after精排
})

12.6 Chapter Summary

• RAG = chunk → embed → ingest → retrieve → inject (five steps);
• MDocument.chunk() supports smart splitting by markdown structure;
• Vector dimension must match the embedding model (1536 for text-embedding-3-small);
• Retrieval quality determines the answer ceiling; add rerank and metadata filters when needed.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What is the main purpose of "chunking" in the RAG pipeline?

2. When creating a vector index, what condition must the dimension parameter satisfy?

3. What does topK: 3 in vector.query()表示?

4. What does rerank() do in the RAG pipeline?

🛠️ Hands-On Exercises

1. Turn your真实 product FAQ doc into a RAG pipeline; ask 5 questions and check hit rate.
2. Compare the effect on answer quality between size: 256 and size: 1024 chunking.
3. Add a rerank step after retrieval; observe the top-3 order change.

Next: Chapter 13 · Storage Layer