📚 AI 工程与智能体教程站

English

简体中文

English

简体中文

Chapter 7 · Advanced Text Generation Techniques and Tools ​

Chapter goals: Go beyond prompt engineering and use LangChain to organize LLM workflows — chained calls, multi-chain orchestration, conversation memory, and ReAct agents.

7.1 Preparation ​

The official notebook corresponding to this chapter is chapter07/Chapter 7 - Advanced Text Generation Techniques and Tools.ipynb.

💡 NOTE: Running the examples in this chapter requires a GPU. In Google Colab, go to Runtime > Change runtime type > Hardware accelerator > GPU > GPU type > T4.

If you are viewing the notebook on Google Colab (or any other cloud environment), you need to uncomment and run the code block below to install the dependencies for this chapter:

python
# %%capture
# !pip install langchain>=0.1.17 openai>=1.13.3 langchain_openai>=0.1.6 transformers>=4.40.1 datasets>=2.18.0 accelerate>=0.27.2 sentence-transformers>=2.5.1 duckduckgo-search>=5.2.2 langchain_community
# !CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python==0.2.69

7.2 Loading an LLM ​

First download the GGUF quantized weights for the Phi-3 mini model:

python
!wget https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-fp16.gguf

# If this command does not work for you, you can use the link directly to download the model
# https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-fp16.gguf

Load the model through LangChain's LlamaCpp wrapper:

python
from langchain import LlamaCpp

# Make sure the model path is correct for your system!
llm = LlamaCpp(
    model_path="Phi-3-mini-4k-instruct-fp16.gguf",
    n_gpu_layers=-1,
    max_tokens=500,
    n_ctx=2048,
    seed=42,
    verbose=False
)

Test basic Q&A:

python
llm.invoke("Hi! My name is Maarten. What is 1 + 1?")

7.3 Chains ​

A Chain combines a prompt template and an LLM into a reusable pipeline. First create a prompt template with an input_prompt variable:

python
from langchain import PromptTemplate

# Create a prompt template with the "input_prompt" variable
template = """<s><|user|>
{input_prompt}<|end|>
<|assistant|>"""
prompt = PromptTemplate(
    template=template,
    input_variables=["input_prompt"]
)

Connect the prompt and LLM into a basic chain using the pipe operator |:

python
basic_chain = prompt | llm
python
# Use the chain
basic_chain.invoke(
    {
        "input_prompt": "Hi! My name is Maarten. What is 1 + 1?",
    }
)

7.4 Multiple Chains Working Together ​

A single chain has limited capability; the real power lies in chaining multiple chains together, letting the output of one chain become the input of the next. Below is a three-stage "story generation" pipeline as an example: generate a title → describe the protagonist → write the story.

Step 1: create the chain responsible for generating the story title:

python
from langchain import LLMChain

# Create a chain for the title of our story
template = """<s><|user|>
Create a title for a story about {summary}. Only return the title.<|end|>
<|assistant|>"""
title_prompt = PromptTemplate(template=template, input_variables=["summary"])
title = LLMChain(llm=llm, prompt=title_prompt, output_key="title")
python
title.invoke({"summary": "a girl that lost her mother"})

Step 2: create the chain that describes the protagonist using the summary and title:

python
# Create a chain for the character description using the summary and title
template = """<s><|user|>
Describe the main character of a story about {summary} with the title {title}. Use only two sentences.<|end|>
<|assistant|>"""
character_prompt = PromptTemplate(
    template=template, input_variables=["summary", "title"]
)
character = LLMChain(llm=llm, prompt=character_prompt, output_key="character")

Step 3: create the chain that synthesizes all three to produce the complete story:

python
# Create a chain for the story using the summary, title, and character description
template = """<s><|user|>
Create a story about {summary} with the title {title}. The main charachter is: {character}. Only return the story and it cannot be longer than one paragraph<|end|>
<|assistant|>"""
story_prompt = PromptTemplate(
    template=template, input_variables=["summary", "title", "character"]
)
story = LLMChain(llm=llm, prompt=story_prompt, output_key="story")

String the three components together into a full pipeline:

python
# Combine all three components to create the full chain
llm_chain = title | character | story
python
llm_chain.invoke("a girl that lost her mother")

7.5 Memory ​

LLMs are inherently "stateless" — each call is independent. Let's verify: first tell it our name,

python
# Let's give the LLM our name
basic_chain.invoke({"input_prompt": "Hi! My name is Maarten. What is 1 + 1?"})

then ask it to repeat the name — it will fail:

python
# Next, we ask the LLM to reproduce the name
basic_chain.invoke({"input_prompt": "What is my name?"})

Memory (Memory) is the mechanism that solves this problem. LangChain provides several memory strategies.

7.5.1 ConversationBuffer Memory ​

The idea is straightforward: dump the chat history into the prompt. First update the prompt template to include a chat_history variable:

python
# Create an updated prompt template to include a chat history
template = """<s><|user|>Current conversation:{chat_history}

{input_prompt}<|end|>
<|assistant|>"""

prompt = PromptTemplate(
    template=template,
    input_variables=["input_prompt", "chat_history"]
)

Introduce ConversationBufferMemory and wire it into the chain:

python
from langchain.memory import ConversationBufferMemory

# Define the type of Memory we will use
memory = ConversationBufferMemory(memory_key="chat_history")

# Chain the LLM, Prompt, and Memory together
llm_chain = LLMChain(
    prompt=prompt,
    llm=llm,
    memory=memory
)
python
# Generate a conversation and ask a basic question
llm_chain.invoke({"input_prompt": "Hi! My name is Maarten. What is 1 + 1?"})

Now the LLM remembers the name we gave it:

python
# Does the LLM remember the name we gave it?
llm_chain.invoke({"input_prompt": "What is my name?"})

7.5.2 ConversationBufferWindowMemory ​

The problem with buffer memory is that the prompt grows indefinitely as the conversation grows. Window memory keeps only the most recent k turns:

python
from langchain.memory import ConversationBufferWindowMemory

# Retain only the last 2 conversations in memory
memory = ConversationBufferWindowMemory(k=2, memory_key="chat_history")

# Chain the LLM, Prompt, and Memory together
llm_chain = LLMChain(
    prompt=prompt,
    llm=llm,
    memory=memory
)

Have two rounds of conversation to fill its memory window:

python
# Ask two questions and generate two conversations in its memory
llm_chain.invoke({"input_prompt":"Hi! My name is Maarten and I am 33 years old. What is 1 + 1?"})
llm_chain.invoke({"input_prompt":"What is 3 + 3?"})

Does it still remember the name?

python
# Check whether it knows the name we gave it
llm_chain.invoke({"input_prompt":"What is my name?"})

What about the age? (The first round has been pushed out of the window)

python
# Check whether it knows the age we gave it
llm_chain.invoke({"input_prompt":"What is my age?"})

7.5.3 ConversationSummary Memory ​

A third strategy: instead of keeping the raw text, let the LLM continuously summarize the conversation — information is retained longer and the length stays可控. First create a summary prompt template:

python
# Create a summary prompt template
summary_prompt_template = """<s><|user|>Summarize the conversations and update with the new lines.

Current summary:
{summary}

new lines of conversation:
{new_lines}

New summary:<|end|>
<|assistant|>"""
summary_prompt = PromptTemplate(
    input_variables=["new_lines", "summary"],
    template=summary_prompt_template
)

Introduce ConversationSummaryMemory and wire it into the chain:

python
from langchain.memory import ConversationSummaryMemory

# Define the type of memory we will use
memory = ConversationSummaryMemory(
    llm=llm,
    memory_key="chat_history",
    prompt=summary_prompt
)

# Chain the LLM, prompt, and memory together
llm_chain = LLMChain(
    prompt=prompt,
    llm=llm,
    memory=memory
)
python
# Generate a conversation and ask for the name
llm_chain.invoke({"input_prompt": "Hi! My name is Maarten. What is 1 + 1?"})
llm_chain.invoke({"input_prompt": "What is my name?"})

Even when early conversations have been compressed into summaries, key information remains accessible:

python
# Check whether it has summarized everything thus far
llm_chain.invoke({"input_prompt": "What was the first question I asked?"})

Inspect the current summary content directly:

python
# Check what the summary is thus far
memory.load_memory_variables({})

7.6 Agents ​

Agents are the climax of this chapter: the LLM no longer answers directly, but instead reasons about which tool to call, observes the result, and loops until it arrives at a final answer. This is the ReAct paradigm (Reasoning + Acting).

Since agents require strong instruction-following ability, we switch to OpenAI's model here:

python
import os
from langchain_openai import ChatOpenAI

# Load OpenAI's LLMs with LangChain
os.environ["OPENAI_API_KEY"] = "MY_KEY"
openai_llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

Write the ReAct prompt template — it defines the Thought → Action → Action Input → Observation reasoning loop format:

python
# Create the ReAct template
react_template = """Answer the following questions as best you can. You have access to the following tools:

{tools}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: {input}
Thought:{agent_scratchpad}"""

prompt = PromptTemplate(
    template=react_template,
    input_variables=["tools", "tool_names", "input", "agent_scratchpad"]
)

Prepare the tool set: DuckDuckGo web search + math calculation:

python
from langchain.agents import load_tools, Tool
from langchain.tools import DuckDuckGoSearchResults

# You can create the tool to pass to an agent
search = DuckDuckGoSearchResults()
search_tool = Tool(
    name="duckduck",
    description="A web search engine. Use this to as a search engine for general queries.",
    func=search.run,
)

# Prepare tools
tools = load_tools(["llm-math"], llm=openai_llm)
tools.append(search_tool)

Construct the ReAct agent and executor:

python
from langchain.agents import AgentExecutor, create_react_agent

# Construct the ReAct agent
agent = create_react_agent(openai_llm, tools, prompt)
agent_executor = AgentExecutor(
    agent=agent, tools=tools, verbose=True, handle_parsing_errors=True
)

Ask a question that requires both "search + calculate" to answer:

python
# What is the Price of a MacBook Pro?
agent_executor.invoke(
    {
        "input": "What is the current price of a MacBook Pro in USD? How much would it cost in EUR if the exchange rate is 0.85 EUR for 1 USD?"
    }
)

7.7 Chapter Summary ​

  • LangChain's Chain combines a prompt template and an LLM into a reusable pipeline using the prompt | llm pipe syntax;
  • Multi-chain orchestration breaks tasks into phased pipelines: in this example the title chain → character chain → story chain pass output_key sequentially;
  • LLMs are stateless; the three memory strategies each have trade-offs: ConversationBuffer saves everything, Window keeps only the most recent k turns, Summary continuously compresses via the LLM;
  • The ReAct agent makes autonomous tool-call decisions through a Thought → Action → Observation loop, combining DuckDuckGo search and llm-math to complete multi-hop tasks;
  • Phi-3 mini in GGUF format can run efficiently on a local GPU via LlamaCpp.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What does the line `basic_chain = prompt | llm` in LangChain do?

2. In the three-chain story pipeline, what are the input variables for the character description chain?

3. After using ConversationBufferWindowMemory(k=2), how does the LLM behave toward information from the earliest round of conversation?

4. Regarding the execution loop of a ReAct agent, which statement is correct?

🛠️ Hands-on Practice ​

  1. Extend the three-chain story pipeline in Section 7.4 into a four-chain pipeline: add an "ending chain" (accepting summary, title, character, and story, returning only a one-sentence ending); test the complete effect with the same input "a girl that lost her mother".
  2. Change the ConversationBufferWindowMemory(k=2) in Section 7.5.2 to k=1; repeat the three-question experiment from the book; record which information is lost; and explain how window size affects memory capability.
  3. Following the ReAct template in Section 7.6, add one more custom Tool to the agent (e.g. a function returning the current date); then pose a question that requires the search, math, and new tool together; observe the full reasoning trace output by verbose=True.