第 6 章 · 提示工程

本章目标：系统掌握通过 prompt engineering 改善模型输出的方法，从基础提示要素到 Chain-of-Thought、Tree-of-Thought 推理，再到约束采样与输出验证。

6.1 准备工作

💡 提示：运行本章示例需要 GPU。如果使用 Google Colab，请在 Runtime > Change runtime type > Hardware accelerator > GPU > GPU type > T4 中切换。

在 Colab（或其他云环境）中需要取消注释并运行以下代码块来安装本章依赖：

python

# %%capture
# !pip install langchain>=0.1.17 openai>=1.13.3 langchain_openai>=0.1.6 transformers>=4.40.1 datasets>=2.18.0 accelerate>=0.27.2 sentence-transformers>=2.5.1 duckduckgo-search>=5.2.2
# !CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python

6.2 加载模型

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    device_map="cuda",
    torch_dtype="auto",
    trust_remote_code=False,
)
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")

# Create a pipeline
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    return_full_text=False,
    max_new_tokens=500,
    do_sample=False,
)

用一条简单提示测试模型生成：

python

# Prompt
messages = [
    {"role": "user", "content": "Create a funny joke about chickens."}
]

# Generate the output
output = pipe(messages)
print(output[0]["generated_text"])

查看 chat template 应用于消息后的实际提示文本：

python

# Apply prompt template
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False)
print(prompt)

尝试较高的 temperature 进行采样：

python

# Using a high temperature
output = pipe(messages, do_sample=True, temperature=1)
print(output[0]["generated_text"])

再试较高的 top_p：

python

# Using a high top_p
output = pipe(messages, do_sample=True, top_p=1)
print(output[0]["generated_text"])

6.3 提示工程入门

提示的基本要素

一条结构良好的提示通常由多个「要素」组合而成：角色设定（persona）、指令（instruction）、上下文（context）、输出格式（data_format）、目标受众（audience）和语气（tone）——下一节的复杂提示会逐一演示这些要素。

6.4 高级提示工程

复杂提示（Complex Prompt）

下面这段用于总结的示例文本来自 https://jalammar.github.io/illustrated-transformer/ ：

python

# Text to summarize which we stole from https://jalammar.github.io/illustrated-transformer/ ;)
text = """In the previous post, we looked at Attention – a ubiquitous method in modern deep learning models. Attention is a concept that helped improve the performance of neural machine translation applications. In this post, we will look at The Transformer – a model that uses attention to boost the speed with which these models can be trained. The Transformer outperforms the Google Neural Machine Translation model in specific tasks. The biggest benefit, however, comes from how The Transformer lends itself to parallelization. It is in fact Google Cloud’s recommendation to use The Transformer as a reference model to use their Cloud TPU offering. So let’s try to break the model apart and look at how it functions.
The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.
Let’s begin by looking at the model as a single black box. In a machine translation application, it would take a sentence in one language, and output its translation in another.
Popping open that Optimus Prime goodness, we see an encoding component, a decoding component, and connections between them.
The encoding component is a stack of encoders (the paper stacks six of them on top of each other – there’s nothing magical about the number six, one can definitely experiment with other arrangements). The decoding component is a stack of decoders of the same number.
The encoders are all identical in structure (yet they do not share weights). Each one is broken down into two sub-layers:
The encoder’s inputs first flow through a self-attention layer – a layer that helps the encoder look at other words in the input sentence as it encodes a specific word. We’ll look closer at self-attention later in the post.
The outputs of the self-attention layer are fed to a feed-forward neural network. The exact same feed-forward network is independently applied to each position.
The decoder has both those layers, but between them is an attention layer that helps the decoder focus on relevant parts of the input sentence (similar what attention does in seq2seq models).
Now that we’ve seen the major components of the model, let’s start to look at the various vectors/tensors and how they flow between these components to turn the input of a trained model into an output.
As is the case in NLP applications in general, we begin by turning each input word into a vector using an embedding algorithm.
Each word is embedded into a vector of size 512. We'll represent those vectors with these simple boxes.
The embedding only happens in the bottom-most encoder. The abstraction that is common to all the encoders is that they receive a list of vectors each of the size 512 – In the bottom encoder that would be the word embeddings, but in other encoders, it would be the output of the encoder that’s directly below. The size of this list is hyperparameter we can set – basically it would be the length of the longest sentence in our training dataset.
After embedding the words in our input sequence, each of them flows through each of the two layers of the encoder.
Here we begin to see one key property of the Transformer, which is that the word in each position flows through its own path in the encoder. There are dependencies between these paths in the self-attention layer. The feed-forward layer does not have those dependencies, however, and thus the various paths can be executed in parallel while flowing through the feed-forward layer.
Next, we’ll switch up the example to a shorter sentence and we’ll look at what happens in each sub-layer of the encoder.
Now We’re Encoding!
As we’ve mentioned already, an encoder receives a list of vectors as input. It processes this list by passing these vectors into a ‘self-attention’ layer, then into a feed-forward neural network, then sends out the output upwards to the next encoder.
"""

# Prompt components
persona = "You are an expert in Large Language models. You excel at breaking down complex papers into digestible summaries.\n"
instruction = "Summarize the key findings of the paper provided.\n"
context = "Your summary should extract the most crucial points that can help researchers quickly understand the most vital information of the paper.\n"
data_format = "Create a bullet-point summary that outlines the method. Follow this up with a concise paragraph that encapsulates the main results.\n"
audience = "The summary is designed for busy researchers that quickly need to grasp the newest trends in Large Language Models.\n"
tone = "The tone should be professional and clear.\n"
text = "MY TEXT TO SUMMARIZE"  # Replace with your own text to summarize
data = f"Text to summarize: {text}"

# The full prompt - remove and add pieces to view its impact on the generated output
query = persona + instruction + context + data_format + audience + tone + data

查看拼装后的完整提示经过 chat template 之后的样子：

python

messages = [
    {"role": "user", "content": query}
]
print(tokenizer.apply_chat_template(messages, tokenize=False))

python

# Generate the output
outputs = pipe(messages)
print(outputs[0]["generated_text"])

In-Context Learning：提供示例

用一个「使用生造词造句」的单示例引导模型：

python

# Use a single example of using the made-up word in a sentence
one_shot_prompt = [
    {
        "role": "user",
        "content": "A 'Gigamuru' is a type of Japanese musical instrument. An example of a sentence that uses the word Gigamuru is:"
    },
    {
        "role": "assistant",
        "content": "I have a Gigamuru that my uncle gave me as a gift. I love to play it at home."
    },
    {
        "role": "user",
        "content": "To 'screeg' something is to swing a sword at it. An example of a sentence that uses the word screeg is:"
    }
]
print(tokenizer.apply_chat_template(one_shot_prompt, tokenize=False))

python

# Generate the output
outputs = pipe(one_shot_prompt)
print(outputs[0]["generated_text"])

链式提示（Chain Prompting）：把问题拆开

第一步，为产品起名并设计口号：

python

# Create name and slogan for a product
product_prompt = [
    {"role": "user", "content": "Create a name and slogan for a chatbot that leverages LLMs."}
]
outputs = pipe(product_prompt)
product_description = outputs[0]["generated_text"]
print(product_description)

第二步，基于上一步的名字和口号生成销售话术：

python

# Based on a name and slogan for a product, generate a sales pitch
sales_prompt = [
    {"role": "user", "content": f"Generate a very short sales pitch for the following product: '{product_description}'"}
]
outputs = pipe(sales_prompt)
sales_pitch = outputs[0]["generated_text"]
print(sales_pitch)

6.5 生成模型的推理

思维链（Chain-of-Thought）：先思考再回答

不带显式推理的直接回答（作为对照）：

python

# # Answering without explicit reasoning
# standard_prompt = [
#     {"role": "user", "content": "Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?"},
#     {"role": "assistant", "content": "11"},
#     {"role": "user", "content": "The cafeteria had 25 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?"}
# ]

# # Run generative model
# outputs = pipe(standard_prompt)
# print(outputs[0]["generated_text"])

使用思维链示例引导推理过程：

python

# Answering with chain-of-thought
cot_prompt = [
    {"role": "user", "content": "Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?"},
    {"role": "assistant", "content": "Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11."},
    {"role": "user", "content": "The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?"}
]

# Generate the output
outputs = pipe(cot_prompt)
print(outputs[0]["generated_text"])

Zero-shot Chain-of-Thought

不给示例，只在提示里加一句「Let's think step-by-step」：

python

# Zero-shot Chain-of-Thought
zeroshot_cot_prompt = [
    {"role": "user", "content": "The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? Let's think step-by-step."}
]

# Generate the output
outputs = pipe(zeroshot_cot_prompt)
print(outputs[0]["generated_text"])

思维树（Tree-of-Thought）：探索中间步骤

让多个「专家」分步作答并互相讨论：

python

# Zero-shot Chain-of-Thought
zeroshot_tot_prompt = [
    {"role": "user", "content": "Imagine three different experts are answering this question. All experts will write down 1 step of their thinking, then share it with the group. Then all experts will go on to the next step, etc. If any expert realises they're wrong at any point then they leave. The question is 'The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?' Make sure to discuss the results."}
]

python

# Generate the output
outputs = pipe(zeroshot_tot_prompt)
print(outputs[0]["generated_text"])

6.6 输出验证

提供示例约束输出格式

Zero-shot：不提供任何格式示例：

python

# Zero-shot learning: Providing no examples
zeroshot_prompt = [
    {"role": "user", "content": "Create a character profile for an RPG game in JSON format."}
]

# Generate the output
outputs = pipe(zeroshot_prompt)
print(outputs[0]["generated_text"])

One-shot：给出确切的输出结构模板：

python

# One-shot learning: Providing an example of the output structure
one_shot_template = """Create a short character profile for an RPG game. Make sure to only use this format:

{
  "description": "A SHORT DESCRIPTION",
  "name": "THE CHARACTER'S NAME",
  "armor": "ONE PIECE OF ARMOR",
  "weapon": "ONE OR MORE WEAPONS"
}
"""
one_shot_prompt = [
    {"role": "user", "content": one_shot_template}
]

# Generate the output
outputs = pipe(one_shot_prompt)
print(outputs[0]["generated_text"])

语法约束采样（Constrained Sampling）

先释放前面占用的 GPU 显存：

python

import gc
import torch
del model, tokenizer, pipe

# Flush memory
gc.collect()
torch.cuda.empty_cache()

改用 llama.cpp 加载 Phi-3 的 GGUF 量化版本：

python

from llama_cpp.llama import Llama

# Load Phi-3
llm = Llama.from_pretrained(
    repo_id="microsoft/Phi-3-mini-4k-instruct-gguf",
    filename="*fp16.gguf",
    n_gpu_layers=-1,
    n_ctx=2048,
    verbose=False
)

通过 response_format 强制模型输出 JSON：

python

# Generate output
output = llm.create_chat_completion(
    messages=[
        {"role": "user", "content": "Create a warrior for an RPG in JSON format."},
    ],
    response_format={"type": "json_object"},
    temperature=0,
)['choices'][0]['message']["content"]

格式化输出便于检查：

python

import json

# Format as json
json_output = json.dumps(json.loads(output), indent=4)
print(json_output)

本章小结

一条复杂提示由 persona、instruction、context、data_format、audience、tone、data 等要素拼装而成，增删要素即可观察其对输出的影响；
temperature 与 top_p 控制采样随机性：do_sample=False 时为确定性贪心解码；
In-Context Learning 通过在提示中给出示例（one-shot）教模型处理生僻任务；链式提示把大问题拆成多次小调用，前一步输出作为后一步输入；
Chain-of-Thought 给出含推理过程的示例可显著改善数学类问题表现；Zero-shot CoT 只需一句 "Let's think step-by-step"；Tree-of-Thought 让多个「专家」分步讨论；
llama.cpp 的 response_format={"type": "json_object"} 通过语法约束采样保证输出一定是合法 JSON，比纯提示约束更可靠。

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. 本章「复杂提示」中，persona 要素的作用是？

2. Zero-shot Chain-of-Thought 的关键做法是？

3. 链式提示（Chain Prompting）的核心思想是？

4. 相比仅靠提示词要求 JSON 输出，llama.cpp 的语法约束采样优势在于？

🛠️ 动手实践

在 6.4 节的复杂提示中，依次删除 persona、context、data_format 三要素各跑一次总结任务，对比三次输出与完整提示的差异，记录哪个要素对输出质量影响最大。
把 6.5 节被注释掉的 standard_prompt（不含思维链）恢复运行，与 cot_prompt 的结果对比，验证思维链示例对「23 个苹果」这类算术题的作用。
参照 6.6 节，用 llama.cpp 的 response_format 约束模型输出一个包含 name、class、level 三个字段的 RPG 角色对象，并用 json.loads() 验证输出确实是合法 JSON。

第 6 章 · 提示工程 ​

6.1 准备工作 ​

6.2 加载模型 ​

6.3 提示工程入门 ​

提示的基本要素 ​

6.4 高级提示工程 ​

复杂提示（Complex Prompt） ​

In-Context Learning：提供示例 ​

链式提示（Chain Prompting）：把问题拆开 ​

6.5 生成模型的推理 ​

思维链（Chain-of-Thought）：先思考再回答 ​

Zero-shot Chain-of-Thought ​

思维树（Tree-of-Thought）：探索中间步骤 ​

6.6 输出验证 ​

提供示例约束输出格式 ​

语法约束采样（Constrained Sampling） ​

本章小结 ​

🧪 随堂测验

🛠️ 动手实践 ​

第 6 章 · 提示工程

6.1 准备工作

6.2 加载模型

6.3 提示工程入门

提示的基本要素

6.4 高级提示工程

复杂提示（Complex Prompt）

In-Context Learning：提供示例

链式提示（Chain Prompting）：把问题拆开

6.5 生成模型的推理

思维链（Chain-of-Thought）：先思考再回答

Zero-shot Chain-of-Thought

思维树（Tree-of-Thought）：探索中间步骤

6.6 输出验证

提供示例约束输出格式

语法约束采样（Constrained Sampling）

本章小结

🛠️ 动手实践