Chapter 8 · Tool Calling

Chapter goals:

• Master the four key elements of a tool: description, inputSchema, execute, strict
• Understand how multi-step calls and stopWhen stop conditions work
• Learn to use prepareStep to dynamically adjust model, messages, and tools per step
• Understand dynamic tools, tool choice, activeTools, and tool error handling

8.1 Tool Definition

As described in Foundations, tools are objects that can be called by the model to perform specific tasks. Function tools and dynamic tools share several core elements:

• description: Optional tool description that influences when the model chooses the tool. Can be a string, or a function that derives the description from the tool context and experimental sandbox;
• inputSchema: Defines the input parameters as a Zod schema or JSON schema. The schema is consumed by the LLM and also used to validate the model's tool calls;
• execute: Optional async function that executes with the tool call's input as argument and produces a result. It is optional — you may want to forward the tool call to a client or queue instead of executing in the current process;
• strict: (Optional boolean) Enables strict tool calling when the provider supports it.

💡 You can use the tool helper function to infer the type of the execute parameter.

The tools parameter for generateText and streamText is an object where keys are tool names and values are the tools:

import { z } from 'zod';
import { generateText, tool, isStepCount } from 'ai';

const result = await generateText({
  model,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      inputSchema: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  stopWhen: isStepCount(5),
  prompt: 'What is the weather in San Francisco?',
});

💡 When the model uses a tool it is called a "tool call"; the tool's output is called a "tool result". Tool calls are not limited to text generation — they can also be used to render user interfaces (Generative UI).

The model in this chapter's examples is constructed from the provider module in Chapter 3:

import { createGateway } from 'ai';

export const gateway = createGateway({
  apiKey: process.env.AI_GATEWAY_API_KEY ?? '',
});

// Custom OpenAI-compatible Provider equivalent:
// import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
// export const myProvider = createOpenAICompatible({ name: 'my-provider', baseURL: ..., apiKey: ... });

8.2 Dynamic Descriptions and Strict Mode

Dynamic Descriptions

Tool descriptions can be a fixed string or a function. Use a function when the description sent to the model should depend on the current tool context or experimental sandbox (e.g., tenant, project, environment, or workspace). The description function resolves before each step's generation, before the tool definition is sent to the model, receiving the matching context from toolsContext and the current experimental_sandbox:

import { generateText, tool } from 'ai';
import { z } from 'zod';

const shell = tool({
  contextSchema: z.object({
    projectName: z.string(),
  }),
  description: ({ context, experimental_sandbox }) =>
    [
      `Run shell commands for the ${context.projectName} project.`,
      experimental_sandbox != null
        ? `Sandbox: ${experimental_sandbox.description}`
        : undefined,
    ]
      .filter(Boolean)
      .join('\n'),
  inputSchema: z.object({
    command: z.string(),
  }),
  execute: async ({ command }, { experimental_sandbox }) => {
    if (!experimental_sandbox) {
      throw new Error('Experimental sandbox is not available');
    }
    return experimental_sandbox.run({ command });
  },
});

const result = await generateText({
  model,
  tools: { shell },
  toolsContext: {
    shell: { projectName: 'web-app' },
  },
  experimental_sandbox,
  prompt: 'List the project files.',
});

Strict Mode

When enabled, providers that support strict tool calling will only generate valid tool calls that conform to your defined inputSchema, improving reliability. But not all schemas are supported in strict mode (depends on the provider). Disabled by default, enabled per-tool:

tool({
  description: 'Get the weather in a location',
  inputSchema: z.object({
    location: z.string(),
  }),
  strict: true, // Enable strict validation for this tool
  execute: async ({ location }) => ({
    // ...
  }),
});

⚠️ Not all providers or models support strict mode; when unsupported, this option is ignored.

Input Examples

You can specify input examples for a tool to help the model understand the structure of input data. Particularly useful when the JSON schema alone cannot fully explain expected usage or when there are optional values:

tool({
  description: 'Get the weather in a location',
  inputSchema: z.object({
    location: z.string().describe('The location to get the weather for'),
  }),
  inputExamples: [
    { input: { location: 'San Francisco' } },
    { input: { location: 'London' } },
  ],
  execute: async ({ location }) => {
    // ...
  },
});

💡 Currently only the Anthropic provider natively supports tool input examples; other providers ignore this setting.

8.3 Multi-Step Calls (stopWhen)

By default, generateText/streamText trigger only one generation. But when tools are provided, the model can choose to generate either a plain text reply or a tool call — if it generates a tool call, this step ends.

You often want the model to generate text after the tool executes (e.g., to summarize the tool result combined with the user's question), or even use multiple tools in sequence within a single response. This is the value of multi-step calls: think of it like talking to a real person — when they don't have the relevant knowledge, they look it up first (using a tool) then answer you; each generation (tool call or text) is one step.

Multi-step calls are enabled via the stopWhen setting. Once set, as long as the model generates a tool call and the stop condition is not met, the SDK feeds the tool result back and triggers a new generation. Built-in stop conditions:

• isStepCount(count) — stop after reaching the specified step count (default isStepCount(20))
• hasToolCall(...toolNames) — stop when any specified tool is called
• isLoopFinished() — never triggers, letting the loop run naturally to completion

Multiple conditions can be combined into an array or custom conditions (see Chapter 19 for loop control).

⚠️ stopWhen conditions are only evaluated when the last step includes a tool result.

The following example has two steps: Step 1 prompt is sent to the model → model generates a tool call → tool is executed; Step 2 tool result is fed back to the model → model generates a response combined with the result:

import { z } from 'zod';
import { generateText, tool, isStepCount } from 'ai';

const { text, steps } = await generateText({
  model,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      inputSchema: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  stopWhen: isStepCount(5), // If tools are called, stop after at most 5 steps
  prompt: 'What is the weather in San Francisco?',
});

streamText works the same way.

8.4 steps and onStepEnd

To access intermediate tool calls and results, use the steps property of the result object or the onEnd callback of streamText. Steps contain all text, tool calls, tool results, and performance data for each step:

import { generateText } from 'ai';

const { steps } = await generateText({
  model,
  stopWhen: isStepCount(10),
  // ...
});

// Extract all tool calls from all steps:
const allToolCalls = steps.flatMap(step => step.toolCalls);

The onStepEnd callback triggers when a step completes (all text deltas, tool calls, and tool results for that step are ready); in multi-step mode it triggers progressively. The callback receives a zero-indexed stepNumber:

import { generateText } from 'ai';

const result = await generateText({
  // ...
  onStepEnd({
    stepNumber,
    text,
    toolCalls,
    toolResults,
    finishReason,
    usage,
    performance,
  }) {
    console.log(`Step ${stepNumber} finished (${finishReason})`, {
      usage,
      performance,
    });
    // Custom logic, e.g., saving chat history or recording usage
  },
});

You can also use onToolExecutionStart / onToolExecutionEnd to observe each tool execution's timing, latency, input/output, and errors (see Chapter 6).

8.5 prepareStep: Per-Step Customization

The prepareStep callback is called before each step begins, with parameters including model, stopWhen, stepNumber, steps, instructions, messages, responseMessages, runtimeContext, etc. You can use it to provide different settings for a particular step, including modifying input messages:

import { generateText } from 'ai';

const result = await generateText({
  // ...
  prepareStep: async ({ model, stepNumber, steps, messages }) => {
    if (stepNumber === 0) {
      return {
        // Swap the model for this step:
        model: modelForThisParticularStep,
        // Force a specific tool call this step:
        toolChoice: { type: 'tool', toolName: 'tool1' },
        // Limit the tools available this step:
        activeTools: ['tool1'],
      };
    }

    // When nothing is returned, default settings are used
  },
});

If instructions are returned, they persist to subsequent steps until a prepareStep returns a new override value; use initialInstructions to restore the top-level instructions.

Message Modification in Long Loops (Context Compaction)

In longer agent loops, you can rewrite the message state that subsequent steps will use via the messages parameter — particularly useful for context compaction. Default messages are initialMessages plus accumulated responseMessages; if a previous prepareStep returned messages, subsequent steps are based on that persisted messages plus the most recent step's response messages. The built-in pruneMessages helper provides a simple compaction strategy:

import { generateText, pruneMessages, type ModelMessage } from 'ai';

const COMPACTION_THRESHOLD = 100_000;

const estimateTokens = (messages: ModelMessage[]) => {
  return JSON.stringify(messages).length / 4;
};

const result = await generateText({
  // ...
  prepareStep: ({ messages }) => {
    if (estimateTokens(messages) > COMPACTION_THRESHOLD) {
      return {
        messages: pruneMessages({
          messages,
          reasoning: 'all',
          toolCalls: 'before-last-3-messages',
          emptyMessages: 'remove',
        }),
      };
    }
  },
});

Returned message changes persist across steps. If you want each step to re-derive from the original input plus discrete response messages, you can reconstruct using initialMessages and responseMessages in each callback:

prepareStep: ({ initialMessages, responseMessages, stepNumber }) => {
  if (stepNumber > 0) {
    return {
      messages: [...initialMessages, ...responseMessages.slice(-10)],
    };
  }
},

8.6 Dynamic Tools and Preliminary Results

dynamicTool

Use dynamic tools when the tool schema is unknown at compile time — typical scenarios: schema-less MCP tools, user-defined functions at runtime, tools loaded from external sources:

import { dynamicTool } from 'ai';
import { z } from 'zod';

const customTool = dynamicTool({
  description: 'Execute a custom function',
  inputSchema: z.object({}),
  execute: async input => {
    // input is typed as 'unknown'
    // Validate/assert at runtime
    const { action, parameters } = input as any;

    // Execute dynamic logic
    return { result: `Executed ${action}` };
  },
});

When using both static and dynamic tools, narrow the type with the dynamic flag:

const result = await generateText({
  model,
  tools: {
    // Static tool with known type
    weather: weatherTool,
    // Dynamic tool
    custom: dynamicTool({
      /* ... */
    }),
  },
  onStepEnd: ({ toolCalls, toolResults }) => {
    // Type-safe iteration
    for (const toolCall of toolCalls) {
      if (toolCall.dynamic) {
        // Dynamic tool: input is 'unknown'
        console.log('Dynamic:', toolCall.toolName, toolCall.input);
        continue;
      }

      // Static tool: full type inference
      switch (toolCall.toolName) {
        case 'weather':
          console.log(toolCall.input.location); // typed as string
          break;
      }
    }
  },
});

Preliminary Tool Results

A tool's execute can return an AsyncIterable producing multiple results, with the last value serving as the final tool result. Paired with generator functions, you can stream status information during tool execution:

tool({
  description: 'Get the current weather.',
  inputSchema: z.object({
    location: z.string(),
  }),
  async *execute({ location }) {
    yield {
      status: 'loading' as const,
      text: `Getting weather for ${location}`,
      weather: undefined,
    };

    await new Promise(resolve => setTimeout(resolve, 3000));

    const temperature = 72 + Math.floor(Math.random() * 21) - 10;

    yield {
      status: 'success' as const,
      text: `The weather in ${location} is ${temperature}°F`,
      temperature,
    };
  },
});

8.7 toolChoice and activeTools

Tool Choice

Use toolChoice to influence when the model selects a tool:

• auto (default): The model decides whether and which tool to call
• required: The model must call a tool; it can choose which one
• none: Tool calls are prohibited
• { type: 'tool', toolName: string }: Must call the specified tool

import { z } from 'zod';
import { generateText, tool } from 'ai';

const result = await generateText({
  model,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      inputSchema: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  toolChoice: 'required', // Force the model to call a tool
  prompt: 'What is the weather in San Francisco?',
});

Active Tools

Language models can only process a有限 number of tools at once. To both statically type a large tool set and limit the model's visible tool set, AI SDK provides the activeTools property — an array of currently active tool names, default undefined meaning all activated:

const { text } = await generateText({
  model,
  tools: myToolSet,
  activeTools: ['firstTool'],
});

8.8 Tool Error Handling

AI SDK has three classes of tool call-related errors:

• NoSuchToolError: The model tries to call a tool that does not exist in the tools object
• InvalidToolInputError: The input the model called does not conform to the tool's input schema
• ToolCallRepairError: Errors occurring during tool call repair

When tool execution fails (your execute throws), the error is added to the result as a tool-error content part, enabling automatic LLM round-trip handling in multi-step scenarios.

Handling in generateText

generateText throws exceptions for schema validation issues that can be handled with try/catch; tool execution errors appear in the result's steps:

try {
  const result = await generateText({
    //...
  });
} catch (error) {
  if (NoSuchToolError.isInstance(error)) {
    // Handle no-such-tool error
  } else if (InvalidToolInputError.isInstance(error)) {
    // Handle invalid tool input error
  } else {
    // Handle other errors
  }
}

const { steps } = await generateText({
  // ...
});

// Check for tool errors in steps
const toolErrors = steps.flatMap(step =>
  step.content.filter(part => part.type === 'tool-error'),
);

toolErrors.forEach(toolError => {
  console.log('Tool error:', toolError.error);
  console.log('Tool name:', toolError.toolName);
  console.log('Tool input:', toolError.input);
});

Handling in streamText

streamText sends errors as part of the stream result: tool execution errors appear as tool-error parts, other errors as error parts. Additionally, the SDK provides a tool call repair mechanism (tool call repair), e.g., retrying parsing with a structured output model, or using a re-ask strategy to have the model re-provide call parameters.

Chapter Summary

• Four tool elements: description (can be a dynamic function), inputSchema (Zod/JSON Schema), execute (optional to forward to client), strict (strict mode when provider supports)
• Multi-step calls driven by stopWhen: isStepCount / hasToolCall / isLoopFinished are composable; conditions are only evaluated when the last step contains a tool result
• steps and onStepEnd provide full intermediate state per step; prepareStep lets you swap models per step, force toolChoice, limit activeTools, or compact context
• For dynamic scenarios use dynamicTool + toolCall.dynamic type narrowing; generator-form execute can produce preliminary progress results
• Three error classes: NoSuchToolError / InvalidToolInputError / ToolCallRepairError; exceptions thrown inside execute become tool-error content participating in automatic round-trips

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. Why is execute optional in tool definition?

2. Regarding the evaluation timing of stopWhen, which is correct?

3. After prepareStep step 0 returns { model, toolChoice, activeTools }, what happens in step 2?

4. How does the AI SDK handle exceptions thrown inside a tool's execute function?

🛠️ Hands-On Exercises

1. Define three tools (weather, exchange rate, time) attached to a generateText with stopWhen: isStepCount(6); ask "What time is it in San Francisco, what's the weather, and how much is 100 USD in CNY"; print steps.length and each step's toolCalls.
2. Use prepareStep to implement a strategy: step 1 forces toolChoice: required and only exposes search-type tools; from step 2 onward all tools are unlocked; compare answer quality with and without prepareStep.
3. Write an execute that intentionally throws; observe both the generateText try/catch branch and the tool-error parts in steps; then add inputExamples and strict: true to the tool and test the behavior change.

Next: Chapter 9 · Multi-Step Tool Loops and MCP