Chapter 3 · Provider Management: AI Gateway and Custom Providers

Chapter goals:

• Master three authentication methods for Vercel AI Gateway and provider instance configuration
• Learn to接入任意 OpenAI-compatible server with createOpenAICompatible
• Use customProvider to implement model aliases, preset settings, and available model restrictions
• Centrally manage multiple providers with createProviderRegistry, accessing models via string IDs

3.1 AI Gateway: One Interface to All Models

AI Gateway connects models from multiple AI providers through a single interface. Access OpenAI, Anthropic, Google, Meta, xAI, and other providers and their models without integrating each SDK separately.

Core features:

• Access multi-provider models without installing additional provider modules/dependencies; -完全 consistent code structure across different AI providers;
• Easily switch models and providers;
• Automatic authentication when deployed to Vercel;
• View cross-provider pricing and observability data in the Vercel console.

3.2 Three Basic Usage Patterns

Usage 1: Pass the model string directly (global provider defaults to Gateway):

import { generateText } from 'ai';

const { text } = await generateText({
  model: 'openai/gpt-5',
  prompt: 'Hello world',
});

Usage 2: Import the default gateway instance from the ai package (v5.0.36+):

import { generateText, gateway } from 'ai';

const { text } = await generateText({
  model: gateway('openai/gpt-5'),
  prompt: 'Hello world',
});

Usage 3: createGateway to create a custom instance (when custom configuration is needed):

import { createGateway } from 'ai';

const gateway = createGateway({
  apiKey: process.env.AI_GATEWAY_API_KEY ?? '',
});

When to use a custom instance: setting a custom API key / Vercel access token / baseURL / headers; using in a provider registry; wrapping with middleware; or using different settings for different parts of the application.

Optional configuration items:

Config	Description
baseURL	API call prefix, default https://ai-gateway.vercel.sh/v4/ai
apiKey	Authentication key, defaults to reading the AI_GATEWAY_API_KEY environment variable
teamIdOrSlug	Limit request scope for multi-team Vercel access token scenarios
headers	Custom request headers
fetch	Custom fetch implementation (can be used for interception or testing)

3.3 Authentication Methods

API Key authentication — via environment variable (recommended):

AI_GATEWAY_API_KEY=your_api_key_here

Or pass directly to the provider.

Vercel Access Token authentication — for dynamic runtime authentication of model requests:

import { createGateway } from 'ai';

const gateway = createGateway({
  apiKey: 'your_vercel_access_token_here',
  teamIdOrSlug: 'your-team', // required for multi-team tokens
});

OIDC authentication — after deploying to Vercel, authenticate without an API key via OIDC token: production/preview deployments handle this automatically; local development with vercel dev auto-refreshes (token expires in 12 hours). If an API key or access token exists, it always takes precedence over OIDC.

3.4 Custom OpenAI-Compatible Provider

When bypassing the Gateway and connecting directly to any OpenAI-compatible server, use createOpenAICompatible from the @ai-sdk/openai-compatible package:

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { streamText } from 'ai';

const myProvider = createOpenAICompatible({
  name: 'my-provider', // provider identifier for error messages and telemetry
  baseURL: process.env.OPENAI_COMPATIBLE_BASE_URL ?? '', // e.g. https://api.custom.com/v1
  apiKey: process.env.OPENAI_COMPATIBLE_API_KEY ?? '',
});

const result = streamText({
  model: myProvider('gpt-4o-mini'), // model name as your server actually provides
  prompt: 'Invent a new holiday and describe its traditions.',
});

💡 Model objects produced by the Gateway and custom providers are fully interchangeable. All example code in every chapter of this course follows this convention — pick either写法 and it runs.

3.5 Custom Provider: Aliases, Presets, and Restrictions

When you need to reuse the same set of models in multiple places, use customProvider to create a provider with semantic aliases:

import { customProvider, gateway } from 'ai';

// custom provider with alias names:
export const anthropic = customProvider({
  languageModels: {
    opus: gateway('anthropic/claude-opus-4.1'),
    sonnet: gateway('anthropic/claude-sonnet-4.5'),
    haiku: gateway('anthropic/claude-haiku-4.5'),
  },
  fallbackProvider: gateway,
});

Then call with anthropic('sonnet') — upgrading the model version later only requires changing this one place.

You can also preset model parameters (using wrapLanguageModel + defaultSettingsMiddleware):

import {
  gateway,
  customProvider,
  defaultSettingsMiddleware,
  wrapLanguageModel,
} from 'ai';

export const openai = customProvider({
  languageModels: {
    // replacement model with custom provider options:
    'gpt-5.1': wrapLanguageModel({
      model: gateway('openai/gpt-5.1'),
      middleware: defaultSettingsMiddleware({
        settings: {
          providerOptions: {
            openai: {
              reasoningEffort: 'high',
            },
          },
        },
      }),
    }),
  },
  fallbackProvider: gateway,
});

You can also restrict the set of available models in the system (omit fallback for a strict whitelist):

import {
  customProvider,
  defaultSettingsMiddleware,
  wrapLanguageModel,
  gateway,
} from 'ai';

export const myProvider = customProvider({
  languageModels: {
    'text-medium': gateway('anthropic/claude-3-5-sonnet-20240620'),
    'text-small': gateway('openai/gpt-5-mini'),
    'reasoning-medium': wrapLanguageModel({
      model: gateway('openai/gpt-5.1'),
      middleware: defaultSettingsMiddleware({
        settings: {
          providerOptions: {
            openai: {
              reasoningEffort: 'high',
            },
          },
        },
      }),
    }),
    'reasoning-fast': wrapLanguageModel({
      model: gateway('openai/gpt-5.1'),
      middleware: defaultSettingsMiddleware({
        settings: {
          providerOptions: {
            openai: {
              reasoningEffort: 'low',
            },
          },
        },
      }),
    }),
  },
  embeddingModels: {
    embedding: gateway.embeddingModel('openai/text-embedding-3-small'),
  },
  // no fallback provider —— models not listed are unavailable
});

3.6 Provider Registry: Centralized Model Registry

createProviderRegistry aggregates multiple providers into one registry, accessing all models with simple string IDs:

import { createProviderRegistry, gateway } from 'ai';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';

export const registry = createProviderRegistry({
  // register provider with prefix and default setup using gateway:
  gateway,

  // register an OpenAI-compatible provider with custom setup:
  custom: createOpenAICompatible({
    name: 'provider-name',
    apiKey: process.env.CUSTOM_API_KEY,
    baseURL: 'https://api.custom.com/v1',
  }),
});

The default separator is :, but it can be customized:

export const customSeparatorRegistry = createProviderRegistry(
  {
    gateway,
  },
  { separator: ' > ' },
);

Access language models via the languageModel method in providerId:modelId format:

import { generateText } from 'ai';
import { registry } from './registry';

const { text } = await generateText({
  model: registry.languageModel('gateway:openai/gpt-5.1'), // default separator
  prompt: 'Invent a new holiday and describe its traditions.',
});

Embedding and image models have corresponding methods:

import { embed } from 'ai';
import { registry } from './registry';

const { embedding } = await embed({
  model: registry.embeddingModel('custom:text-embedding-3-small'),
  value: 'sunny day at the beach',
});

import { generateImage } from 'ai';
import { registry } from './registry';

const { image } = await generateImage({
  model: registry.imageModel('custom:image-gen-v1'),
  prompt: 'A beautiful sunset over a calm ocean',
});

3.7 Comprehensive Practice: One File Managing All Models

The core idea of provider management is: declare all providers and models in one file — preset settings, define aliases, restrict scope. Below is the official comprehensive example (adapted for Gateway + OpenAI-compatible组合):

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import {
  createProviderRegistry,
  customProvider,
  defaultSettingsMiddleware,
  gateway,
  wrapLanguageModel,
} from 'ai';

export const registry = createProviderRegistry(
  {
    // pass through gateway with a namespace prefix
    gateway,

    // access an OpenAI-compatible provider with custom setup
    custom: createOpenAICompatible({
      name: 'provider-name',
      apiKey: process.env.CUSTOM_API_KEY,
      baseURL: 'https://api.custom.com/v1',
    }),

    // setup model name aliases + pre-configured settings
    smart: customProvider({
      languageModels: {
        fast: gateway('openai/gpt-5-mini'),

        writing: gateway('anthropic/claude-sonnet-4.5'),

        reasoning: wrapLanguageModel({
          model: gateway('anthropic/claude-sonnet-4.5'),
          middleware: defaultSettingsMiddleware({
            settings: {
              maxOutputTokens: 100000, // example default setting
            },
          }),
        }),
      },
      fallbackProvider: gateway,
    }),
  },
  { separator: ' > ' },
);

// usage:
const model = registry.languageModel('smart > reasoning');

This pattern achieves:

• Namespace-prefixed Gateway passthrough (gateway > *);
• OpenAI-compatible provider with custom API key/baseURL (custom > *);
• Model name aliases (smart > fast, smart > writing, smart > reasoning);
• Pre-configured model settings (smart > reasoning);
• Fallback provider (smart > *);
• Strict-whitelist provider with no fallback.

Global Provider

AI SDK also supports a global provider feature — specify the model with a plain string directly:

import { streamText } from 'ai';

const result = await streamText({
  model: 'openai/gpt-5.1', // Uses the global provider (defaults to gateway)
  prompt: 'Invent a new holiday and describe its traditions.',
});

The default global provider is Vercel AI Gateway; it can also be replaced at startup:

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';

globalThis.AI_SDK_DEFAULT_PROVIDER = createOpenAICompatible({
  name: 'my-provider',
  baseURL: process.env.OPENAI_COMPATIBLE_BASE_URL ?? '',
  apiKey: process.env.OPENAI_COMPATIBLE_API_KEY ?? '',
});

import { streamText } from 'ai';

const result = await streamText({
  model: 'my-model', // Uses your custom global provider without prefix
  prompt: 'Invent a new holiday and describe its traditions.',
});

This simplifies provider usage, letting you switch providers without modifying model references in business code.

3.8 Chapter Summary

• AI Gateway has three usage patterns: plain model string, gateway default instance, createGateway custom instance;
• Authentication priority: explicitly passed key > AI_GATEWAY_API_KEY env var > OIDC;
• createOpenAICompatible is the通用 key for接入任意 self-hosted/OpenAI-compatible server;
• customProvider provides three capabilities: model aliases, defaultSettingsMiddleware presets, and no-fallback whitelist restrictions;
• createProviderRegistry centrally orchestrates multi-provider access via providerId:modelId strings, used with languageModel/embeddingModel/imageModel methods.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. When apiKey, AI_GATEWAY_API_KEY env var, and OIDC all exist simultaneously, what is the Gateway authentication priority?

2. Which package and function should you use to接入任意 OpenAI-compatible server?

3. What does it mean when customProvider has no fallbackProvider set?

4. Regarding the Provider Registry, which statement is FALSE?

🛠️ Hands-On Exercises

1. Write a provider.ts exporting two functions getModel(mode: 'gateway' | 'custom') that each return equally-capable models constructed via Gateway and OpenAI-compatible instances; verify interchangeability.
2. Design a semantic alias system (chat-fast, chat-smart, embed-base) for your team基于 customProvider; explain where to change when upgrading models.
3. Use createProviderRegistry to register both Gateway and a custom provider; call each once via languageModel('gateway:...') and languageModel('custom:...') and compare responses.

Next: Chapter 4 · Prompts and Message Models