Chapter 14 · Custom Providers and OpenAI-Compatible Endpoints

Chapter goals: Connect to local inference servers, enterprise gateways, and OpenAI-compatible endpoints like DeepSeek using createProvider(), and resolve compatibility differences with compat flags.

14.1 Why Custom Providers Are Needed

The built-in provider catalog covers major vendors, but the real world has three categories of needs:

1. Local inference — Ollama, llama.cpp, and other local servers;
2. Enterprise proxies/gateways — internal LLM gateways with unified auth, auditing, and rate limiting;
3. OpenAI-compatible third parties — vendors like DeepSeek and Together AI that are compatible with the openai-completions API.

pi-ai's answer is a unified assembly function: createProvider().

14.2 Four Parts of createProvider()

createProvider() builds a provider from four parts: identity (id), auth (auth), model list (models), and API implementation (api):

import { createModels, createProvider, envApiKeyAuth, type Model } from '@earendil-works/pi-ai';

// Assemble a local Ollama provider
const ollama = createProvider({
  id: 'ollama',
  name: 'Ollama (local)',
  auth: envApiKeyAuth('OLLAMA_API_KEY'), // Auth resolution method
  api: 'openai-completions',             // Reuse the OpenAI-compatible API implementation
  baseUrl: 'http://localhost:11434/v1',
  models: [
    {
      id: 'qwen3:32b',
      name: 'Qwen3 32B',
      reasoning: false,
      input: ['text'],
      // ... other Model metadata
    } as Model,
  ],
});

const models = createModels();
models.setProvider(ollama);

Dynamic publish mechanism

createProvider() automatically handles dynamic model publishing and persistence. The custom provider's refreshModels() receives a read-only snapshot of context.stored and publishes changes via context.publish({ persist?, update? }) — do not mutate state directly before publishing.

14.3 Connecting to OpenAI-Compatible Endpoints like DeepSeek

DeepSeek is the most typical OpenAI-compatible vendor. The good news is that pi-ai auto-detects compatibility settings for known compatible vendors (DeepSeek is in the built-in list) and works out of the box with zero configuration:

import { createModels, createProvider, envApiKeyAuth } from '@earendil-works/pi-ai';

const deepseek = createProvider({
  id: 'deepseek',
  name: 'DeepSeek',
  auth: envApiKeyAuth('DEEPSEEK_API_KEY'),
  api: 'openai-completions',
  baseUrl: 'https://api.deepseek.com/v1',
  models: [
    {
      id: 'deepseek-chat',
      name: 'DeepSeek Chat',
      reasoning: false,
      input: ['text'],
    },
    {
      id: 'deepseek-reasoner',
      name: 'DeepSeek Reasoner',
      reasoning: true, // R1 supports reasoning
      input: ['text'],
    },
  ],
});

14.4 compat: Fine-Grained Compatibility Flags

For custom proxies not on the auto-detection list, you can manually declare differences with the compat field. Common flags:

const proxy = createProvider({
  id: 'corp-proxy',
  name: 'Corp Gateway',
  auth: envApiKeyAuth('CORP_LLM_KEY'),
  api: 'openai-completions',
  baseUrl: 'https://llm-gateway.corp.internal/v1',
  models: [/* ... */],
  compat: {
    supportsStore: false,            // The gateway does not support the store field
    supportsDeveloperRole: false,    // Only recognizes system, not developer role
    maxTokensField: 'max_tokens',    // Older gateways do not recognize max_completion_tokens
    thinkingFormat: 'deepseek',      // Send thinking parameters in DeepSeek format
  },
});

Quick reference for common compat flags:

Flag	Purpose
supportsStrictMode	Whether tool definitions support strict JSON schema
requiresToolResultName	Whether tool results must carry a name field
requiresThinkingAsText	Whether thinking blocks must be converted to text
thinkingFormat	Reasoning parameter format (openai/deepseek/qwen, etc.)
cacheControlFormat: 'anthropic'	Anthropic-style prompt caching control

Partial settings inherit detection results

You only need to write fields that differ in compat — unspecified fields fall back to auto-detected defaults based on baseUrl.

14.5 Direct API Implementation Calls and Model-Level Headers

A more low-level approach is to bypass the catalog and call the API implementation directly; additionally, individual models can carry dedicated headers (e.g. headers needed to bypass bot detection) that are automatically merged into requests:

// Custom models can carry headers; automatically merged into requests
const gateway = createProvider({
  id: 'gateway',
  name: 'LLM Gateway',
  auth: envApiKeyAuth('GATEWAY_KEY'),
  api: 'openai-completions',
  baseUrl: 'https://gw.example.com/v1',
  models: [
    {
      id: 'internal-chat',
      name: 'Internal Chat',
      headers: { 'X-Bot-Detection-Bypass': 'team-token' }, // Model-level request headers
    },
  ],
});

// Multi-tenant gateway: same implementation, different baseUrl
const tenantGateway = createProvider({ /* similar, swap baseUrl and tenant key */ });

Chapter Summary

• Three scenarios need custom providers: local inference, enterprise gateways, OpenAI-compatible third parties;
• createProvider() assembles from id + auth + models + api; dynamic publishing is managed by the library;
• DeepSeek and other known compatible vendors have compat settings auto-detected based on baseUrl, usually zero-config;
• compat only needs to declare differing fields; the rest inherit from detection results;
• Model-level headers can inject custom request headers to bypass detection.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What core parts are needed to build a provider with createProvider()?

2. When using a known OpenAI-compatible vendor like DeepSeek, how are compat settings handled?

3. If only thinkingFormat is set in compat, what happens to the other fields?

4. A model's response is blocked by a corporate gateway's bot detection. What does the document recommend?

🛠️ Hands-on Practice

1. Use createProvider() to switch the Agent from Chapter 4 to a local Ollama model and verify offline operation.
2. Configure DeepSeek's two models (chat/reasoner), run the same question with each, and compare results and costs.
3. Set up a minimal Nginx reverse proxy as an "enterprise gateway," intentionally remove support for a field, and fix it with a compat flag.

Test-driven development离不开 mocks — next chapter introduces the Faux Provider designed specifically for testing.