Skip to main content

Overview

The LLM Gateway exposes one gateway URL, https://llm-gateway.aptible.com, and speaks several API dialects from it. Pick whichever matches how you’re already integrating:
  • OpenAI-compatible: Chat Completions, Responses, and Embeddings, in both a strict /openai namespace and as un-namespaced defaults
  • Anthropic-compatible: Messages, in a strict /anthropic namespace
  • Decisions: typed questions about an input, with one structured answer per question, in OpenAI’s shape and in the gateway’s own
  • Agent harnesses: a dedicated, never-changing base URL per supported coding assistant or agent harness, under /harness/<name>
Every request authenticates with an LLM Key and selects a model by name in the request body — see Supported Models for the current model catalog. The List Models endpoints return only the models your key can call, in the format of that API. Paths must match exactly. A path with a trailing slash, such as /openai/v1/chat/completions/, returns 404 Not Found.

Authentication

Send your LLM Key as a bearer token:
Anthropic’s SDKs (and Claude Code) instead send an x-api-key header — the gateway accepts either:

OpenAI-compatible API

Use the /openai namespace for strict OpenAI compatibility: Chat Completions, Responses, and Embeddings are also available un-namespaced (/v1/chat/completions, /v1/responses, /v1/embeddings) as the gateway’s default dialect. Use whichever your SDK or tooling expects. The un-namespaced /v1/decisions takes the gateway’s own request shape; see Decisions API. For full request and response documentation, see OpenAI’s API reference.

Anthropic-compatible API

Use the /anthropic namespace for Anthropic Messages: For full request and response documentation, see Anthropic’s API reference.

Decisions API

A decision asks one or more typed questions about an input, such as “Does this message contain a refund request?” It returns one answer per question, in the order asked. Use it to classify, score, or check content without parsing a model’s free-form text. Both routes take a model, an input, and a questions array. Each question has a type, instructions, and an optional name that its answer carries back: An answer whose type is refusal means the model declined to answer that question. Every response names the model you requested in model. The two routes differ in their usage fields. /openai/v1/decisions reports input_tokens and output_tokens, as OpenAI does. /v1/decisions reports prompt_tokens. /v1/decisions returns 422 for a request in the deprecated map form, which names its questions as keys of a questions object and sends its input as state. Send questions as an array and the input as input. Decisions work on many models, including Claude and open-weight models, not only on models with their own decisions endpoint, such as gpt-6-luna. For a model without one, the gateway answers the decision through the model’s Responses API. It bills that decision at the model’s chat token rates. The gateway tells every provider not to store these requests, so a decision keeps the model’s data retention mode. Check the Decisions column in Supported Models for the models that support decisions.

Agent harness support

Some coding assistants and agent harnesses need a single base URL they can be pointed at once and never have to edit again — even as the underlying API dialect they speak evolves. The LLM Gateway provides one under /harness/<name>, which resolves internally to the right compatibility endpoint: See the Claude Code guide for a full walkthrough of harness setup.
Don’t see your harness or coding assistant listed? Any tool that speaks OpenAI- or Anthropic-compatible APIs already works against the /openai or /anthropic endpoints above — a dedicated /harness route just saves you from ever having to change the base URL again. Contact us if you’d like a harness added.

Example requests

Every example below uses the model field to select a model — see Supported Models for valid IDs.

Request IDs

Every response includes an x-trace-id header with the gateway’s ID for the request. Include it when you contact Aptible Support about a request. To match gateway responses to your own logs, send an x-request-id header with your own ID. The gateway returns it in the response’s x-request-id header, cut to its first 64 characters. If you don’t send one, x-request-id carries the same ID as x-trace-id.

Prompt caching

Prompt caching works through the LLM Gateway the same way it does with the provider directly: mark cacheable content with cache_control (Anthropic), or rely on the provider’s automatic caching (OpenAI). The gateway serves each model from a pool of upstream providers. A provider’s cache only helps requests that reach that same provider, so to keep a conversation’s cache warm, send a session-id (or x-session-id) header that stays the same for the whole conversation:
Requests that share a session ID are routed to the same upstream for up to an hour after the last request. Use a new, unique value, such as a UUID, for each conversation or agent run. Without a session ID, requests are spread across upstreams and see more cache misses.
Claude Code, Codex CLI, OpenCode, and Pi already send their own session headers, which the gateway recognizes, so no configuration is needed for them.
A request that fails over to another provider during an outage starts with a cold cache on that provider.

Automatic provider failover and client-side fallbacks

If a model is available from more than one provider, the gateway automatically fails over to another available provider during an outage — no client changes required. You can also specify your own fallback models in a request; see Supported Models for details on both.