Choose a channel that supports the features your workflow uses. A successful connection test verifies connectivity; it does not test every tool or application version.

Not available: Responses background jobs and their polling/cancellation lifecycle, Realtime/WebSocket, images/edits and rerank. Use foreground requests for Responses.

Once output starts, an upstream stream failure is returned in the stream; the gateway does not join answers from different channels. Retrying a tool workflow should follow the application’s recovery logic.

Point any OpenAI- or Anthropic-compatible SDK at GetLLM by overriding two things: the base URL and the API key (create one). Everything else — models, tools, streaming — stays the same.

OpenAI Python SDK

OpenAI-family SDKs append /chat/completions to base_url, so the base must include /v1.

from openai import OpenAI

client = OpenAI(
    api_key="gl-live-...",
    base_url="https://www.getllm.ai/v1",
)
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role":"user","content":"hello"}],
)
print(resp.choices[0].message.content)

Anthropic Python SDK

The Anthropic SDK appends /v1/messages itself, so the base is the bare host — no /v1.

from anthropic import Anthropic

client = Anthropic(
    api_key="gl-live-...",
    base_url="https://www.getllm.ai",
)
resp = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role":"user","content":"hello"}],
)
print(resp.content[0].text)

Streaming with curl

curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","stream":true,"messages":[{"role":"user","content":"write a poem"}]}'

Image, audio and embedding endpoints follow the standard OpenAI shape and are billed per upstream call.

These examples use plain curl and a gl-live-... key against the OpenAI-style base https://www.getllm.ai/v1.

# Image generation
curl https://www.getllm.ai/v1/images/generations \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"dall-e-3","prompt":"a cat","n":1,"size":"1024x1024"}'

# Text-to-speech
curl https://www.getllm.ai/v1/audio/speech \
  -H "Authorization: Bearer gl-live-..." \
  -o speech.mp3 \
  -d '{"model":"tts-1","input":"hello world","voice":"alloy"}'

# Speech-to-text (Whisper)
curl https://www.getllm.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer gl-live-..." \
  -H "X-GetLLM-Model: whisper-1" \
  -F file=@audio.mp3 \
  -F response_format=json

# Embeddings
curl https://www.getllm.ai/v1/embeddings \
  -H "Authorization: Bearer gl-live-..." \
  -d '{"model":"voyage-3","input":"hello world"}'

Set X-GetLLM-Model when the request body can't carry a model field (for example multipart uploads to /v1/audio/transcriptions).

Connect Nous Research's Hermes Agent to GetLLM with its custom OpenAI-compatible provider.

After installing Hermes Agent, update the model section of ~/.hermes/config.yaml (or your active profile's config.yaml) with the values below. Replace the key and MODEL_ID, preserving your other settings. Select a model with tool calling and at least 64,000 tokens of context, as required by Hermes Agent.

model:
  provider: "custom"
  base_url: "https://www.getllm.ai/v1"
  api_key: "gl-live-..."
  default: "MODEL_ID"

Save the configuration and start a new Hermes Agent session. Confirm that its welcome banner shows your selected model:

hermes

This configures the main chat model. Hermes Agent manages tools, skills, memory, and messaging connections; separately configured auxiliary models keep their own provider settings.

Compatibility: the client supports a custom API endpoint matching GetLLM. This guide is based on official configuration documentation; client end-to-end validation is pending.

Replace MODEL_ID with an available model ID from the Models page or GET /v1/models. Use a model and route that support Chat Completions. Tool calling, image input, and reasoning require matching model capabilities; enable those client settings only when supported.

Start with a short streamed conversation, then try a simple tool call if the model supports it. Check the model, token usage, and cost in your request logs.

Connect WorkBuddy to GetLLM through its Custom model provider. Requests use the standard Chat Completions API.

https://www.getllm.ai/v1/chat/completions

1. Open Settings → Model → Add model, then select Custom.
2. Enable Custom Protocol, paste the full URL above, and enter your GetLLM API key.
3. Enter an exact model ID, such as gpt-6-astra, that is available to your account and API key. Test the connection, then save.
4. Select your custom model in the conversation model selector instead of Auto.

Custom Protocol: ON
URL: https://www.getllm.ai/v1/chat/completions
API Key: gl-live-...
Model: gpt-6-astra

Alternative: turn Custom Protocol off and enter https://www.getllm.ai/v1. WorkBuddy will append /chat/completions. Match the URL to the switch setting.

If the model service rejects the test request, first check the switch and URL pairing above, then your API key, exact model ID, and models available to that key. Do not append /chat/completions twice.

Compatibility: the client supports a custom API endpoint matching GetLLM. This guide is based on official configuration documentation; client end-to-end validation is pending.

Replace MODEL_ID with an available model ID from the Models page or GET /v1/models. Use a model and route that support Chat Completions. Tool calling, image input, and reasoning require matching model capabilities; enable those client settings only when supported.

Start with a short streamed conversation, then try a simple tool call if the model supports it. Check the model, token usage, and cost in your request logs.

Connect OpenClaw to GetLLM as a custom model provider using the openai-completions API.

Merge this provider into your OpenClaw configuration (normally ~/.openclaw/openclaw.json). Replace the API key and MODEL_ID; preserve your existing providers and agent settings.

{
  "models": {
    "mode": "merge",
    "providers": {
      "getllm": {
        "baseUrl": "https://www.getllm.ai/v1",
        "apiKey": "gl-live-...",
        "api": "openai-completions",
        "models": [{ "id": "MODEL_ID", "name": "GetLLM model" }]
      }
    }
  }
}

Select the model explicitly after saving the provider:

openclaw models set getllm/MODEL_ID

GetLLM supplies model inference. OpenClaw continues to manage its tools, memory, scheduled tasks, and messaging connections. Set context limits, vision, and reasoning capabilities to match your chosen model.

Compatibility: the client supports a custom API endpoint matching GetLLM. This guide is based on official configuration documentation; client end-to-end validation is pending.

Replace MODEL_ID with an available model ID from the Models page or GET /v1/models. Use a model and route that support Chat Completions. Tool calling, image input, and reasoning require matching model capabilities; enable those client settings only when supported.

Start with a short streamed conversation, then try a simple tool call if the model supports it. Check the model, token usage, and cost in your request logs.

Connect OpenCode through a custom OpenAI-compatible provider. The example uses Chat Completions.

1. Run /connect, select Other, enter the provider ID getllm, and save your GetLLM API key.
2. Add a provider with the same ID to opencode.json, using the /v1 base URL and an available model ID.
3. Run /models and select the model under GetLLM.

OpenCode 1.x configuration example. OpenCode 2.x uses different configuration fields; follow its version-specific provider guide linked below with the same endpoint, key, and model ID.

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "getllm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "GetLLM",
      "options": { "baseURL": "https://www.getllm.ai/v1" },
      "models": { "MODEL_ID": { "name": "GetLLM model" } }
    }
  }
}

For a model and route that support /v1/responses, select the OpenAI Responses provider package for your OpenCode version instead of the Chat Completions package.

Compatibility: the client supports a custom API endpoint matching GetLLM. This guide is based on official configuration documentation; client end-to-end validation is pending.

Replace MODEL_ID with an available model ID from the Models page or GET /v1/models. Use a model and route that support Chat Completions. Tool calling, image input, and reasoning require matching model capabilities; enable those client settings only when supported.

Start with a short streamed conversation, then try a simple tool call if the model supports it. Check the model, token usage, and cost in your request logs.

Anthropic's claude CLI reads ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN from the environment — set both to point it at GetLLM and every session runs through your wallet.

Prerequisites

• Claude Code installed (npm i -g @anthropic-ai/claude-code).
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

Set the two environment variables (in your shell profile to make them persistent). GetLLM speaks the Anthropic dialect, so the base URL is the bare host — no /v1 suffix.

# Point Claude Code at GetLLM
export ANTHROPIC_BASE_URL="https://www.getllm.ai"
export ANTHROPIC_AUTH_TOKEN="gl-live-..."

Verify

# Verify it works
claude --version
claude "hello, who am I talking to?"

GetLLM speaks the full Anthropic /v1/messages dialect, including tools, vision and streaming, so every Claude Code feature works unchanged.

Claude Code can burn through tokens fast on long sessions. Pair it with a X-GetLLM-Max-Price-Microcents cap (see below) to bound a single call's cost.

The official codex and openai CLIs both read OPENAI_API_KEY and OPENAI_BASE_URL. Set them once and every command flows through GetLLM.

Prerequisites

• The Codex or OpenAI CLI installed (e.g. npm i -g @openai/codex or pip install openai).
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

Set the two environment variables. OpenAI-family tools append /chat/completions to the base, so it must include /v1.

export OPENAI_API_KEY="gl-live-..."
export OPENAI_BASE_URL="https://www.getllm.ai/v1"

Verify

# Codex
codex --model gpt-4o "refactor this function"

# Plain OpenAI CLI
openai api chat.completions.create -m gpt-4o-mini -g user "say hi"

Any tool built on top of the OpenAI Python or Node SDK (Aider, sgpt, llm, …) works the same way — just set the same two environment variables.

Cline is a VS Code agent extension. Configure it to talk to GetLLM via the OpenAI-compatible dialect:

Prerequisites

• Install the Cline extension in VS Code.
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

1. Open the Cline sidebar and click the settings gear icon.
2. Set API Provider to OpenAI Compatible and fill in:
   • Base URL: https://www.getllm.ai/v1
   • API Key: your gl-live-... key
   • Model ID: any GetLLM model id, e.g. gpt-4o or claude-sonnet-4-6

Verify

Save and start a new task — Cline now talks to GetLLM.

Watch the per-call cost on the recent generations page if you're iterating on prompts.

Continue lets you mix providers in one config. Set apiBase per model — use the /v1 base for provider: openai entries and the bare host for provider: anthropic entries.

Prerequisites

• The Continue extension installed in VS Code or JetBrains.
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

Edit ~/.continue/config.json and set apiKey and apiBase on each model. Mind the base per provider: /v1 for OpenAI, bare host for Anthropic.

// ~/.continue/config.json
{
  "models": [
    {
      "title": "Claude Sonnet 4.6 (via GetLLM)",
      "provider": "anthropic",
      "model": "claude-sonnet-4-6",
      "apiKey": "gl-live-...",
      "apiBase": "https://www.getllm.ai"
    },
    {
      "title": "GPT-4o (via GetLLM)",
      "provider": "openai",
      "model": "gpt-4o",
      "apiKey": "gl-live-...",
      "apiBase": "https://www.getllm.ai/v1"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Tab Autocomplete",
    "provider": "openai",
    "model": "gpt-4o-mini",
    "apiKey": "gl-live-...",
    "apiBase": "https://www.getllm.ai/v1"
  }
}

Verify

Open the Continue chat panel, pick one of the models you declared and send a message — the reply confirms the route.

tabAutocompleteModel is called on every keystroke, so prefer a small / cheap model like gpt-4o-mini there.

Zed's assistant supports custom api_url per provider block. Configure both openai and anthropic to route through GetLLM:

Prerequisites

• Zed installed (the assistant panel ships with the editor).
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

Edit ~/.config/zed/settings.json and set api_url under each provider — /v1 for openai, bare host for anthropic. Put your gl-live-... key in Zed's API-key prompt for each provider.

// ~/.config/zed/settings.json
{
  "language_models": {
    "openai": {
      "api_url": "https://www.getllm.ai/v1",
      "available_models": [
        { "name": "gpt-4o",      "max_tokens": 128000 },
        { "name": "gpt-4o-mini", "max_tokens": 128000 }
      ]
    },
    "anthropic": {
      "api_url": "https://www.getllm.ai",
      "available_models": [
        { "name": "claude-sonnet-4-6", "max_tokens": 200000 },
        { "name": "claude-haiku-4-5",  "max_tokens": 200000 }
      ]
    }
  },
  "assistant": {
    "default_model": {
      "provider": "anthropic",
      "model":    "claude-sonnet-4-6"
    },
    "version": "2"
  }
}

Verify

Open the assistant panel, pick a model from available_models and send a prompt — a reply confirms the route.

Restart Zed after editing settings.json. The assistant picker will then list every model you declared under available_models.

Cursor's Models setting accepts a custom OpenAI base URL. Use GetLLM there to route Cursor's chat / Cmd-K / Composer traffic through your wallet.

Prerequisites

• Cursor installed.
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

1. Open Cursor Settings → Models.
2. Scroll to OpenAI API Key, paste your gl-live-... key, and click Verify.
3. Expand OpenAI Base URL (under the key field), enable the toggle and set the URL to https://www.getllm.ai/v1.
4. Under Model Names, enable the OpenAI-compatible models you want (e.g. gpt-4o, gpt-4o-mini).
5. Disable Cursor's built-in models or set GetLLM-routed ones as default — that's it.

Verify

Start a chat with one of the enabled OpenAI-compatible models (or hit Cmd-K) — a reply confirms Cursor is routing through GetLLM.

Cursor's Background agents and Privacy mode still apply on top of GetLLM. Make sure your team's policy allows routing through a third-party gateway before enabling this in shared workspaces.

Anthropic models in Cursor go through Cursor's own infra and can't be redirected — only the OpenAI-compatible slot is user-configurable. Use Claude Code or Continue if you need GetLLM-routed Claude inside an editor.

When something looks off, drop down to curl. The recipes below mirror what the SDKs do under the hood — they're the fastest way to confirm the gateway, your key and a given model are all healthy.

You'll need curl and jq installed, plus a gl-live-... key (create one). Management-key recipes use a gl-mgmt-... key instead.

Health check

curl -s https://www.getllm.ai/healthz | jq
# {
#   "status": "ok",
#   "db": true,
#   "tunnel": true
# }

List models

No auth required. Browse the live catalogue exactly as the dashboard does.

curl -s https://www.getllm.ai/v1/models \
  | jq '.data[] | {id, provider: .hermes.provider}'

Non-streaming chat completion

curl -i https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "max_tokens": 100,
    "messages": [{"role":"user","content":"Reply with the single word: pong."}]
  }'

The response includes the x-getllm-* cost and token headers — useful for budgeting and for confirming which upstream model was billed.

Streaming Anthropic call

curl -N https://www.getllm.ai/v1/messages \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 500,
    "stream": true,
    "messages": [{"role":"user","content":"Count from 1 to 5, one per line."}]
  }'

The -N flag disables curl's output buffering so you can see the SSE deltas land in real time. The final event is always message_stop, immediately followed by hermes.usage.

Cap the cost of a single call

# Reject the call if its estimated cost exceeds 10 cents.
# Value is in microcents (1 cent = 1,000,000).
curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -H "X-GetLLM-Max-Price-Microcents: 10000000" \
  -d '{
    "model": "gpt-4o",
    "max_tokens": 4000,
    "messages": [{"role":"user","content":"hi"}]
  }'

Auto-translate user content

curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -H "X-GetLLM-Translate: 1" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 200,
    "messages": [{"role":"user","content":"What's the weather like today?"}]
  }'

With X-GetLLM-Translate: 1 the gateway translates non-English user content into English before sending it upstream and translates the model's reply back. Tool / function calls are preserved verbatim. See Claude Code for an end-to-end setup.

Run the full smoke test

The repo ships a scripts/smoke-test.sh script that exercises every dialect (chat, messages, streaming, embeddings, …) against a live key. Use it after rotating a key or before reporting a bug.

# Run all checks with your key.
HERMES_KEY=gl-live-... ./scripts/smoke-test.sh

# Expected tail of the output:
# -- Test 5: /v1/messages — Anthropic dialect, streaming --
#   OK  message_start (claude-haiku-4-5)
#   OK  39 content delta(s)
#   OK  message_stop
#
# 7/7 checks passed

Look up a single call

# Fetch the canonical record for one request id.
curl https://www.getllm.ai/v1/generations/<request-id> \
  -H "Authorization: Bearer gl-mgmt-..."
# Returns the billed model, status, token counts and cost
# as well as duration_ms and whether the call used BYOK.
# Authenticate with a management key (gl-mgmt-...), not an inference API key.

GetLLM is the memory layer (/v1/memory/*). Ingest what the user says, then retrieve relevant context to inject into your own model calls — memory that survives across sessions, scoped to your key (optionally partitioned per end-user with sub_tenant).

Uses the same gl-live-... key (create one) as the gateway, against https://www.getllm.ai/v1/memory/*.

# 1) Store a turn (optionally extracting durable facts)
curl https://www.getllm.ai/v1/memory/ingest \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"user_said":"I prefer Python over Go","extract_facts":true}'

# 2) Retrieve relevant memory for the next prompt
curl https://www.getllm.ai/v1/memory/query \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"text":"what languages do I like?","top_k":8}'
# → { "rendered": "...context to inject...", "items": [ ... ] }

Inject the returned `rendered` string as a system message in your own /v1/chat/completions call — that's the whole loop. Or skip building it yourself and use the Chat API below.

The Chat API (/v1/bot/*) is the assistant layer: it queries Memory, calls the model, persists the turn, and streams the reply — the same engine the GetLLM web Chat runs on. Sessions hold the transcript; you only send the next message.

Uses the same gl-live-... key (create one) as the gateway, against https://www.getllm.ai/v1/bot/*.

Streaming chat with memory

# Omit session_id to start a new session (its id arrives in the first event)
curl https://www.getllm.ai/v1/bot/chat \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"message":"hi — remember my name is Sam","model":"gpt-4o"}'
# Server-Sent Events: event: session → delta chunks → event: done

Generate images & video

# Image generation is synchronous — returns the stored assistant message
curl https://www.getllm.ai/v1/bot/generate-image \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"prompt":"a red panda coding at night"}'

# Video generation is async — submit, then poll the job
curl https://www.getllm.ai/v1/bot/generate-video \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"prompt":"a timelapse of a city at dusk"}'
# → { "job_id": "...", "status": "processing" }  —  poll /v1/bot/video-jobs/{id}

Manage conversations with /v1/bot/sessions (list / create / get / rename / delete). Attachments use /v1/bot/upload-ticket. See the API Reference for full schemas.