Point any OpenAI- or Anthropic-compatible SDK at GetLLM by overriding two things: the base URL and the API key (create one). Everything else — models, tools, streaming — stays the same.

OpenAI Python SDK

OpenAI-family SDKs append /chat/completions to base_url, so the base must include /v1.

from openai import OpenAI

client = OpenAI(
    api_key="gl-live-...",
    base_url="https://www.getllm.ai/v1",
)
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role":"user","content":"hello"}],
)
print(resp.choices[0].message.content)

Anthropic Python SDK

The Anthropic SDK appends /v1/messages itself, so the base is the bare host — no /v1.

from anthropic import Anthropic

client = Anthropic(
    api_key="gl-live-...",
    base_url="https://www.getllm.ai",
)
resp = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role":"user","content":"hello"}],
)
print(resp.content[0].text)

Streaming with curl

curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","stream":true,"messages":[{"role":"user","content":"write a poem"}]}'

Image, audio and embedding endpoints follow the standard OpenAI shape and are billed per upstream call.

These examples use plain curl and a gl-live-... key against the OpenAI-style base https://www.getllm.ai/v1.

# Image generation
curl https://www.getllm.ai/v1/images/generations \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"dall-e-3","prompt":"a cat","n":1,"size":"1024x1024"}'

# Text-to-speech
curl https://www.getllm.ai/v1/audio/speech \
  -H "Authorization: Bearer gl-live-..." \
  -o speech.mp3 \
  -d '{"model":"tts-1","input":"hello world","voice":"alloy"}'

# Speech-to-text (Whisper)
curl https://www.getllm.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer gl-live-..." \
  -H "X-GetLLM-Model: whisper-1" \
  -F file=@audio.mp3 \
  -F response_format=json

# Embeddings
curl https://www.getllm.ai/v1/embeddings \
  -H "Authorization: Bearer gl-live-..." \
  -d '{"model":"voyage-3","input":"hello world"}'

Set X-GetLLM-Model when the request body can't carry a model field (for example multipart uploads to /v1/audio/transcriptions).

Anthropic's claude CLI reads ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN from the environment — set both to point it at GetLLM and every session runs through your wallet.

Prerequisites

• Claude Code installed (npm i -g @anthropic-ai/claude-code).
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

Set the two environment variables (in your shell profile to make them persistent). GetLLM speaks the Anthropic dialect, so the base URL is the bare host — no /v1 suffix.

# Point Claude Code at GetLLM
export ANTHROPIC_BASE_URL="https://www.getllm.ai"
export ANTHROPIC_AUTH_TOKEN="gl-live-..."

Verify

# Verify it works
claude --version
claude "hello, who am I talking to?"

GetLLM speaks the full Anthropic /v1/messages dialect, including tools, vision and streaming, so every Claude Code feature works unchanged.

Claude Code can burn through tokens fast on long sessions. Pair it with a X-GetLLM-Max-Price-Microcents cap (see below) to bound a single call's cost.

The official codex and openai CLIs both read OPENAI_API_KEY and OPENAI_BASE_URL. Set them once and every command flows through GetLLM.

Prerequisites

• The Codex or OpenAI CLI installed (e.g. npm i -g @openai/codex or pip install openai).
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

Set the two environment variables. OpenAI-family tools append /chat/completions to the base, so it must include /v1.

export OPENAI_API_KEY="gl-live-..."
export OPENAI_BASE_URL="https://www.getllm.ai/v1"

Verify

# Codex
codex --model gpt-4o "refactor this function"

# Plain OpenAI CLI
openai api chat.completions.create -m gpt-4o-mini -g user "say hi"

Any tool built on top of the OpenAI Python or Node SDK (Aider, sgpt, llm, …) works the same way — just set the same two environment variables.

Cline is a VS Code agent extension. Configure it to talk to GetLLM via the OpenAI-compatible dialect:

Prerequisites

• Install the Cline extension in VS Code.
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

1. Open the Cline sidebar and click the settings gear icon.
2. Set API Provider to OpenAI Compatible and fill in:
   • Base URL: https://www.getllm.ai/v1
   • API Key: your gl-live-... key
   • Model ID: any GetLLM model id, e.g. gpt-4o or claude-sonnet-4-6

Verify

Save and start a new task — Cline now talks to GetLLM.

Watch the per-call cost on the recent generations page if you're iterating on prompts.

Continue lets you mix providers in one config. Set apiBase per model — use the /v1 base for provider: openai entries and the bare host for provider: anthropic entries.

Prerequisites

• The Continue extension installed in VS Code or JetBrains.
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

Edit ~/.continue/config.json and set apiKey and apiBase on each model. Mind the base per provider: /v1 for OpenAI, bare host for Anthropic.

// ~/.continue/config.json
{
  "models": [
    {
      "title": "Claude Sonnet 4.6 (via GetLLM)",
      "provider": "anthropic",
      "model": "claude-sonnet-4-6",
      "apiKey": "gl-live-...",
      "apiBase": "https://www.getllm.ai"
    },
    {
      "title": "GPT-4o (via GetLLM)",
      "provider": "openai",
      "model": "gpt-4o",
      "apiKey": "gl-live-...",
      "apiBase": "https://www.getllm.ai/v1"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Tab Autocomplete",
    "provider": "openai",
    "model": "gpt-4o-mini",
    "apiKey": "gl-live-...",
    "apiBase": "https://www.getllm.ai/v1"
  }
}

Verify

Open the Continue chat panel, pick one of the models you declared and send a message — the reply confirms the route.

tabAutocompleteModel is called on every keystroke, so prefer a small / cheap model like gpt-4o-mini there.

Zed's assistant supports custom api_url per provider block. Configure both openai and anthropic to route through GetLLM:

Prerequisites

• Zed installed (the assistant panel ships with the editor).
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

Edit ~/.config/zed/settings.json and set api_url under each provider — /v1 for openai, bare host for anthropic. Put your gl-live-... key in Zed's API-key prompt for each provider.

// ~/.config/zed/settings.json
{
  "language_models": {
    "openai": {
      "api_url": "https://www.getllm.ai/v1",
      "available_models": [
        { "name": "gpt-4o",      "max_tokens": 128000 },
        { "name": "gpt-4o-mini", "max_tokens": 128000 }
      ]
    },
    "anthropic": {
      "api_url": "https://www.getllm.ai",
      "available_models": [
        { "name": "claude-sonnet-4-6", "max_tokens": 200000 },
        { "name": "claude-haiku-4-5",  "max_tokens": 200000 }
      ]
    }
  },
  "assistant": {
    "default_model": {
      "provider": "anthropic",
      "model":    "claude-sonnet-4-6"
    },
    "version": "2"
  }
}

Verify

Open the assistant panel, pick a model from available_models and send a prompt — a reply confirms the route.

Restart Zed after editing settings.json. The assistant picker will then list every model you declared under available_models.

Cursor's Models setting accepts a custom OpenAI base URL. Use GetLLM there to route Cursor's chat / Cmd-K / Composer traffic through your wallet.

Prerequisites

• Cursor installed.
• A GetLLM API key with the gl-live- prefix — create one from the dashboard.

Configure

1. Open Cursor Settings → Models.
2. Scroll to OpenAI API Key, paste your gl-live-... key, and click Verify.
3. Expand OpenAI Base URL (under the key field), enable the toggle and set the URL to https://www.getllm.ai/v1.
4. Under Model Names, enable the OpenAI-compatible models you want (e.g. gpt-4o, gpt-4o-mini).
5. Disable Cursor's built-in models or set GetLLM-routed ones as default — that's it.

Verify

Start a chat with one of the enabled OpenAI-compatible models (or hit Cmd-K) — a reply confirms Cursor is routing through GetLLM.

Cursor's Background agents and Privacy mode still apply on top of GetLLM. Make sure your team's policy allows routing through a third-party gateway before enabling this in shared workspaces.

Anthropic models in Cursor go through Cursor's own infra and can't be redirected — only the OpenAI-compatible slot is user-configurable. Use Claude Code or Continue if you need GetLLM-routed Claude inside an editor.

When something looks off, drop down to curl. The recipes below mirror what the SDKs do under the hood — they're the fastest way to confirm the gateway, your key and a given model are all healthy.

You'll need curl and jq installed, plus a gl-live-... key (create one). Management-key recipes use a gl-mgmt-... key instead.

Health check

curl -s https://www.getllm.ai/healthz | jq
# {
#   "status": "ok",
#   "db": true,
#   "tunnel": true
# }

List models

No auth required. Browse the live catalogue exactly as the dashboard does.

curl -s https://www.getllm.ai/v1/models \
  | jq '.data[] | {id, provider: .hermes.provider}'

Non-streaming chat completion

curl -i https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "max_tokens": 100,
    "messages": [{"role":"user","content":"Reply with the single word: pong."}]
  }'

The response includes the x-getllm-* cost and token headers — useful for budgeting and for confirming which upstream model was billed.

Streaming Anthropic call

curl -N https://www.getllm.ai/v1/messages \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 500,
    "stream": true,
    "messages": [{"role":"user","content":"Count from 1 to 5, one per line."}]
  }'

The -N flag disables curl's output buffering so you can see the SSE deltas land in real time. The final event is always message_stop, immediately followed by hermes.usage.

Cap the cost of a single call

# Reject the call if its estimated cost exceeds 10 cents.
# Value is in microcents (1 cent = 1,000,000).
curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -H "X-GetLLM-Max-Price-Microcents: 10000000" \
  -d '{
    "model": "gpt-4o",
    "max_tokens": 4000,
    "messages": [{"role":"user","content":"hi"}]
  }'

Auto-translate user content

curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -H "X-GetLLM-Translate: 1" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 200,
    "messages": [{"role":"user","content":"What's the weather like today?"}]
  }'

With X-GetLLM-Translate: 1 the gateway translates non-English user content into English before sending it upstream and translates the model's reply back. Tool / function calls are preserved verbatim. See Claude Code for an end-to-end setup.

Run the full smoke test

The repo ships a scripts/smoke-test.sh script that exercises every dialect (chat, messages, streaming, embeddings, …) against a live key. Use it after rotating a key or before reporting a bug.

# Run all checks with your key.
HERMES_KEY=gl-live-... ./scripts/smoke-test.sh

# Expected tail of the output:
# -- Test 5: /v1/messages — Anthropic dialect, streaming --
#   OK  message_start (claude-haiku-4-5)
#   OK  39 content delta(s)
#   OK  message_stop
#
# 7/7 checks passed

Look up a single call

# Fetch the canonical record for one request id.
curl https://www.getllm.ai/v1/generations/<request-id> \
  -H "Authorization: Bearer gl-mgmt-..."
# Returns the billed model, status, token counts and cost
# as well as duration_ms and whether the call used BYOK.
# Authenticate with a management key (gl-mgmt-...), not an inference API key.

GetLLM is the memory layer (/v1/memory/*). Ingest what the user says, then retrieve relevant context to inject into your own model calls — memory that survives across sessions, scoped to your key (optionally partitioned per end-user with sub_tenant).

Uses the same gl-live-... key (create one) as the gateway, against https://www.getllm.ai/v1/memory/*.

# 1) Store a turn (optionally extracting durable facts)
curl https://www.getllm.ai/v1/memory/ingest \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"user_said":"I prefer Python over Go","extract_facts":true}'

# 2) Retrieve relevant memory for the next prompt
curl https://www.getllm.ai/v1/memory/query \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"text":"what languages do I like?","top_k":8}'
# → { "rendered": "...context to inject...", "items": [ ... ] }

Inject the returned `rendered` string as a system message in your own /v1/chat/completions call — that's the whole loop. Or skip building it yourself and use the Chat API below.

The Chat API (/v1/bot/*) is the assistant layer: it queries Memory, calls the model, persists the turn, and streams the reply — the same engine the GetLLM web Chat runs on. Sessions hold the transcript; you only send the next message.

Uses the same gl-live-... key (create one) as the gateway, against https://www.getllm.ai/v1/bot/*.

Streaming chat with memory

# Omit session_id to start a new session (its id arrives in the first event)
curl https://www.getllm.ai/v1/bot/chat \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"message":"hi — remember my name is Sam","model":"gpt-4o"}'
# Server-Sent Events: event: session → delta chunks → event: done

Generate images & video

# Image generation is synchronous — returns the stored assistant message
curl https://www.getllm.ai/v1/bot/generate-image \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"prompt":"a red panda coding at night"}'

# Video generation is async — submit, then poll the job
curl https://www.getllm.ai/v1/bot/generate-video \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -d '{"prompt":"a timelapse of a city at dusk"}'
# → { "job_id": "...", "status": "processing" }  —  poll /v1/bot/video-jobs/{id}

Manage conversations with /v1/bot/sessions (list / create / get / rename / delete). Attachments use /v1/bot/upload-ticket. See the API Reference for full schemas.