Gateway

GetLLM exposes three layered APIs over one base URL and one inference key (gl-live-…). Pick the layer that matches what you're building — they're additive, not exclusive.

How they relate: Chat = Memory + Gateway, composed for you. Gateway is the foundation (raw model access), Memory adds long-term memory, and Chat is the ready-to-use layer — the same one the GetLLM web app runs on. Full endpoint reference is in the API Reference tab.

1. Sign in and create an API key in the dashboard.
2. Copy the key — it starts with gl-live-. Treat it like a password and never commit it to source control.
3. Top up your wallet on the billing page. Calls deduct from this balance based on upstream token usage — see Models for current pricing.
4. Point your SDK's base URL at https://www.getllm.ai/v1 (OpenAI dialect) or https://www.getllm.ai (Anthropic dialect).
5. Make a request. Every response includes an x-getllm-request-id header and per-call cost / token headers — see Response headers below.

There is no separate org / project / region setup. One key works against every supported model. Use X-GetLLM-* headers to opt into gateway features like fallback, max-price caps or BYOK.

All API calls require a Bearer token in the Authorization header. The token is the API key you generated in the dashboard.

• Production keys start with gl-live- and bill against the workspace wallet.
• Account / management endpoints (manage inference keys, usage, generations, token counting, BYOK) authenticate with a management key (gl-mgmt-...) — see Management keys — not an inference API key.
• Keys can be revoked at any time from the dashboard. Revocation takes effect within a few seconds.

GetLLM routes to the model you specify by id. The same id works regardless of which dialect you call — under the hood the gateway translates between OpenAI- and Anthropic-style request shapes.

• Anthropic — claude-opus-4-*, claude-sonnet-4-*, claude-haiku-4-*.
• OpenAI — gpt-4o, gpt-4o-mini, o1, o3, dall-e-3, tts-1, whisper-1.
• Google — gemini-2.5-pro, gemini-2.5-flash.
• Voyage — embeddings such as voyage-3 and voyage-3-lite.
• DeepSeek — deepseek-chat, deepseek-reasoner.

For the live catalogue, prices and capabilities call GET /v1/models or browse the Models page.

Pass "stream": true in the body to receive a Server-Sent Events stream. GetLLM forwards upstream events unchanged so existing SSE clients keep working.

In addition, the gateway emits a final hermes.usage event right before the stream closes. It carries the canonical request id, the billed model and per-call cost and token counts:

event: hermes.usage
data: {"request_id":"...","model":"gpt-4o","cost_microcents":12345,
       "usage":{"input_tokens":120,"output_tokens":420,
                "cache_read_tokens":0,"cache_creation_tokens":0,"reasoning_tokens":0}}

Clients that don't care about cost can safely ignore unknown event types. The values match the x-getllm-* headers returned on non-streaming calls.

Every request is routed across providers by a quality × fair-share algorithm. You can steer it per API key (in the dashboard under Routing) or per request with the X-GetLLM-Route header:

• auto (default) — balances quality and price automatically.
• cheapest — bias toward the lowest-priced channel for the model.
• fastest — bias toward the lowest-latency channel.
• pool:<slug> — pin requests to a specific line pool (Direct / Turbo / Economy).

Set a per-key default in the dashboard; the X-GetLLM-Route header overrides it for a single call.

# Per-request override — route this call to the cheapest channel.
curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "X-GetLLM-Route: cheapest" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}]}'

Sticky sessions & prompt caching

Provider prompt caches live per upstream account, so the gateway pins each session to one upstream and your cache discounts actually apply. Session identity is resolved automatically, most explicit signal first:

• x-session-id header or a top-level session_id body field — explicit and recommended for long agent sessions (the body field is stripped before forwarding).
• Anthropic metadata.user_id or OpenAI prompt_cache_key/user — picked up automatically; Claude Code already sends a per-session value, no configuration needed.
• No signal at all — requests from one account still stick together, they just don't separate by session.

Pinning applies when a model's cache pricing makes it worthwhile (discounted cache reads), and falls back to the next-best channel automatically if the pinned one degrades. Verify hits via cache_read_tokens in the usage block of every response.

Send X-GetLLM-Max-Price-Microcents to reject a call before it runs if its estimated cost exceeds the cap. Useful for agents and long-running jobs where you want a hard ceiling per call.

• Unit: microcents. 1,000,000 = 1 cent, 100,000,000 = $1.
• The estimate uses your declared max_tokens and the upstream price for the model.
• A capped call fails fast with HTTP 403 and never bills.

curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "Content-Type: application/json" \
  -H "X-GetLLM-Max-Price-Microcents: 5000000" \  # 5 cent
  -d '{"model":"gpt-4o","max_tokens":1024,"messages":[{"role":"user","content":"hi"}]}'

Send a comma-separated X-GetLLM-Fallback-Models header to have the gateway retry the call against the next model in the list if the primary model returns 5xx or is unavailable.

• Order matters. The gateway tries each model left-to-right.
• Only transient upstream failures (5xx, timeout, overload) trigger the fallback. 4xx errors from the primary are returned as-is.
• The actually-billed model is reported in x-getllm-billed-model.
• Streaming responses fall back only if the failure happens before the first byte is sent — once the stream starts it cannot be retried.

curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "X-GetLLM-Fallback-Models: gpt-4o, gpt-4o-mini" \
  -d '{"model":"gpt-5","messages":[{"role":"user","content":"hi"}]}'

Send X-GetLLM-Translate: 1 to have the gateway translate non-English user messages into English before forwarding to the model, then translate the reply back to the user's language.

• Detection is automatic — English content is passed through unchanged.
• System prompts are never modified.
• Streaming is supported; deltas are buffered into sentence-level chunks before translation.
• Tool / function calls and structured-output JSON are preserved verbatim.

# Translate user content while keeping tool calls intact.
curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "X-GetLLM-Translate: 1" \
  -d '{
    "model":"claude-sonnet-4-6",
    "stream":true,
    "messages":[{"role":"user","content":"weather in SF?"}],
    "tools":[{"type":"function","function":{"name":"get_weather",
              "parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}]
  }'

Already have a contract with OpenAI, Anthropic or another upstream? Upload your key to GetLLM and route traffic through it with X-GetLLM-BYOK: 1. You still get GetLLM's logging, headers and dashboards, but the upstream bill goes to your account.

• Keys are encrypted at rest and only decrypted in memory for the call.
• GetLLM charges a flat per-call gateway fee for BYOK requests; no margin on tokens.
• You can upload one key per provider per workspace and rotate it at any time.

# Upload a key (one-time, via a management key — see Management keys).
curl https://www.getllm.ai/v1/byok/keys \
  -H "Authorization: Bearer gl-mgmt-..." \
  -d '{"provider":"openai","api_key":"sk-...","label":"my OpenAI"}'

# Use the uploaded key on a regular API call.
curl https://www.getllm.ai/v1/chat/completions \
  -H "Authorization: Bearer gl-live-..." \
  -H "X-GetLLM-BYOK: 1" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}]}'

Management keys (gl-mgmt-...) authenticate the account / management API programmatically — manage inference keys, query usage / generations, and upload BYOK keys — without exposing your browser session. Mint and revoke them in the dashboard under Management keys.

• A management key authenticates the account endpoints only; it cannot make LLM calls — use a regular API key (gl-live-...) for inference.
• A regular API key (gl-live-...) calls models but cannot manage your account.

# Create a new inference API key with a management key.
curl https://www.getllm.ai/v1/management-keys \
  -H "Authorization: Bearer gl-mgmt-..." \
  -d '{"name":"new inference key"}'

# Query your recent usage with a management key.
curl https://www.getllm.ai/v1/usage/recent \
  -H "Authorization: Bearer gl-mgmt-..."

OpenAI-compatible batch processing: upload a JSONL of requests, get results within the completion window at half the standard per-token price (the provider's batch discount passes through to you).

• Billing happens once, when the batch completes; output downloads unlock after the debit lands (402 until then — top up and they unlock automatically).
• Files and batches are pinned to the upstream account that created them; always create the batch from a file you uploaded here.
• Input/output files are capped at ~32 MiB each in this release.

# upload → create → poll → download
curl https://www.getllm.ai/v1/files   -H "Authorization: Bearer gl-live-..."   -F purpose=batch -F file=@requests.jsonl

curl https://www.getllm.ai/v1/batches   -H "Authorization: Bearer gl-live-..."   -d '{"input_file_id":"file-...","endpoint":"/v1/chat/completions","completion_window":"24h"}'

curl https://www.getllm.ai/v1/batches/batch_... -H "Authorization: Bearer gl-live-..."
curl https://www.getllm.ai/v1/files/file-.../content -H "Authorization: Bearer gl-live-..."

An organization gives a team one shared wallet, per-project budgets, and per-member roles on top of the same API you already use. Structure: Organization → Project → API key. Solo accounts are simply a personal org of one — nothing changes until you create or join a team.

Joining an organization

• Two ways in: an org owner or admin adds you on the dashboard's Members page by your registered email, or they share an invite code that you redeem from the account menu at the top right of the console — code joins wait for an admin's approval.
• Your role decides what you can do: owner (everything incl. billing), admin (projects/members/keys), member (create keys, see own usage), billing (reports + top-ups only).
• You can belong to many orgs at once; your personal account is untouched — keys you created before joining keep billing your personal wallet.
• Personal BYOK stays personal-only: a team key's X-GetLLM-BYOK never uses your personal provider key — it uses the organization's keys, which only an org admin can configure.

Personal vs. organization — same tools, different owner

The API, models, key scopes, routing modes, and BYOK mechanism are identical in both contexts. What changes is who OWNS the wallet and the settings — a team org just adds shared, admin-managed layers on top of the personal ones:

• Wallet — Personal: your own balance, topped up on Credits / Redeem. Organization: one shared wallet all members bill against, funded by a card top-up or a one-way transfer-in from a member's personal balance. A request always bills the wallet of the key's owner (personal key to your wallet, team key to the org wallet).
• Plans and quota — Personal only: a monthly subscription and its quota apply to keys in your personal account. Team orgs are pay-as-you-go only — there is no org subscription, so team keys always draw from the org wallet at usage rates.
• BYOK — Same encryption and provider list; only the owner differs. Personal BYOK keys (BYOK page, personal context) are used only by your personal keys. Organization BYOK keys (BYOK page, team context, admin only) are used by any team key that sends X-GetLLM-BYOK; a project-scoped key overrides the org-wide one. A team key can never reach a member's personal BYOK — closing the gap where private keys would invisibly carry org traffic at zero cost.
• Routing — Same modes (auto / cheapest / fastest / pool:<tier>). Personal: a default per key, overridable per request with X-GetLLM-Route. Organization adds two lower default layers admins set on the Routing page — an org-wide default and per-project overrides. The effective preference resolves request header, then key, then project, then org, then auto; it only picks WITHIN the pool tier and never widens it.
• Guardrails cascade — A model allowlist can be set at the org, the project, and the key; the effective set is their intersection (org AND project AND key). Each layer can only narrow what the layer above allows, never widen it — so an admin's org-level ceiling holds no matter what a member sets on their own key.

How team billing works

• A request made with a team key bills the organization wallet, pay-as-you-go — never the member personal balance or personal plan quota.
• Admins can set a monthly budget per project; once spent, that project keys get 429 until the month rolls over or the budget is raised. Budgets are ceilings, not balances — money lives only on the org wallet.
• Projects can also carry a model allowlist that narrows what any key inside them may call (key-level scopes narrow further, never widen).

Topping up the org wallet

• Members with owner, admin, or billing roles top up from the Org billing page → "Top up this org wallet" — the checkout banner confirms the credits go to the org, not your personal balance.
• Org top-ups do not affect your personal account: they never clear a personal block and do not count toward referral rewards.
• When the org wallet runs low, the org owner gets the low-balance email (the person who can fund it).
• Besides card top-ups, anyone with billing capability can transfer credits from their personal balance into the org wallet (one-way), and the Org billing page shows the org balance plus a transaction history of every top-up, usage debit, refund, and transfer.

Sharing & attribution

• Create keys under an org/project from the dashboard key form, or via the API with org_id + project_id:
• Every request is attributed to its project — the chargeback report (Org usage page, with the team organization selected) breaks spend down by project × model for any window.
• Members see their own usage; billing/admin/owner roles see the whole org usage.
• Org admins and owners can view and revoke every team key from the API keys page with the team organization selected — and removing a member automatically revokes their team keys.
• Org admins also set routing defaults (org-wide and per-project, on the Routing page) and organization BYOK keys (on the BYOK page) — a team key that sends X-GetLLM-BYOK then runs on the organization's own provider account.

# mint a key that bills the org and reports under the project
curl https://www.getllm.ai/v1/keys   -H "Authorization: Bearer gl-mgmt-..."   -d '{"name":"backend-prod","org_id":"<org>","project_id":"<project>"}'

Full API shapes are in the API reference (orgs, members, projects, usage endpoints).

Every response — streaming or not — carries a uniform set of GetLLM headers so you can attribute cost, debug routing and reconcile billing without parsing the body.

x-getllm-request-id        # Canonical id; quote it in support requests.
x-getllm-cost-microcents   # Total cost of this call in microcents.
x-getllm-tokens-input
x-getllm-tokens-output
x-getllm-tokens-cache-read
x-getllm-tokens-cache-creation
x-getllm-tokens-reasoning
x-getllm-billed-model      # May differ from requested model if fallback fired.

On streaming calls the cost / token headers are sent with the initial response, then refined in the final hermes.usage SSE event.

GetLLM passes upstream rate limits through and adds a thin layer of its own based on your plan. Limits and current usage are reported on the dashboard. The gateway uses standard HTTP status codes; the error field in the JSON body always carries a human-readable reason.

Common error responses:

• 401 — missing or invalid API key. Check that the Authorization header is Bearer gl-live-....
• 402 — wallet balance is zero or below the per-call minimum. Top up on the billing page.
• 403 with error.code = max_price_exceeded — the estimated cost is above your X-GetLLM-Max-Price-Microcents cap.
• 403 with error.code = quota_exceeded — workspace daily / monthly cap reached.
• 404 — unknown model id or unknown request id on the generations endpoint.
• 429 — upstream or gateway rate limit hit. Retry after the Retry-After header value.
• 5xx — upstream is unhealthy. The same request will be retried automatically if you set X-GetLLM-Fallback-Models.