LLM gateway

One LLM gateway for every model you ship.

Swap the base URL and route OpenAI, Anthropic, Gemini, Mistral and Groq through one endpoint — with routing policies, spend caps, a per-request cost ledger and EU or US regional routing.

Two lines to move behind the gateway

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.odnoga.com/functions/v1/airouter-openai-compat/v1",   // <- the gateway
  apiKey: process.env.ODNOGA_API_KEY,        // <- your odnoga key
});

const res = await client.chat.completions.create({
  model: "auto",                             // or "gpt-5-mini", "claude-haiku-4-5", ...
  messages: [{ role: "user", content: "Hello" }],
});

What the gateway does

Routing in front, accounting behind.

One OpenAI-compatible endpoint

Point your existing SDK at odnoga and call OpenAI, Anthropic, Gemini, Mistral, Groq and more without rewriting client code.

Routing policies and failover

Route by cost, latency or capability, with automatic fallback to a healthy model when a vendor degrades.

Budgets, quotas and concurrency

Hard and soft spend caps per tenant, workspace, end user and model, plus a tenant-wide concurrency pool.

Per-request cost ledger

Every call writes an exact ledger entry: tokens in and out, vendor price, your margin, who to bill.

Regional routing with evidence

Choose EU or US vendor endpoints and keep the recorded evidence of where each request was actually served.

Versioned prompts and A/B

Prompts live in the gateway, not hardcoded in your repo — version them, A/B them, roll back in one click.

Architecture

How an LLM gateway works, request by request

A gateway is a control plane in front of the model vendors. Your app authenticates once against the gateway; the gateway resolves policy, picks a model, calls the vendor with its own stored credentials, and writes the accounting record before the response is handed back.

Authenticate and identify

The gateway API key resolves the tenant, workspace and — when you send an end-user header — the individual customer the call belongs to.

Admission control

Budget, quota and concurrency checks run before any vendor is contacted, so an overspend is refused rather than discovered on next month’s invoice.

Routing decision

A routing policy resolves the requested alias to a concrete model based on cost, latency, capability and residency, and keeps ordered fallbacks for degraded vendors.

Normalized vendor call

One OpenAI-shaped request is translated into each vendor’s dialect — tools, vision, reasoning effort and web search differ per provider.

Metering and ledger write

Token counts and vendor list prices produce an exact per-request ledger entry with your margin, attributable to a workspace or an end user.

Observability and evidence

Request logs keep the model actually used, the fallback path, the region served and the latency, so incidents and audits have a record to read.

your app ──▶ gateway ──▶ auth + budget check ──▶ routing policy ──▶ vendor (EU/US endpoint)
                 │                                                        │
                 └──────────── ledger entry + request log ◀───────────────┘

When you do not need one: a single app calling a single model, with no per-customer cost question and no compliance requirement, is usually better off calling the vendor directly. A gateway earns its place the moment there is more than one model, more than one customer, or more than one person changing prompts.

When you need one

Signs your app has outgrown direct vendor calls

Model names and prompts are hardcoded across edge functions and nobody knows which version is live.
You cannot say how much a single customer costs you in AI this month.
A vendor outage or an out-of-credit key takes your feature down with no fallback.
You need EU routing for some customers and cannot evidence where requests were served.
Adding a new provider means another SDK, another key store and another billing line.

FAQ

LLM gateway questions

What is an LLM gateway?

An LLM gateway is a single API endpoint that sits between your application and every model provider you use. Instead of your code holding an OpenAI key here and an Anthropic key there, the gateway owns the vendor keys and gives you one interface plus the operational layer around it: routing, retries, spend limits, logging and cost attribution.

How is a gateway different from an LLM proxy?

A proxy forwards requests. A gateway also decides where the request should go, enforces limits before the call, and records what it cost afterwards. odnoga is a gateway: policy in front, ledger behind.

Do I have to change my code?

Change the base URL and the API key. The chat completions and embeddings surfaces are OpenAI-compatible, so official SDKs, LangChain, n8n and most tools work unchanged.

Whose vendor keys are used?

Yours. You add your own OpenAI, Anthropic or Gemini keys to a workspace and odnoga routes through them, so you keep your vendor pricing and rate limits.

Can I bill my own customers through it?

Yes. Send one header identifying your end user and every request is attributed and billable — see multi-tenant billing.

Can my coding agent manage it?

Yes. odnoga exposes its control plane over MCP, so Claude, Cursor or your own agent can read usage, manage keys, prompts and routing policies directly.