Admin Implementation Guide — Getting Started with LLM Invocations

This guide is for workspace admins of newly onboarded odnoga customers. It assumes your tenant, workspace, and LLM provider configuration (OpenAI, Anthropic, etc.) are already set up. This guide covers the operational steps required to enable your first LLM invocations.

Phase 1: Authentication & Access Control

1.1 Mint your first API key

1. Log in to the odnoga dashboard.

2. Navigate to Workspace → Virtual keys → New key.

3. Choose a scope based on your deployment model:

   Scope	Use for	Permissions
   server	Backend services, edge functions, Lambda	Full access. No restrictions. Best for backend-only systems.
   browser	Browser apps, mobile clients	Restricted: cannot pin explicit version_id for prompts; must use label instead.

4. Choose an environment:

   • sk_live_… → Production; real costs incurred.
   • sk_test_… → Sandbox; no costs; test models only (if configured).

5. Copy the key immediately and store it securely:

   # .env or your secrets manager
   AIROUTER_API_KEY=sk_live_xxxxxxxxxxxxxxxxxxxxxxxx
   AIROUTER_BASE_URL=https://api.odnoga.com/functions/v1/airouter-openai-compat
   

6. Add these credentials to your CI/CD secrets, environment variables, or secrets manager (AWS Secrets Manager, HashiCorp Vault, etc.).

Security notes:

• Treat API keys as passwords; rotate them quarterly via Virtual keys → Rotate.
• Revoke keys when team members leave: Virtual keys → Revoke.
• Every key action is logged in Key audit for compliance.

Phase 2: Fire Your First Request

2.1 Validate connectivity with a test call

Use cURL to verify the routing is live and credentials are valid:

curl "$AIROUTER_BASE_URL/v1/chat/completions" \
  -H "authorization: Bearer $AIROUTER_API_KEY" \
  -H "content-type: application/json" \
  -H "x-airouter-end-user: test_user_001" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello, odnoga!"}]
  }' -i

Expected response (HTTP 200):

x-airouter-request-id: 8b1a…
x-airouter-vendor: openai
x-airouter-model: gpt-4o-mini
x-airouter-cost-usd: 0.000043
x-airouter-latency-ms: 612
x-airouter-cache: miss
x-ratelimit-remaining-usd: 49.9999

Troubleshooting:

• 401 Unauthorized → Check AIROUTER_API_KEY format and validity. Re-copy from dashboard.
• 404 Not found → Verify AIROUTER_BASE_URL matches your Supabase project.
• 400 Model not allowed → The model is not in your workspace's allowed list. See Phase 3.1.

Phase 3: Configure Routing & Model Access

3.1 Define which models your team can use

1. Navigate to Workspace → Routing → Allowed models.

2. Toggle the setting:

   • Off (default) → All active platform models are allowed.
   • On → Only models you explicitly select are allowed.

3. If toggled on, select the subset you want to allow:

   • gpt-4o-mini (recommended for cost efficiency)
   • gpt-4o (for complex reasoning)
   • claude-3-5-sonnet (if Anthropic is configured)
   • etc.

4. Save. Any requests for models outside this list will now return 400 model_not_allowed.

Recommendation: Start with a small set (e.g., gpt-4o-mini + one other) and expand as patterns emerge.

3.2 (Optional) Set up fallback models

If a primary model becomes unavailable, odnoga can automatically retry with a secondary:

1. Navigate to Workspace → Routing → Failover policies.
2. Define a chain (e.g., gpt-4o → gpt-4o-mini → claude-3-5-sonnet).
3. Requests to the primary will fall back if they fail due to rate limits, server errors, or availability issues.

Phase 4: Per-End-User Identification

4.1 Always send x-airouter-end-user

This is the most critical step for observability and billing.

On every LLM request, include:

-H "x-airouter-end-user: your_stable_user_id"

Rules:

• Max 256 characters.
• Must be stable (e.g., your internal user ID, not a session token).
• Use the same value consistently for a given user.

Why it matters:

• ✅ Per-user cost rollups in Overview → Top end users.
• ✅ Per-user budgets (if you enable them).
• ✅ Sticky A/B bucket assignment for managed prompts.
• ✅ End-user attribution in audit logs and request history.

Bad examples:

• uuid() (changes every request — breaks analytics).
• Session tokens (can expire).
• Anonymous (omitted header).

Good examples:

• user_12345
• org_acme_user_42
• customer:stripe_cus_xyz

4.2 Verify end-user tracking

1. Make a few requests with x-airouter-end-user headers.
2. Go to Workspace → Overview → Top end users.
3. You should see your user IDs and their cumulative usage.

If the list is empty, check that the header is being sent on every request.

Phase 5: Budget & Cost Controls

5.1 Set a workspace-level budget

1. Navigate to Workspace → Budgets → Add budget.

2. Enter:

   • Amount (USD, e.g., $100.00).
   • Renewal (daily, weekly, monthly, or custom).
   • Alert thresholds (e.g., notify at 50%, 80%, 100%).

3. Save.

Behavior:

• At 80%: An alert is generated; you receive email notification.
• At 100%: All new requests return 402 Payment required until the budget resets.

5.2 Set per-model costs (optional, for chargeback)

If you bill your own customers based on model usage:

1. Navigate to Superadmin → Models/Pricing (superadmin only).
2. For each model, set input and output token costs.
3. odnoga will calculate per-request costs and include them in x-airouter-cost-usd header.

You can then bill your customers using this cost data.

Phase 6: Use SDK Integration

6.1 Choose your SDK

odnoga is OpenAI-compatible, so you can use any OpenAI client library. Pick one:

Node.js (OpenAI SDK)

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.AIROUTER_API_KEY,
  baseURL: process.env.AIROUTER_BASE_URL,
  defaultHeaders: {
    'x-airouter-end-user': 'user_123',
  },
});

const response = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Hello!' }],
});

Python (OpenAI SDK)

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv('AIROUTER_API_KEY'),
    base_url=os.getenv('AIROUTER_BASE_URL'),
)

response = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role': 'user', 'content': 'Hello!'}],
    headers={'x-airouter-end-user': 'user_123'},
)

cURL (reference)

curl "$AIROUTER_BASE_URL/v1/chat/completions" \
  -H "authorization: Bearer $AIROUTER_API_KEY" \
  -H "content-type: application/json" \
  -H "x-airouter-end-user: user_123" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello!"}]}'

Vercel AI SDK

import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
  apiKey: process.env.AIROUTER_API_KEY,
  baseURL: process.env.AIROUTER_BASE_URL,
});

const result = await generateText({
  model: openai('gpt-4o-mini'),
  prompt: 'Hello!',
  headers: {
    'x-airouter-end-user': 'user_123',
  },
});

See SDK Cookbooks for language-specific examples (Go, Java, Ruby, etc.).

Phase 7: Managed Prompts (Recommended)

Instead of hardcoding prompts in code, define them once in odnoga and reuse them.

7.1 Create a managed prompt

1. Navigate to Workspace → Prompts → Create prompt.

2. Enter:

   • Slug (identifier, e.g., welcome-email).
   • System message (role/instructions).
   • User message template (can include {{variables}}).

   Example:

   System: You are a friendly billing assistant for {{tenant_name}}.
   
   User: I am {{user_full_name}}. My plan is {{plan}}.
   Can you help me with {{question}}?
   

3. Save. The prompt is now in draft state.

4. To promote to production, click Activate next to the version.

7.2 Call a managed prompt

curl "$AIROUTER_BASE_URL/v1/chat/completions" \
  -H "authorization: Bearer $AIROUTER_API_KEY" \
  -H "content-type: application/json" \
  -H "x-airouter-end-user: user_123" \
  -d '{
    "model": "gpt-4o-mini",
    "prompt": {
      "slug": "welcome-email",
      "label": "production",
      "variables": {
        "tenant_name": "Acme Corp",
        "user_full_name": "Alice Brown",
        "plan": "Pro",
        "question": "Can I upgrade?"
      }
    }
  }'

Benefits:

• ✅ Prompts are versioned and audited.
• ✅ Non-technical team members can edit prompts in the dashboard.
• ✅ A/B experiments support via labels.
• ✅ Reduced payload size (send slug + variables, not full prompt).

7.3 A/B test prompts

1. Create a new version of the prompt with different wording.

2. Activate it with a label (e.g., variant_b).

3. Set up an experiment:

   • Navigate to Workspace → Prompts → Experiments.
   • Canary (%) → Pick how many users see the new version.
   • Save.

4. Send requests with x-airouter-end-user. odnoga will:

   • Deterministically bucket users (same user always sees the same variant).
   • Route requests to the appropriate prompt version.
   • Track metrics separately per variant.

Phase 8: Observability & Monitoring

8.1 Monitor requests and costs

1. Navigate to Workspace → Requests.
2. View:
   • Model used.
   • Cost (USD).
   • Latency (ms).
   • Cache hit/miss.
   • Error details (if any).
3. Filter by:
   • Date range.
   • Model.
   • End user.
   • Status (success, error, etc.).

8.2 View daily usage trends

1. Go to Workspace → Overview → Daily usage.
2. See:
   • Total requests.
   • Total spend.
   • Top models.
   • Top end users.
3. Drill down by model to see per-model costs.

8.3 Set up alerts

1. Navigate to Workspace → Alerts → Create alert.

2. Pick an alert type:

   • Budget threshold (80%, 100%).
   • Error rate (e.g., > 5% failures).
   • Latency (p95 > 2 seconds).

3. Set notification channels:

   • Email.
   • (Future: webhooks, Slack).

4. Save. You'll receive notifications when thresholds are breached.

Phase 9: Caching (Optional, for Cost Reduction)

If you have repeated requests or long context windows:

9.1 Enable prompt caching

1. Navigate to Workspace → Cache → Toggle cache on.
2. Ensure your model supports caching (most recent models do).

How it works:

• odnoga checksums the prompt/context.
• If an identical prompt is seen again within the TTL (default: 5 minutes), the cached response is returned.
• Cost is 90% lower for cache hits vs. full inference.

9.2 Monitor cache effectiveness

In Workspace → Requests, the x-airouter-cache header will show:

• hit → Served from cache.
• miss → Fresh inference.
• off → Cache disabled.

Phase 10: Hardening & Scale (Optional)

10.1 Rate limiting (prevent cost spikes)

If you want to prevent individual users or apps from over-using:

1. Navigate to Workspace → Rate limits.

2. Define per-user or per-key limits:

   • Tokens/minute (e.g., 100,000).
   • Requests/hour (e.g., 1,000).

3. Save. Requests exceeding limits return 429 rate_limited.

10.2 Audit and compliance

All actions are logged in Workspace → Key audit:

• Who used which key.
• When.
• Outcome (success/error).
• Request ID (for correlation).

Export or review logs for compliance/debugging.

Checklist: Go-Live Readiness

Use this checklist to verify you're ready for production:

• API Key minted and stored securely (server scope, production environment).
• First request validated (HTTP 200 from test call).
• End-user identification implemented (all requests include x-airouter-end-user).
• Allowed models defined (routing rules set).
• Budget configured with alert thresholds.
• SDK integrated into backend/edge code.
• Observability tested (requests visible in dashboard).
• Error handling implemented (gracefully handle 402, 400, 5xx, etc.).
• Monitoring set up (daily cost review, alerts configured).
• Prompts managed (if using managed prompts, at least one is active).
• Key audit reviewed (understanding who accesses what).

Common Patterns

Pattern 1: Backend service with per-tenant isolation

// backend/routes/ask.ts
async function handleAsk(req: Request) {
  const { tenant_id, user_id, question } = await req.json();
  
  const response = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    prompt: {
      slug: 'customer-support',
      label: 'production',
      variables: {
        company_name: await getCompanyName(tenant_id),
        user_name: await getUserName(user_id),
      },
    },
    messages: [{ role: 'user', content: question }],
  }, {
    headers: {
      'x-airouter-end-user': `tenant:${tenant_id}:user:${user_id}`,
    },
  });

  return {
    answer: response.choices[0].message.content,
    cost_usd: response.headers['x-airouter-cost-usd'],
  };
}

Pattern 2: Browser app with backend proxy

// Backend: proxy endpoint
app.post('/api/chat', async (req, res) => {
  const { message, session_id } = req.body;
  
  const response = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: message }],
  }, {
    headers: {
      'x-airouter-end-user': session_id,
    },
  });

  res.json({ reply: response.choices[0].message.content });
});

// Frontend: call your backend (never call odnoga directly from browser)
const response = await fetch('/api/chat', {
  method: 'POST',
  body: JSON.stringify({ message, session_id }),
});

Pattern 3: Streaming responses (e.g., chat UI)

const stream = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Write a poem' }],
  stream: true,
}, {
  headers: {
    'x-airouter-end-user': user_id,
  },
});

for await (const event of stream) {
  if (event.choices[0]?.delta?.content) {
    process.stdout.write(event.choices[0].delta.content);
  }
}

Troubleshooting

Next Steps

1. Concepts — deeper dive into budgets, caching, prompts.
2. Using your own data in prompts — pull from your DB, render via {{variables}}.
3. End-user billing — charge your own customers per-usage via Stripe.
4. Headers reference — exhaustive list of response headers.
5. Errors reference — all error codes and recovery strategies.

Support

• Dashboard: https://[your-workspace].airouter.io
• Email: support@airouter.io
• Docs: https://airouter.io/docs