Batch LLM API

POST /llm/completions is a non-realtime, request/response LLM call over the same account, the same API key, and the same balance as your voice agents. Use it for the text-only steps around a call — classification, extraction, summarisation, JSON generation — without standing up a separate LLM provider.

It routes through the same LLM gateway your agents use, so you don't add a vendor or a key. No agent required — you pick the model directly.

Request

POST /llm/completions

{
  "messages": [
    { "role": "user", "content": "Extract the caller's intent as one word." }
  ],
  "model": "gpt-oss-120b",
  "temperature": 0.7,
  "system_prompt": "You are a precise classifier.",
  "response_format": { "type": "json_schema", "schema": { "type": "object", "required": ["intent"] } },
  "max_json_retries": 2
}

response_format

• { "type": "text" } — default.
• { "type": "json_object" } — the model is asked to return valid JSON.
• { "type": "json_schema", "schema": { ... } } — the response is parsed and validated against your JSON Schema server-side. On a validation failure the model is re-prompted with the validator error, up to max_json_retries times. If it still can't produce schema-valid JSON, the request returns 422 with code: "schema_validation_failed" — you never get back a payload that doesn't match your schema.

Response

{
  "id": "e7c1...",
  "model": "gpt-oss-120b",
  "content": "{\"intent\":\"reschedule\"}",
  "usage": { "prompt_tokens": 812, "completion_tokens": 143, "total_tokens": 955 },
  "upstream_cost_usd": 0.00097,
  "cost_usd": 0.00107
}

usage is the real token accounting from the provider.

Billing

You pay the model's list price for the tokens it used, plus a flat 10% routing fee:

cost_usd = upstream_cost_usd × 1.10

upstream_cost_usd is the per-token list price for the chosen model; cost_usd is what was debited from your balance. Debited from the same balance as calls. If the balance can't cover it, the request returns 402 with code: "insufficient_balance" before the model is called.

Errors