# Batch LLM API

`POST /llm/completions` is a non-realtime, request/response LLM call over the same account, the same API key, and the same balance as your voice agents. Use it for the text-only steps around a call — classification, extraction, summarisation, JSON generation — without standing up a separate LLM provider.

It routes through the same LLM gateway your agents use, so you don't add a vendor or a key. **No agent required** — you pick the model directly.

## Request

```
POST /llm/completions
```

```json
{
  "messages": [
    { "role": "user", "content": "Extract the caller's intent as one word." }
  ],
  "model": "gpt-oss-120b",
  "temperature": 0.7,
  "system_prompt": "You are a precise classifier.",
  "response_format": { "type": "json_schema", "schema": { "type": "object", "required": ["intent"] } },
  "max_json_retries": 2
}
```

| field | type | notes |
|---|---|---|
| `messages` | array | **required** — OpenAI-style, `role` is `system` \| `user` \| `assistant` |
| `model` | string | a concrete model id from `GET /providers` — takes precedence when set |
| `model_tier` | string | `fast` \| `balanced` \| `quality` — default `balanced`. Used only when `model` is omitted; maps to a concrete model server-side |
| `temperature` | number | `0`–`2`, default `0.7` |
| `system_prompt` | string | optional; defaults to a generic assistant prompt |
| `response_format` | object | see below — omit for plain text |
| `max_json_retries` | integer | `0`–`5`, default `2`. Only used with `json_schema` |

### `response_format`

- `{ "type": "text" }` — default.
- `{ "type": "json_object" }` — the model is asked to return valid JSON.
- `{ "type": "json_schema", "schema": { ... } }` — the response is parsed and validated against your [JSON Schema](https://json-schema.org/) server-side. On a validation failure the model is re-prompted with the validator error, up to `max_json_retries` times. If it still can't produce schema-valid JSON, the request returns `422` with `code: "schema_validation_failed"` — you never get back a payload that doesn't match your schema.

## Response

```json
{
  "id": "e7c1...",
  "model": "gpt-oss-120b",
  "content": "{\"intent\":\"reschedule\"}",
  "usage": { "prompt_tokens": 812, "completion_tokens": 143, "total_tokens": 955 },
  "upstream_cost_usd": 0.00097,
  "cost_usd": 0.00107
}
```

`usage` is the real token accounting from the provider.

## Billing

You pay the model's **list price** for the tokens it used, plus a flat **10% routing fee**:

```
cost_usd = upstream_cost_usd × 1.10
```

`upstream_cost_usd` is the per-token list price for the chosen model; `cost_usd` is what was debited from your balance. Debited from the same balance as calls. If the balance can't cover it, the request returns `402` with `code: "insufficient_balance"` before the model is called.

## Errors

| status | when |
|---|---|
| `401` | missing/invalid API key |
| `402` | insufficient balance |
| `422` | invalid body, unknown `model_tier`, or `schema_validation_failed` after retries |
| `502` | upstream LLM provider error (`code: "provider_error"`) |