The talk WebSocket

WS /calls/{call_id}/talk is the streaming protocol for a live call: you send mic audio, you get transcript text, reply text, and reply audio back, one turn at a time. call_id comes from POST /agents/{agent_id}/calls and is single-use.

Connecting

const socket = new WebSocket(wsUrl); // wss://api.wakili.dev/calls/{call_id}/talk
socket.binaryType = "arraybuffer";

No auth header, no query-string token — see Authentication for why. The server closes with code 4404 if call_id is unknown, already used, or expired.

call_config frame

Immediately after the connection opens, the server sends one JSON text frame carrying the agent's realtime knobs — barge-in, endpointing, language lock, and tier — so the client can configure its own capture/playback behaviour:

{
  "type": "call_config",
  "interruption": { "enabled": true, "sensitivity": 0.5 },
  "endpointing": { "sensitivity": 0.5 },
  "language_lock": null,
  "model_tier": "balanced",
  "latency_profile": "balanced"
}

These come from the agent's configuration. The frame is additive — a client that doesn't recognise type: "call_config" can ignore it and behave exactly as before.

Sending a turn

1. While the user is speaking, send their mic audio as binary frames (any chunk size — the server just concatenates everything it receives per turn). The reference client records audio/webm via MediaRecorder and sends a chunk every 250ms.
2. When they stop, send one text frame: {"type":"end_turn"}.

The server buffers all binary frames received since the last turn ended, and on end_turn runs them through STT → LLM → TTS for that turn.

Sending end_turn with no buffered audio is a no-op (ignored, no events emitted) — safe to call defensively.

Receiving events

Every server→client message is either a JSON text frame (an event, shape below) or a binary frame (raw reply audio bytes — TTS output, not chunked/framed further).

type TalkEvent =
  | { type: "call_config"; interruption: object; endpointing: object; language_lock: string | null; model_tier: string; latency_profile: string } // sent once, right after connect
  | { type: "transcript"; text: string }        // STT result for the turn
  | { type: "reply_text"; text: string }         // full LLM reply text
  | { type: "reply_audio_end" }                  // all binary audio frames for this reply have been sent
  | { type: "error"; message: string }           // "stt_failed" | "llm_failed" | "tts_failed"
  | { type: "call_ended"; reason: string };       // "idle_timeout" | "provider_error" | normal disconnect

Sequence for one turn:

client → binary frame(s)      (mic audio)
client → {"type":"end_turn"}
server → {"type":"transcript","text":"..."}
server → {"type":"reply_text","text":"..."}
server → binary frame(s)      (reply audio)
server → {"type":"reply_audio_end"}

Concatenate the binary frames received between reply_text and reply_audio_end to reassemble the full reply audio (e.g. AudioContext.decodeAudioData on the merged buffer).

Errors and call end

If STT, the LLM, or TTS fails mid-turn, the server sends {"type":"error","message":"stt_failed"|"llm_failed"|"tts_failed"} immediately followed by {"type":"call_ended","reason":"provider_error"}, then closes the socket (code 1011). The call is billed for whatever duration elapsed before the failure.

If no message arrives from the client for the idle timeout window, the server sends {"type":"call_ended","reason":"idle_timeout"} and closes (code 1000). A normal client-initiated close ends the call with no call_ended event (just a disconnect).

The server always closes the socket right after call_ended — don't wait on anything further.

Billing

The whole call is billed as one debit when it ends: wall-clock duration × the agent's per-minute rate (sum of its LLM/STT/TTS component rates). Individual turns aren't billed separately. If your account balance can't cover the call, POST /agents/{agent_id}/calls returns 402 before a call_id is ever issued — the WebSocket step is never reached.

Reference client

A minimal push-to-talk implementation (TalkClient, ~90 lines, no dependencies) is in the Next.js integration guide — send mic audio while a button is held, send end_turn on release, play back the reply.