WS /v1/tts/stream WS

One WebSocket, many utterances, served in order. The client sends JSON text messages; the server answers with JSON events (text frames) and audio (binary frames).

wss://api.vartalaap.si/v1/tts/stream

Auth on the handshake: x-api-key: <key> or Authorization: Bearer <key> header, or ?api_key=<key> on the URL (browsers). Missing/invalid key → the upgrade is refused with 401.

Client → server

A JSON object per utterance, same fields as POST /v1/tts:

{"text": "आपकी EMI ₹12,450 है, जो 5 नवंबर को due है.", "voice": "anjura", "format": "pcm16", "request_id": "turn-12-s0"}

• format: pcm16 or opus. wav is downgraded to pcm16 (no container on a message stream); the start event says which is in effect.
• raw, en_digits, temperature, seed behave as in REST. stream is ignored (always streaming).
• Send the next utterance any time; it queues behind the current one on this connection. Binary messages from the client are rejected with an error event (expected a JSON text message).

Server → client

Text frames are JSON events, each with the utterance's request_id:

type	Fields	When
start	voice, format, chunks, normalized	request accepted and a worker is assigned; normalized is exactly what will be spoken
chunk	chunk (index), text, tokens	a chunk begins synthesis
first_audio_ms	chunk, ms	sent right after the first binary audio frame of the utterance; ms is measured from the request message
chunk_end	chunk, frames, audio_s, prefill_ms, prefill_cached (bool), eos (bool)	a chunk finished; prefill_cached tells you whether its text prefill came from the cache
end	audio_s, frames, chunks, ttfa_ms (engine-side), wall_ms, rtf	utterance complete; the last binary frame (e.g. final Ogg page) was sent before it
error	error, optional status, retry_after	validation (400/413), capacity (status: 503, retry_after), timeout, engine error. The connection stays open.

Binary frames are audio for the utterance currently in progress: raw pcm16 samples (s16le, 24 kHz, mono; the first frame is 80 ms = 3 840 bytes, later frames ~320 ms) or Ogg/Opus pages. For Opus the first binary frame is the Ogg header pages.

Timeline of one utterance

→ {"text":"…","voice":"anjura","format":"pcm16","request_id":"t1"}
← start          chunks=2  normalized="…"
← chunk 0        tokens=31
← <audio 3840 B>
← first_audio_ms ms=44.2
← <audio …> ×N
← chunk_end 0    frames=39 audio_s=3.12 prefill_cached=true eos=true
← chunk 1 … <audio …> … chunk_end 1
← end            audio_s=4.9 ttfa_ms=17.8 wall_ms=690.3 rtf=0.141

Capacity and errors

If every worker is busy and the queue is full, or the queued request waited more than 2 s, the utterance gets {"type":"error","request_id":"t1","status":503,"retry_after":1,"error":"busy"} and nothing is billed; resend it after retry_after seconds. The connection is unaffected. Per-utterance wall limit: 120 s → {"type":"error","error":"timeout"}.

Cancelling

Close the socket. Synthesis of the current utterance stops within one frame and queued utterances are dropped; you are billed for the audio produced so far. Open a new socket for the next turn (see voice agents & barge-in).

Example

import asyncio, json, os, websockets

async def speak(lines):
    headers = {"x-api-key": os.environ["VARTALAAP_API_KEY"]}
    async with websockets.connect("wss://api.vartalaap.si/v1/tts/stream", additional_headers=headers) as ws:
        for i, line in enumerate(lines):
            await ws.send(json.dumps({"text": line, "voice": "anjura", "format": "pcm16", "request_id": f"s{i}"}))
        pending = len(lines)
        async for msg in ws:
            if isinstance(msg, bytes): play(msg); continue
            ev = json.loads(msg)
            if ev["type"] == "first_audio_ms": print(ev["request_id"], "first audio", ev["ms"], "ms")
            elif ev["type"] == "end": print(ev["request_id"], ev["audio_s"], "s rtf", ev["rtf"]); pending -= 1
            elif ev["type"] == "error":
                if ev.get("status") == 503: await asyncio.sleep(ev.get("retry_after", 1)); await ws.send(json.dumps({...}))
                else: print("error", ev); pending -= 1
            if pending == 0: break