Build voice conversations

Use WS /v1/voice to build a spoken conversation with an available model or saved Agent. One connection handles transcription, model replies, and speech generation. For the ready-made user experience, see Voicebot.

Before you begin

You need the public BSQAI API origin, an authenticated identity in the intended tenant, chat.message.create, an accessible model, and configured transcription and speech-generation services. Using an Agent also requires agent.configuration.use and read access to that Agent in the tenant.

For browser microphone capture, use a secure context and request microphone permission. Implement capture and playback separately from the WebSocket: the socket transports raw audio, not browser media controls.

1. Discover a voice

Call GET /v1/tts/engines with your normal API authentication. Choose an engine name and one of its voice IDs with the required language. Do not hard-code a voice that may not be installed.

tts must specify voice, lang, or both. When both are supplied, the voice must support that language. You can omit engine only when the deployment has a single engine.

2. Connect and configure

Connect to the API origin over wss: at /v1/voice. Browser clients pass token and tenant_id as URL query parameters because the browser WebSocket interface cannot set authentication headers. token accepts a bearer JWT or BSQAI API key. tenant_id is required for JWT identities with multiple tenants; API keys and single-tenant identities resolve their tenant automatically.

Build the query using URL encoding. Keep token-bearing URLs out of logs, screenshots, analytics, and error messages.

Send exactly one JSON text message before sending any audio. This example uses a model, an installed English voice, and mono float32 PCM captured at 48 kHz. Replace the placeholders and set the sample rate to the actual capture rate:

{
  "agent": { "model": "<accessible-model-id>" },
  "audio": { "encoding": "pcm_float32", "sample_rate_hz": 48000 },
  "tts": { "engine": "<installed-engine>", "voice": "<english-voice-id>", "lang": "en" },
  "stt": {
    "inference_config": { "language": "en" },
    "pipeline_config": { "partial_enabled": true, "partial_interval_s": 0.5 }
  },
  "chat_id": "<client-generated-UUID>",
  "store": true
}

To use an Agent, replace agent with { "agent_id": "<agent-UUID>" }. Supply exactly one of model and agent_id. An Agent contributes only its model and system prompt in this endpoint, not its MCP tools, Skills, file retrieval, or web search.

The required top-level fields are agent, audio, and tts. Omit chat_id and store for an ephemeral session; store defaults to false. When store is true, chat_id is required. Use a new UUID for a new conversation or an existing caller-owned chat ID to continue its history. Providing chat_id with store: false seeds history without persisting new turns.

The deployment's live API reference exposes the full VoiceConfigFrame schema, including supported audio formats and turn settings. Its GET /v1/voice entry documents the WebSocket; it is not an HTTP endpoint that starts an audio conversation.

3. Stream and render

After the configuration, send microphone audio as binary messages in the declared format. Do not send further text messages or a media container while claiming raw PCM.

Read both kinds of server frames:

• Binary: raw signed PCM int16, 24 kHz, mono, in whole 40 ms chunks. Queue it in a playback buffer configured for that format; it is not a WAV file.
• Text: JSON events. Dispatch on type and ignore unknown event types for forward compatibility.

For example, {"type":"agent_text","text":"Good morning."} appends text; it is not another synthesis request.

Keep sending microphone audio while playing the answer to support interruption. On end or navigation, close the socket and release microphone tracks and playback resources.

4. Handle incomplete turns and failures

The default wait for a final transcript after speech stops is three seconds. If it arrives too late, the service promotes the last partial to a final with promoted: true. That text may be incomplete and can still trigger an answer. Display a notice and let the user correct it. With no usable transcript, there is no answer.

A fatal in-session error can be followed by close code 1000, so a normal-looking close code alone does not prove success. A minute with neither side speaking also closes normally, without an error. Connection and processing failures are not automatically retried by the service.

5. Verify conversation continuity

With store: true, finished user turns and delivered agent_text are saved incrementally. Read the conversation through GET /ui/chats/{id} or reopen it in the Portal. An interrupted reply preserves only its already-delivered text, and a dropped connection preserves completed turns rather than unfinished speech.

Test a short exchange, interrupt an answer, end the session, and reopen the same chat. Check the transcript, retained reply portions, microphone cleanup, and behavior when permission or connectivity is lost.

Related pages

• Voicebot user guide
• Use models through the API
• Speaches