# Build voice conversations (/docs/develop/voice-conversations) Use `WS /v1/voice` to build a spoken conversation with an available model or saved Agent. One connection handles transcription, model replies, and speech generation. For the ready-made user experience, see [Voicebot](/docs/ai/voicebot). ## Before you begin [#before-you-begin] You need the public BSQAI API origin, an authenticated identity in the intended tenant, `chat.message.create`, an accessible model, and configured transcription and speech-generation services. Using an Agent also requires `agent.configuration.use` and read access to that Agent in the tenant. For browser microphone capture, use a secure context and request microphone permission. Implement capture and playback separately from the WebSocket: the socket transports raw audio, not browser media controls. ## 1. Discover a voice [#1-discover-a-voice] Call `GET /v1/tts/engines` with your normal [API authentication](/docs/develop/use-local-models-via-api). Choose an engine name and one of its voice IDs with the required language. Do not hard-code a voice that may not be installed. `tts` must specify `voice`, `lang`, or both. When both are supplied, the voice must support that language. You can omit `engine` only when the deployment has a single engine. ## 2. Connect and configure [#2-connect-and-configure] Connect to the API origin over `wss:` at `/v1/voice`. Browser clients pass `token` and `tenant_id` as URL query parameters because the browser WebSocket interface cannot set authentication headers. `token` accepts a bearer JWT or BSQAI API key. `tenant_id` is required for JWT identities with multiple tenants; API keys and single-tenant identities resolve their tenant automatically. Build the query using URL encoding. Keep token-bearing URLs out of logs, screenshots, analytics, and error messages. Send exactly one JSON text message before sending any audio. This example uses a model, an installed English voice, and mono float32 PCM captured at 48 kHz. Replace the placeholders and set the sample rate to the actual capture rate: ```json { "agent": { "model": "" }, "audio": { "encoding": "pcm_float32", "sample_rate_hz": 48000 }, "tts": { "engine": "", "voice": "", "lang": "en" }, "stt": { "inference_config": { "language": "en" }, "pipeline_config": { "partial_enabled": true, "partial_interval_s": 0.5 } }, "chat_id": "", "store": true } ``` To use an Agent, replace `agent` with `{ "agent_id": "" }`. Supply exactly one of `model` and `agent_id`. An Agent contributes only its model and system prompt in this endpoint, not its MCP tools, Skills, file retrieval, or web search. The required top-level fields are `agent`, `audio`, and `tts`. Omit `chat_id` and `store` for an ephemeral session; `store` defaults to `false`. When `store` is `true`, `chat_id` is required. Use a new UUID for a new conversation or an existing caller-owned chat ID to continue its history. Providing `chat_id` with `store: false` seeds history without persisting new turns. The deployment's [live API reference](/docs/reference) exposes the full `VoiceConfigFrame` schema, including supported audio formats and turn settings. Its `GET /v1/voice` entry documents the WebSocket; it is not an HTTP endpoint that starts an audio conversation. ## 3. Stream and render [#3-stream-and-render] After the configuration, send microphone audio as binary messages in the declared format. Do not send further text messages or a media container while claiming raw PCM. Read both kinds of server frames: * **Binary:** raw signed PCM int16, 24 kHz, mono, in whole 40 ms chunks. Queue it in a playback buffer configured for that format; it is not a WAV file. * **Text:** JSON events. Dispatch on `type` and ignore unknown event types for forward compatibility. | Event | Client behavior | | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | `speech_started`, `speech_stopped` | Show detected microphone activity. | | `transcript` | Use `text`, `is_partial`, and `promoted`. Partial text can change; a final is part of the user's turn. | | `user_turn_started`, `user_turn_stopped` | Group one or more final transcripts into a user turn. A stopped turn with usable text triggers the answer. | | `tts_started`, `tts_stopped` | Mark the reply's audio boundaries, even if that reply produced no audio. | | `agent_text` | Append `text` exactly as received, including Markdown. Each piece precedes the audio that speaks it. | | `interruption` | Stop playback and clear queued audio immediately. Keep reply text already displayed. | | `error` | Inspect `code` and show a recoverable reply failure or a session failure as appropriate. | For example, `{"type":"agent_text","text":"Good morning."}` appends text; it is not another synthesis request. Keep sending microphone audio while playing the answer to support interruption. On end or navigation, close the socket and release microphone tracks and playback resources. ## 4. Handle incomplete turns and failures [#4-handle-incomplete-turns-and-failures] The default wait for a final transcript after speech stops is three seconds. If it arrives too late, the service promotes the last partial to a final with `promoted: true`. That text may be incomplete and can still trigger an answer. Display a notice and let the user correct it. With no usable transcript, there is no answer. | Failure | Session behavior and recovery | | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Authentication failure during connection | HTTP 401. Correct the credential or tenant selection before reconnecting. | | Invalid configuration before the session starts | Standard API `error` envelope, with no top-level `type`; field details identify the problem. The socket closes with 1008 or 1011. Correct the frame and open a new connection. | | `transcription_failed` | A partial transcription failed; later partials can continue. Final-transcript failure is handled by the turn timeout. | | `llm_failed` or `synthesis_failed` | That reply failed. The session can continue; let the user ask again. | | `stt_provider_unavailable`, `tts_provider_unavailable`, `llm_provider_unavailable` | The session ends. Correct service/model availability before restarting. | | `invalid_message` | Text was sent after configuration. Correct the client before restarting. | | `unsupported_audio_format` | Correct the declared encoding and binary framing. Malformed input can end the session. | A fatal in-session `error` can be followed by close code **1000**, so a normal-looking close code alone does not prove success. A minute with neither side speaking also closes normally, without an error. Connection and processing failures are not automatically retried by the service. ## 5. Verify conversation continuity [#5-verify-conversation-continuity] With `store: true`, finished user turns and delivered `agent_text` are saved incrementally. Read the conversation through `GET /ui/chats/{id}` or reopen it in the Portal. An interrupted reply preserves only its already-delivered text, and a dropped connection preserves completed turns rather than unfinished speech. Test a short exchange, interrupt an answer, end the session, and reopen the same chat. Check the transcript, retained reply portions, microphone cleanup, and behavior when permission or connectivity is lost. ## Related pages [#related-pages] * [Voicebot user guide](/docs/ai/voicebot) * [Use models through the API](/docs/develop/use-local-models-via-api) * [Speaches](/docs/ai/components/speaches)