Speaches

Speech inference service used by the BSQAI API for transcription and speech generation.

Agentic Friendly

Component Category

AI inference / speech processing

Component Description

Speaches is an OpenAI-compatible speech server for streaming transcription, translation, and speech generation. BullSequana AI deploys it as a backend-owned service that provides the internal speech endpoint used by the BSQAI API.

Why It Is Used

In BullSequana AI, Speaches provides one runtime for batch and real-time speech-to-text and text-to-speech workflows. FasterWhisper handles transcription, while pre-cached Piper and Kokoro voices support speech generation without requiring runtime model downloads.

User and developer workflows

Use Voicebot for live spoken conversations in the Portal, or build a voice client through the authenticated BSQAI API. Portal dictation and read-aloud are separate speech controls; applications do not connect directly to the internal Speaches service.

Learn More

Deployment notes

The Speaches service deploys inside the BSQAI API release as a separate single-replica Kubernetes Deployment and internal ClusterIP service. BullSequana AI deploys Speaches 0.8.3. The default CPU configuration uses int8 FasterWhisper inference. An environment override selects the CUDA image, float16 inference, and one NVIDIA GPU.

The CPU profile caches the transcription model Systran/faster-whisper-base, and the GPU profile deepdml/faster-whisper-large-v3-turbo-ct2. Both cache the Kokoro voices and Piper voices for English, German, French, Czech, Slovak, and Greek. The [speaches] table of the llm-backend component sets the lists in models_cpu, models_gpu, and extra_models.

The deployment uses a Recreate rollout strategy because its 10Gi model-cache volume is ReadWriteOnce. A pre-deployment Job fills that cache from a version-pinned model image by default, models_cache_mode = "copy", which keeps speech startup reproducible in restricted-network environments. With models_cache_mode = "download", Speaches downloads the models from Hugging Face at deploy time instead, which needs network access. The cache contains the active profile's Whisper models and the configured TTS voices.

The BSQAI API reads the installed models from Speaches on each request instead of at startup. The API therefore starts without Speaches, speech requests work once Speaches is reachable, and clients choose among the installed models.

Interacts With

  • BSQAI API, which exposes authenticated batch and real-time STT and TTS endpoints.
  • FasterWhisper, which performs speech transcription inside Speaches.
  • AI Web Portal, which uses speech services for microphone input and generated audio.
  • NVIDIA GPU Operator, which provides GPU scheduling support when CUDA mode is enabled.

On this page