KubeAI

Component Category

Inference / model serving orchestration

Component Description

KubeAI is a Kubernetes-native inference operator for deploying and scaling AI models in production.

Why It Is Used

In BullSequana AI Runtime, KubeAI provides the operational layer that helps run model-serving workloads on Kubernetes with more predictable scaling, routing, and platform integration.

Learn More

• KubeAI documentation
• substratusai/kubeai on GitHub

Deployment notes

KubeAI deploys into the kubeai namespace at sync wave 3 in the coreai tier. The controller manages model server pods dynamically based on Model custom resources. Model servers include vLLM (with NVIDIA, AMD, CPU, and GH200 variants), OLlama, and FasterWhisper. Model weights are cached on shared RWX filesystem volumes, while Rook Ceph RGW provides the S3-compatible object storage endpoint for model artifacts. GPU resource profiles define per-accelerator resource requests for 12 GPU types.

Interacts With

• CSI S3, which mounts Rook Ceph S3 buckets as local volumes for model weight caching.
• Rook Ceph, which provides the S3-compatible storage backend (RGW) for model artifacts.
• vLLM and FasterWhisper, which are part of the model-serving runtime KubeAI orchestrates.
• KServe, which is planned to fully replace KubeAI as the inference orchestration layer.
• Model Installer, which targets the KubeAI service endpoint to register and manage models.
• Gateway API, which routes inference traffic to KubeAI endpoints.