vLLM

Component Category

Inference / LLM serving engine

Component Description

vLLM is a high-throughput and memory-efficient inference engine designed for serving large language models.

Why It Is Used

In BullSequana AI Runtime, vLLM powers efficient LLM inference with strong performance characteristics for production workloads, especially where throughput and GPU utilization matter.

Learn More

• vLLM documentation
• vllm-project/vllm on GitHub

Deployment notes

vLLM is not deployed as a standalone component. KubeAI manages vLLM pods dynamically based on Model custom resources, using platform-registry container images with variants for NVIDIA, GH200, AMD, and CPU.

Interacts With

• KubeAI, which uses vLLM as one of its inference backends for model serving.
• Model Installer and other serving workflows, which deploy or operate models on top of vLLM-backed runtimes.