# KServe (/docs/ai/components/kserve)



## Component Category [#component-category]

AI serving / model inference

## Component Description [#component-description]

KServe is the Kubernetes-native model-serving framework used by BullSequana AI 1.3.0. The platform installs KServe 0.20.0 and its LLM inference APIs, then uses `LLMInferenceService` resources for backend-managed model deployments. Predictive serving runtimes remain available through KServe `InferenceService` resources.

## Why It Is Used [#why-it-is-used]

In BullSequana AI, KServe provides declarative model lifecycle, autoscaling, runtime configuration, workload status, and an internal inference data path.

## Learn More [#learn-more]

* [KServe documentation](https://kserve.github.io/website/)
* [kserve/kserve on GitHub](https://github.com/kserve/kserve)

## Deployment notes [#deployment-notes]

KServe resources run in the `kserve` namespace and are owned by the separate `inference` Argo CD parent application. The layer includes KServe CRDs and resources, LLM inference CRDs, KAI Scheduler, Envoy AI Gateway, the Gateway API Inference Extension, runtime profiles, and a private internal gateway.

The shipped vLLM profiles include CPU and NVIDIA GPU configurations for 1, 2, 4, and 8 GPUs. Optional profiles stream Safetensors from the platform object store or from a dedicated `[model_storage]` S3 endpoint.

Administrators curate the deployment choices exposed by the guided model setup through [Portal Model Presets](/docs/use/portal#model-presets).

Model workloads have no public route. Requests pass through the BSQAI API and LiteLLM before reaching the internal KServe gateway.

## Interacts With [#interacts-with]

* `BSQAI API`, which creates deployments and exposes workload health, pod, and log operations.
* `LiteLLM`, which routes authenticated model requests to the private inference path.
* `vLLM`, which serves LLM workloads through platform runtime profiles.
* `KAI Scheduler`, which places and quotas GPU workloads.
* `Envoy AI Gateway`, which provides the internal model-serving data path.
* `Rook-Ceph` or external S3-compatible storage, which supplies model artifacts.
* `Grafana Alloy` and `Grafana`, which collect and display inference telemetry.