# Share GPUs with time slicing and fractioning (/docs/deployment/playbooks/gpu-sharing)



The platform can share one physical GPU between several model workloads in two ways:

* **Time slicing**: the NVIDIA device plugin advertises one GPU as several `nvidia.com/gpu.shared` replicas. All replicas see the full GPU memory, and the GPU switches between them in time. There is no memory isolation.
* **Fractioning**: KAI Scheduler gives a pod a fraction of a GPU (for example `0.25`) through the `gpu-fraction` annotation. The KAI `gpu-fractioning` components enforce a memory limit for each fraction on the node. This limit is not a security or fault-isolation boundary. Fractioning requires NVIDIA driver major version 615 or later on the GPU nodes.

You can enable both modes on the same cluster. Each GPU node pool uses one mode, which you select with a node label. Whole-GPU profiles keep working on GPU nodes without a sharing mode and on fractioning nodes. A time-sliced node offers only shared replicas, so whole-GPU pods do not run there.

## How the modes are separated [#how-the-modes-are-separated]

| Mechanism                                    | Time slicing                                        | Fractioning                                           |
| -------------------------------------------- | --------------------------------------------------- | ----------------------------------------------------- |
| Node label `nvidia.com/device-plugin.config` | `timeslicing`                                       | `fractioning`                                         |
| Device-plugin config on the node             | GPU split into `replicas` × `nvidia.com/gpu.shared` | GPU stays whole (`nvidia.com/gpu`)                    |
| Who splits the GPU                           | NVIDIA device plugin                                | KAI Scheduler                                         |
| GPU memory                                   | No limit; vLLM limits itself to its share           | Limit per fraction, enforced by KAI `gpu-fractioning` |
| Isolation boundary                           | None                                                | None; pods share one physical GPU                     |
| Minimum NVIDIA driver                        | Any driver supported by the GPU Operator            | Major version **615** or later                        |

A time-sliced node advertises `nvidia.com/gpu: 0`, so KAI cannot place a fractional pod on it. Each shared runtime profile also has a required node affinity on the matching label value.

<Callout type="warning" title="Shared GPUs are not an isolation boundary">
  Neither mode isolates workloads from each other. Workloads on the same GPU share one physical device and can affect each other's stability and performance. With fractioning, a fatal GPU fault in one workload also stops the other workloads on that GPU. Do not put workloads that need strict isolation on a shared pool. Use whole-GPU profiles on dedicated GPUs, or validated MIG partitions, for them.
</Callout>

## Prerequisites [#prerequisites]

* The platform GPU Operator is installed (`bsqai helm-apply gpu-operator`). See [Enable GPU for AI components](/docs/deployment/playbooks/enable-gpu).
* `[dra] enabled = false` in the `gpu-operator` component. Time slicing needs the device plugin, and DRA mode removes it. The render fails if you enable both.
* For fractioning: GPU nodes with NVIDIA driver major version 615 or later. If the provider image has an older driver, let the GPU Operator install the driver (see [Driver ownership](#driver-ownership)).
* For fractioning: containerd 2.0 or later with NRI enabled on the GPU nodes, and Kubernetes 1.28 or later.
* The KAI `gpu-fractioning` images (`gpu-fractioning/operator`, `fractiond`, `metricsd`, `mpsd`) are mirrored into the platform registry.

## Driver ownership [#driver-ownership]

The `gpu-operator` `[driver] managed` setting applies to the whole cluster. When it is `true`, the GPU Operator installs the driver **and** the NVIDIA container toolkit on the GPU nodes. On a node that already has a host driver, the GPU Operator detects that driver and does not install its own, but it still deploys the container toolkit. Set it to `true` only when the cluster has at least one GPU node pool without a host driver.

<Callout type="warning" title="Host-driver nodes must opt out of the toolkit">
  When `managed = true`, nodes that already have a host driver and container runtime (for example AKS node pools created with the default GPU driver) must opt out with the node label `nvidia.com/gpu.deploy.container-toolkit=false`. Without this label, the toolkit changes the host containerd configuration and containerd does not start. On AKS, node auto-repair then reboots the node in a loop.
</Callout>

| Node pool                                                             | Driver source  | `[driver] managed` | Required node label                             |
| --------------------------------------------------------------------- | -------------- | ------------------ | ----------------------------------------------- |
| All GPU pools have a host driver (≥ 615 for fractioning)              | Provider image | `false`            | none                                            |
| Pool without a host driver (on AKS, created with `--gpu-driver none`) | GPU Operator   | `true`             | none                                            |
| Pool with a host driver, in a cluster with `managed = true`           | Provider image | `true`             | `nvidia.com/gpu.deploy.container-toolkit=false` |

To keep one driver source per cluster, create all GPU node pools without a host driver when `managed = true`.

## Configure the platform [#configure-the-platform]

Both modes are off by default. Turn them on in `platform.env`, as for every site setting; do not edit `platform.toml`:

```dotenv
GPU_SHARING_TIME_SLICING_ENABLED=true
# Integer from 2 to 16
GPU_SHARING_TIME_SLICING_REPLICAS=4

GPU_SHARING_FRACTIONING_ENABLED=true
# JSON list of unique values, each at least 0.06 and less than 1; it replaces the whole list
GPU_SHARING_FRACTIONING_FRACTIONS='[0.25,0.5]'
```

If a fractioning pool needs the operator-installed driver, also set:

```dotenv
GPU_OPERATOR_DRIVER_MANAGED=true
```

| Override                                                                | Default                | Key                                                                 | Effect                                                                                                                       |
| ----------------------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `GPU_SHARING_TIME_SLICING_ENABLED`, `GPU_SHARING_TIME_SLICING_REPLICAS` | `false`, `4`           | `[gpu_sharing.time_slicing] enabled`, `replicas` in `platform.toml` | Adds the `timeslicing` device-plugin config and the `-timesliced` runtime profiles                                           |
| `GPU_SHARING_FRACTIONING_ENABLED`, `GPU_SHARING_FRACTIONING_FRACTIONS`  | `false`, `[0.25, 0.5]` | `[gpu_sharing.fractioning] enabled`, `fractions` in `platform.toml` | Sets KAI `gpuSharingMode: NvFractions`, deploys `gpu-fractioning`, adds one `-fraction-<value>` runtime profile per fraction |
| `GPU_OPERATOR_DRIVER_MANAGED`                                           | `false`                | `[driver] managed` in `gpu-operator/variables.toml`                 | GPU Operator installs the driver and container toolkit                                                                       |
| `GPU_OPERATOR_IMAGES_DRIVER`                                            | `615.71.09`            | `[images] driver` in `gpu-operator/variables.toml`                  | Driver version that the GPU Operator installs (must be ≥ 615 for fractioning)                                                |
| `GPU_OPERATOR_DRA_ENABLED`                                              | `false`                | `[dra] enabled` in `gpu-operator/variables.toml`                    | Must stay `false`                                                                                                            |

The render stops when a value is out of range or when time slicing is on with `GPU_OPERATOR_DRA_ENABLED=true`.

## Prepare the node pools [#prepare-the-node-pools]

Set the labels and taints on the node pool, not on single nodes, so that they persist when nodes are replaced, reimaged, or scaled out. The platform's GPU workloads tolerate any value of the `nvidia.com/gpu` taint; `present` matches the other GPU pages.

| Label or taint                            | Time-slicing pool                                                               | Fractioning pool                                                                |
| ----------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| `workload`                                | `gpu`                                                                           | `gpu`                                                                           |
| `nvidia.com/device-plugin.config`         | `timeslicing`                                                                   | `fractioning`                                                                   |
| `nvidia.com/gpu.deploy.container-toolkit` | `false` on a host-driver pool when `[driver] managed = true`; otherwise not set | `false` on a host-driver pool when `[driver] managed = true`; otherwise not set |
| Taint                                     | `nvidia.com/gpu=present:NoSchedule`                                             | `nvidia.com/gpu=present:NoSchedule`                                             |

Example for AKS:

```bash
# Time slicing on the AKS-supplied driver
az aks nodepool add --resource-group <rg> --cluster-name <cluster> --name gputs \
  --node-vm-size <gpu-sku> \
  --labels workload=gpu nvidia.com/device-plugin.config=timeslicing nvidia.com/gpu.deploy.container-toolkit=false \
  --node-taints nvidia.com/gpu=present:NoSchedule

# Fractioning on the operator-installed driver
az aks nodepool add --resource-group <rg> --cluster-name <cluster> --name gpufrac \
  --node-vm-size <gpu-sku> --gpu-driver none \
  --labels workload=gpu nvidia.com/device-plugin.config=fractioning \
  --node-taints nvidia.com/gpu=present:NoSchedule
```

## Apply the change [#apply-the-change]

```bash
bsqai apply all -r
bsqai push-manifests "Enable GPU sharing"
bsqai helm-apply gpu-operator
```

The GPU Operator is not deployed through GitOps, so you must run `bsqai helm-apply gpu-operator` again after you change `[driver] managed`, `[gpu_sharing.time_slicing]`, or the driver image.

## Select a profile for a model [#select-a-profile-for-a-model]

Each mode adds `LLMInferenceServiceConfig` profiles next to the whole-GPU profiles. Every GPU profile, shared or whole, uses `schedulerName: kai-scheduler` and the RuntimeClass named by `GPU_OPERATOR_RUNTIME_CLASS`, `nvidia-container-runtime` by default. See [NVIDIA RuntimeClass](/docs/deployment/playbooks/enable-gpu#nvidia-runtimeclass).

|                                      | Time-sliced profile                           | Fractional profile                                                         |
| ------------------------------------ | --------------------------------------------- | -------------------------------------------------------------------------- |
| Profile name                         | `bsqai-config-llm-gpu-timesliced`             | `bsqai-config-llm-gpu-fraction-025`, `bsqai-config-llm-gpu-fraction-05`, … |
| GPU request                          | `nvidia.com/gpu.shared: 1`                    | none; annotation `gpu-fraction: "<value>"`                                 |
| vLLM GPU memory utilization          | `0.9 / replicas`                              | `0.80`, as on a whole GPU; KAI limits the pod to its fraction's memory     |
| Node affinity                        | `nvidia.com/device-plugin.config=timeslicing` | `nvidia.com/device-plugin.config=fractioning`                              |
| Label `bsqai.io/gpu-allocation-mode` | `timesliced`                                  | `fractional`                                                               |

The S3-streaming profile family (`bsqai-config-llm-gpu-s3-streaming`) gets the same variants.

## Allow the shared profiles for tenants [#allow-the-shared-profiles-for-tenants]

A tenant deploys only on the resource profiles its [deployment policy](/docs/administration#set-what-a-tenant-may-deploy) allows. What a new policy allows depends on the tenant:

| Tenant                                       | Resource profiles allowed when the tenant is first set up                                                                                 |
| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Global Organization, on a fresh installation | The CPU profile `bsqai-config-llm-cpu`, the whole-GPU profiles, and the time-sliced and fractional profiles that the installation renders |
| Every other tenant                           | Only the whole-GPU profiles `bsqai-config-llm-gpu`, `-gpu-2`, `-gpu-4` and `-gpu-8`, so its models never share a GPU                      |

No policy allows the S3-streaming profiles, including their shared variants, or the GPU profiles that platform administrators define at run time, until a platform administrator adds them. Platform administrators define those profiles in [Define GPU profiles](/docs/administration/tenant-settings#define-gpu-profiles). The base profile `bsqai-config-llm-gpu-shared` is never offered for deployment.

To let a tenant use shared GPUs, a platform administrator opens **Administration → Tenants**, selects the tenant, opens **Deployment policy**, and adds the shared profiles to its resource profiles. The Global Organization needs the same step when GPU sharing is enabled after installation, or after an upgrade from an earlier release: an existing policy keeps the profiles it already has.

<Callout type="warning" title="Shared profiles carry no tenant affinity">
  One shared GPU can hold models of different tenants. Keep a tenant that must not share hardware on whole-GPU profiles.
</Callout>

### Set the tenant's GPU allocation [#set-the-tenants-gpu-allocation]

A tenant's models are also bound by its GPU allocation. A platform administrator sets it in the same tenant, on the **Quotas** tab, in the [GPU allocation](/docs/administration#gpu-allocation) card:

| Field               | Effect                                                                                               |
| ------------------- | ---------------------------------------------------------------------------------------------------- |
| **Guaranteed GPUs** | GPUs the tenant's models are guaranteed across all its deployments. Empty or `0` guarantees nothing. |
| **GPU limit**       | GPUs the tenant's models may use together. Empty means no cap. It must be at least the guarantee.    |

Both accept fractions of one GPU, such as `0.25` or `0.5`, with at most three decimals. A shared profile counts for its share of the card: `1 / replicas` for a time-sliced profile, and the fraction for a fractional profile. The allocation applies once inference is enabled for the tenant.

A guarantee above `0` needs GPUs that the platform reserves for tenants with `KAI_SCHEDULER_EXTRAS_TENANT_GPU_QUOTA`; see [Tenant GPU guarantees](/docs/deployment/configuration-model#tenant-gpu-guarantees). For how the tenant's GPU queues enforce the allocation, see [Tenant GPU scheduling](/docs/foundation/components/tenant-operator#tenant-gpu-scheduling). A pod on a shared GPU is metered for its share of the card; see [Manage monthly BCU allowances and caps](/docs/administration/usage-budgets#what-each-balance-counts).

To check where shared models run and which tenants hold each GPU, a platform administrator opens **Administration → Platform → GPU usage**; see [Review GPU usage](/docs/administration/gpu-usage).

## Verify [#verify]

```bash
# Node labels per pool
kubectl get nodes -l workload=gpu \
  -L nvidia.com/device-plugin.config,nvidia.com/gpu.deploy.driver,nvidia.com/gpu.deploy.container-toolkit

# Time-slicing nodes advertise shared replicas
kubectl get nodes -l nvidia.com/device-plugin.config=timeslicing \
  -o custom-columns=NAME:.metadata.name,SHARED:.status.allocatable.nvidia\\.com/gpu\\.shared

# Fractioning nodes report a supported driver
kubectl get nodes -l nvidia.com/device-plugin.config=fractioning -L nvidia.com/cuda.driver-version.major
```

## Troubleshooting [#troubleshooting]

| Symptom                                                                                                       | Cause                                                           | Action                                                                                                                                                                                   |
| ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| GPU node goes `NotReady` about one minute after each boot, with `ContainerRuntimeIsDown` and repeated reboots | Container toolkit deployed on a node that has a host driver     | Set `nvidia.com/gpu.deploy.container-toolkit=false` on the node pool, run `bsqai helm-apply gpu-operator` again, and reimage the node if containerd still fails                          |
| Node condition `gpu-fractioning.nvidia.com/Ready=False`, reason `GPUDriverVersionMissing`                     | Driver older than 615 on a node that runs KAI `gpu-fractioning` | Use a driver ≥ 615 on fractioning pools: a host driver, or the operator-installed driver when the image has an older one; on time-slicing pools this condition does not block scheduling |
| `fractiond` or `mpsd` pods in `ImagePullBackOff`                                                              | `gpu-fractioning` images not in the platform registry           | Mirror the images for the KAI Scheduler chart version in use                                                                                                                             |
| Time-sliced model runs out of GPU memory                                                                      | Replicas do not have memory isolation                           | Lower `replicas`, or use a fractional profile                                                                                                                                            |
| A deployment on a shared profile fails with `deployment_not_allowed`                                          | The tenant's deployment policy does not allow the profile       | Add the profile to the tenant's deployment policy (see [Allow the shared profiles for tenants](#allow-the-shared-profiles-for-tenants))                                                  |

## Related pages [#related-pages]

* [Enable GPU for AI components](/docs/deployment/playbooks/enable-gpu)
* [Set what a tenant may deploy](/docs/administration#set-what-a-tenant-may-deploy)
* [Inference](/docs/foundation/inference)
* [Node taints and affinities](/docs/foundation/components/node-taints-and-affinities)
* [Platform CLI](/docs/deployment/platform-cli)
