# Enable GPU for AI components (/docs/deployment/playbooks/enable-gpu)



Two CoreAI components ship with GPU acceleration that is **off by default** (CPU mode): speech-to-text (Speaches/Whisper) in `llm-backend`, and `docling-server`. This playbook explains how to enable GPU for each one.

GPU is controlled by a per-component boolean in `variables.toml`. Flipping it through the standard environment-variable override switches the container image and adds `nvidia.com/gpu` resource requests, `runtimeClassName: nvidia`, and GPU node affinity and tolerations.

You set these overrides in `platform.env` (shared, committed) or `local.env` (personal, gitignored). See [Configuration model](/docs/deployment/configuration-model) for how the two files are layered.

Prerequisites [#prerequisites]

The cluster must be able to schedule GPU pods. GPU nodes must provide:

* node label `workload=gpu`
* node taint `nvidia.com/gpu=present`
* the `nvidia` RuntimeClass, with NVIDIA drivers and the NVIDIA device plugin installed at cluster level

For the full GPU footprint and sizing guidance, see [Minimum requirements](/docs/deployment/minimum-requirements).

By default the platform uses **preferred (soft)** GPU affinity, so GPU pods prefer GPU nodes but can still schedule elsewhere. To make GPU placement a **hard requirement**, set:

```dotenv
# Pin GPU pods to gpu-labeled nodes ([common].enforce_affinity, default false)
COMMON_ENFORCE_AFFINITY="true"
```

Enable GPU [#enable-gpu]

Add the toggles you need to `platform.env` (or `local.env`):

```dotenv
# Enable GPU for speech-to-text (Speaches/Whisper) in llm-backend
LLM_BACKEND_SPEACHES_GPU="true"

# Enable GPU for docling-server
DOCLING_SERVER_COMPONENT_GPU="true"
```

Speech-to-text (Speaches/Whisper) [#speech-to-text-speacheswhisper]

|                   |                                      |
| ----------------- | ------------------------------------ |
| Component         | `llm-backend`                        |
| Toggle            | `[speaches].gpu` in `variables.toml` |
| Override variable | `LLM_BACKEND_SPEACHES_GPU`           |

When set to `true`:

* image switches to `speaches-ai/speaches:latest-cuda`
* sets `WHISPER__INFERENCE_DEVICE=cuda` and `WHISPER__COMPUTE_TYPE=float16`
* requests `nvidia.com/gpu: "1"`
* selects the GPU Whisper model (`deepdml/whisper-large-v3-turbo`) instead of the CPU model (`Systran/faster-whisper-base`), and propagates it to `PIPECAT_WHISPER_MODEL`

Docling [#docling]

|                   |                                               |
| ----------------- | --------------------------------------------- |
| Component         | `docling-server`                              |
| Toggle            | `gpu` under `[component]` in `variables.toml` |
| Override variable | `DOCLING_SERVER_COMPONENT_GPU`                |

When set to `true`:

* image switches from `docling-serve-cpu` to `docling-serve-cu128`
* adds `nvidia.com/gpu: "1"` to both requests and limits

Apply the change [#apply-the-change]

After editing `platform.env`, re-render and push manifests so ArgoCD picks up the CUDA images and GPU resource requests:

```bash
uv run -m src.cli.main apply all -r
uv run -m src.cli.main push-manifests "Enable GPU for Speaches and Docling"
```

See [Platform CLI](/docs/deployment/platform-cli) for details on these commands.

Environment-variable override convention [#environment-variable-override-convention]

The override variable names above follow the platform's standard convention, so any future GPU-capable component is self-explanatory:

```
COMPONENT_NAME + TOML_TABLE + KEY   (uppercased, hyphens replaced with underscores)
```

* The component name is uppercased with `-` replaced by `_` (`llm-backend` → `LLM_BACKEND`, `docling-server` → `DOCLING_SERVER`).
* The TOML table and key are joined with `_`.
* Keys that live under the `[component]` table include the `COMPONENT_` segment — that is why Docling uses `DOCLING_SERVER_COMPONENT_GPU` while Speaches (under `[speaches]`) uses `LLM_BACKEND_SPEACHES_GPU`.

See the [variable resolution chain](/docs/deployment/configuration-model#4-variable-resolution) for the complete rules.

Related pages [#related-pages]

* [Configuration model](/docs/deployment/configuration-model)
* [Minimum requirements](/docs/deployment/minimum-requirements)
* [Platform CLI](/docs/deployment/platform-cli)
* [Docling component](/docs/coreai/components/docling)