Inference
Deploy, change, remove and publish a tenant's own models.
Inference permissions let a customer organization deploy and operate its own models. They are evaluated in the selected tenant and apply to the models in that tenant's own namespace. Inference Deployer and Inference Operator grant the five deployment permissions. Tenant administration grants only inference.endpoint.publish, so a tenant administrator who deploys models also needs one of these roles or a custom role.
The Global organization's models are platform services and use platform permissions instead: platform.global_service.manage to deploy, change and remove them, and platform.global.read to inspect them. Inference permissions have no effect in the Global organization. See Platform.
A permission is the operation gate, not access to a model. Whoever deploys a model owns it and calls it without a rule while a member of the organization; everyone else needs an allow rule, as Model access describes. Every member sees the organization's models without one. The deployment request's callers sets the first rules; without them, only the owner can call the model until they add one. The tenant must have inference enabled by a platform administrator, and its deployments stay within the deployment policy and GPU allocation a platform administrator set. See Organizations and namespaces for the Model Installer behavior.
inference.model.deploy does not imply the list permission. A user with only the deploy permission can deploy a model but cannot list the tenant's deployments afterwards.
Permissions
This family contains 6 permissions, 4 selectable in custom roles. “Custom role” means the identifier can be selected when defining a custom role.
inference.deployment.delete
Remove a model deployment of the selected tenant, with its routes, owner and allow rules.
Remove a model deployment of the selected tenant, with its routes, owner and allow rules.
| Property | Value |
|---|---|
| Scope | Selected tenant |
| Granted by | Inference Deployer, Inference Operator |
| Custom role | Yes |
| Also implies | inference.deployment.list, inference.deployment.read |
inference.deployment.list
List the selected tenant's model deployments and inspect their status, pods and logs.
List the selected tenant's model deployments and inspect their status, pods, pod logs and the deployments of a registered model.
| Property | Value |
|---|---|
| Scope | Selected tenant |
| Granted by | Inference Deployer, Inference Operator |
| Custom role | Yes |
| Also implies | inference.deployment.read |
inference.deployment.read
Read one model deployment of the selected tenant.
Read one model deployment of the selected tenant. It follows from the list permission.
| Property | Value |
|---|---|
| Scope | Selected tenant |
| Granted by | Inference Deployer, Inference Operator |
| Custom role | No. Computed from inference.deployment.list; select that permission instead. |
inference.deployment.update
Change a model deployment of the selected tenant, such as its LoRA adapters.
Change a model deployment of the selected tenant, such as its LoRA adapters. An adapter update restarts the deployment and gives every adapter the base model's owner and allow rules.
| Property | Value |
|---|---|
| Scope | Selected tenant |
| Granted by | Inference Deployer, Inference Operator |
| Custom role | Yes |
| Also implies | inference.deployment.list, inference.deployment.read |
inference.endpoint.publish
Publish a language model the selected tenant deployed to every tenant, or unpublish it.
Publish a language model the selected tenant deployed to every tenant, or unpublish it. In the Portal it is Publish to all tenants in the model's Access. A platform administrator must first let the tenant publish; otherwise the request returns 409 serving_endpoint_publishing_disabled. Predictive models cannot be published (409 serving_endpoint_not_publishable), and neither can Global models or models whose tenants a platform administrator chose (409 serving_endpoint_audience_set_by_platform). Each receiving tenant still decides through its own allow rules which of its members may use a published model. The publishing tenant keeps running the deployment and its capacity counts for that tenant.
| Property | Value |
|---|---|
| Scope | Selected tenant |
| Granted by | Tenant administrator |
| Custom role | No. Publishing a tenant's models remains tenant-administrator authority. |
inference.model.deploy
Deploy and import models into the selected tenant, and use its recipe catalogue.
Deploy a model into the selected tenant's namespace, import a model into its private MLflow, follow its downloads, and read and refresh its vLLM recipe catalogue and Hugging Face model cards. The deployer becomes the model's owner. Deploying a registered version also needs read access on that model, which every member of the organization has.
| Property | Value |
|---|---|
| Scope | Selected tenant |
| Granted by | Inference Deployer, Inference Operator |
| Custom role | Yes |