Multi-Tenancy and Tiered RBAC in Kubeflow

How Kubeflow profile namespaces get differentiated, least-privilege access instead of one flat role.

Kubeflow's built-in profile system gives every user or team their own namespace (a "profile"), but the upstream profiles-controller grants every member of a profile the same role: a single default-editor ServiceAccount bound to kubeflow-edit. That means a profile's owner and its collaborators end up with identical, overly broad permissions, with no way to grant a read-only or admin-only tier.

BullSequana AI replaces that flat model with tiered, per-profile RBAC, generated automatically for every Kubeflow profile namespace.

The Tier Model

Every Kubeflow profile namespace gets four ServiceAccounts instead of one:

ServiceAccount	Bound Role	Grants
`default-viewer`	`kubeflow-view`	Read-only: browse notebooks, pipelines, and logs
`default-editor`	`kubeflow-edit`	Create notebooks, run pipelines; no raw pod access
`default-admin`	`kubeflow-admin`	Full namespace access; intended for the profile owner only
`pipeline-runner`	—	Dedicated Argo Workflows executor identity, kept separate from every user-facing tier

Keeping pipeline-runner separate from the human-facing tiers means pipeline execution permissions never inherit from, or get confused with, whatever access level a human collaborator happens to have.

How Generation Works

The ServiceAccounts and RoleBindings are not part of the Kubeflow manifests themselves — they are generated per-namespace by Kyverno ClusterPolicy resources using the classic kyverno.io/v1 generate engine:

A policy watches for Namespace objects labeled app.kubernetes.io/part-of: kubeflow-profile (the label the Kubeflow profiles-controller applies to every profile namespace it creates).
On match, it generates the four ServiceAccounts into that namespace.
A second policy generates the corresponding RoleBindings, binding each ServiceAccount to its ClusterRole.

Both policies set generateExisting: true, so they also backfill RBAC into profile namespaces that already existed before the policies were deployed, not just newly created ones.

Why the classic generate engine, not GeneratingPolicy

Kyverno's newer policies.kyverno.io/v1beta1 GeneratingPolicy (gpol) engine was evaluated and abandoned for this use case: in Kyverno v1.16.x, its WatchManager deletes downstream resources on every reconcile cycle when synchronize: true is set, which would repeatedly wipe out the generated ServiceAccounts and RoleBindings. The classic kyverno.io/v1 ClusterPolicy generate engine does not have this bug and is used consistently for every generate-style policy in the cluster.

Escalation prevention and the background-controller's own permissions

Kubernetes' built-in RBAC escalation prevention blocks creating a RoleBinding that references a ClusterRole with permissions the creator does not already hold. The escalate verb exempts Role/ClusterRole writes from this check, but not RoleBinding writes. As a result, the kyverno-background-controller ServiceAccount must itself be bound to kubeflow-view, kubeflow-edit, and kubeflow-admin before it can generate RoleBindings that reference those roles for other subjects.

Sync Ordering

The RBAC resources deploy across three sync waves so dependencies are always ready before they're needed:

Wave 31 — the ClusterRole/ClusterRoleBinding that let Kyverno's controllers manage ServiceAccounts and RoleBindings, plus the bindings that grant kyverno-background-controller the three Kubeflow roles it needs to generate RoleBindings safely.
Wave 32 — the two generate ClusterPolicy resources (ServiceAccounts, then RoleBindings).
Wave 60+ — the Kubeflow notebook and pipeline controllers that expect this RBAC to already exist in every profile namespace.

Interacts With

Kubeflow, whose profile namespaces are the target of every generated ServiceAccount and RoleBinding described here.
Kyverno, which performs the actual generation via ClusterPolicy resources.
Argo Workflows (via the pipeline-runner ServiceAccount), which executes Kubeflow Pipelines runs under a permission set independent of any human user's tier.