Kyverno

Kubernetes policy engine that runs the Kubeflow and platform policies for validation, mutation, and resource generation.

Agentic Friendly

Component Category

Platform governance / policy enforcement

Component Description

Kyverno is a Kubernetes-native policy engine that validates, mutates, and generates resources using declarative policies defined as ClusterPolicy and Policy custom resources. It operates as an admission controller, intercepting API requests before resources are persisted.

Why It Is Used

In BullSequana AI, Kyverno runs the Kubeflow profile RBAC and pipeline policies that Kubeflow ships as ClusterPolicy and GeneratingPolicy resources, which only work while the Kyverno controllers run. It also runs the platform's own policies for Kubeflow profile namespaces and storage clients.

Learn More

Platform policies

PolicyWhat it does
Profile network policiesGenerates the allow NetworkPolicies in every Kubeflow profile namespace, then a default-deny once the namespace is marked ready
Profile network policy validationChecks pod creation in an enrolled profile namespace for its required allow policies and any default-deny it opted into. In Enforce mode it refuses the pod until they exist; in Audit mode it only reports.
Spark ports in profilesOpens the fixed Spark driver and executor ports inside each profile namespace, so a notebook or pipeline Spark driver reaches its executors
S3 credentials broker identityGives opted-in profile pods a rotating service-account token for the S3 credentials broker and points the AWS SDK at the broker
Spark user identityGives pods that run as a per-user Spark ServiceAccount a rotating token for Lakekeeper, so the Iceberg catalog client authenticates as that user's workload
CA bundle trustMounts the internal CA bundle into opted-in pods of storage-client namespaces and points SSL_CERT_FILE and JAVA_TOOL_OPTIONS at it
Argo example imageRewrites the Argo Workflows UI example image docker.io/argoproj/argosay:v2 on workflow pods to the platform registry copy, so the examples run on air-gapped clusters

Profile default-deny is staged per namespace. Kyverno creates the ingress or egress default-deny only in profile namespaces labelled network.bsqai.io/ingress-ready=true or network.bsqai.io/egress-ready=true. The validation policy runs in Audit mode by default ([network_policy] validation_failure_action of the kyverno component). See Roll out default-deny in stages.

Other admission controls

Kyverno is not the platform's only admission control. The platform also ships Kubernetes ValidatingAdmissionPolicy resources, which the API server evaluates without Kyverno. They cover, for example, tenant GPU queues (tenant-gpu-queue), tenant model deployments (tenant-model-deployment), the serving-namespace label (model-serving-namespace-label), agent sandbox pods (agent-sandbox-pod-isolation), the BSQAI API's Secrets in tenant namespaces, MCP lifecycle operator network policies, S3 credentials broker and Spark user identities, the Tuples Kafka user template, and storage classes of removed Ceph storage modes.

Deployment notes

Kyverno v1.18.2 deploys in the proai tier at sync wave 5, before Kubeflow at sync wave 6. Its admission webhooks are scoped to namespaces whose app.kubernetes.io/part-of label is kubeflow-profile, bsqai-storage-client (CA bundle trust), or bsqai-image-rewrite (Argo example image). System namespaces such as kube-system, argocd, and keycloak are unaffected. The forceFailurePolicyIgnore setting ensures that if Kyverno pods are unavailable, resource admission requests pass through instead of blocking the cluster.

Four independent controllers run: admission (2 replicas), background (1), cleanup (1), and reports (1), five pods in total. KYVERNO_ADMISSION_REPLICAS and the matching variables for the other controllers change the counts.

Interacts With

  • Kubeflow, which defines the ClusterPolicy and GeneratingPolicy resources that Kyverno enforces, and whose profile namespaces receive the platform's network policies.
  • Lakekeeper and the S3 credentials broker, whose workload identities Kyverno injects into profile and Spark pods.
  • trust-manager, whose ca-bundle Kyverno mounts into opted-in storage clients.
  • Argo Workflows, whose UI example image Kyverno rewrites to the platform registry on workflow pods.
  • Argo CD, which reconciles Kyverno as part of the proai parent application and must handle controller-mutated policy fields in its ignore-differences configuration.

On this page