Kubeflow

Pipelines, notebooks, and experiment management for ML, DL, and GenAI workflows.

Note Kubeflow capabilities will expand in future platform versions. This component is subject to change as new features are integrated.

Component Category

Machine learning / pipelines and notebooks

Kubeflow is a Kubernetes-native platform for building and managing ML, deep learning, and GenAI workflows. In BullSequana AI it provides Kubeflow Pipelines for orchestrating experiments, a notebook server manager (Developer Workspace) for interactive development environments (JupyterLab, VSCode), and a profile system for multi-tenant namespace isolation. It is deployed as raw Kustomize manifests (not a Helm chart), based on upstream tag 26.03 with local overlays.

Why It Is Used

In BullSequana AI, Kubeflow provides the experimentation and pipeline execution layer for the Data tier. It gives data scientists and AI engineers a structured environment to build, run, and track ML, DL, and GenAI workflows with pipeline versioning, artifact tracking through Rook Ceph S3 storage, and per-profile namespace isolation. Kubeflow complements MLflow (experiment tracking) and Argo Workflows (general-purpose automation) by focusing specifically on pipeline orchestration and notebook-based development.

Learn More

Deployment notes

Kubeflow deploys in the proai tier at sync wave 6. It depends on two components that must be running first:

Kyverno (sync wave 5) — enforces ClusterPolicy and GeneratingPolicy resources for profile-level RBAC
Metacontroller (sync wave 5) — provides the DecoratorController CRD used by Kubeflow Pipelines

Kubeflow uses Kustomize format instead of Helm. The chart_name field is intentionally empty, so ArgoCD applies the raw manifests from Git instead of pulling a chart from the OCI registry.

Storage uses per-profile S3 buckets (one per Kubeflow profile namespace) backed by Rook Ceph RGW for pipeline artifacts. Authentication is handled through an internal OAuth2 Proxy instance configured against Keycloak, with access controlled through the COREAI-KUBEFLOW-ADMIN-GROUP and COREAI-KUBEFLOW-USERS-GROUP groups.

Custom container images are built for the central dashboard, Jupyter web app, and KFP frontend, stored in the platform registry.

Interacts With

Kyverno, which enforces the ClusterPolicy resources shipped by Kubeflow for profile RBAC and pipeline access controls.
Metacontroller, which provides the DecoratorController CRD required by Kubeflow Pipelines.
Keycloak, which provides SSO authentication through an internal OAuth2 Proxy instance.
Rook Ceph, which provides S3-compatible storage for pipeline artifacts through RGW.
MLflow, which handles experiment tracking alongside Kubeflow's pipeline execution.
trust-manager, which distributes CA trust bundles into Kubeflow namespaces for TLS verification.