Lakekeeper

Iceberg REST catalog for tenant-scoped lakehouse projects and warehouses.

Agentic Friendly

Component Category

Lakehouse catalog

Component Description

Lakekeeper provides an Apache Iceberg REST Catalog for data-engineering and analytics workloads. BullSequana AI creates a separate Lakekeeper project with a default warehouse for each tenant with Data services. The Kubeflow profile bucket provisioner ensures personal and shared-space namespaces within that tenant's default warehouse. Object-storage buckets and catalog namespaces are separate resources; a bucket does not create another warehouse.

Why It Is Used

In BullSequana AI, Lakekeeper gives Spark and other Iceberg clients one governed catalog rather than coupling table metadata to a specific processing engine. Keycloak authenticates users and OpenFGA enforces project and warehouse access. Tuples also assigns project-level create and modify privileges directly to users who hold the tenant's data_admin functional role.

Learn More

Deployment notes

BullSequana AI deploys Lakekeeper 0.12.2 with chart 0.11.0 at sync wave 7. Table data is stored in the configured S3-compatible object store; catalog metadata uses PostgreSQL. Each tenant warehouse uses its own key-prefix in the shared lakekeeper bucket. Tenant resolution determines the project and shared warehouse used when the provisioner ensures a personal or space namespace.

A bootstrap Job creates the project and default warehouse for each tenant declared in the [auth] organizations list of the Keycloak component. Lakekeeper authenticates to OpenFGA with its own pre-shared key.

Lakekeeper also accepts Kubernetes ServiceAccount tokens with the lakekeeper audience. Per-user Spark pods present such a projected token and authenticate as the user's Spark workload identity, which the profile bucket provisioner registers with that user's catalog grants. On Rook-Ceph with STS federation, per-tenant web-identity roles scoped to each tenant's key prefix back Spark storage access through the optional S3 credentials broker.

Interacts With

  • Spark Operator, whose per-user Spark pods authenticate to the REST catalog with a projected ServiceAccount token.
  • Keycloak, which authenticates users and services.
  • OpenFGA, which authorizes projects and warehouses.
  • Tuples, which assigns project-level create and modify privileges to tenant data administrators.
  • Kubeflow, whose profile bucket provisioner ensures personal and shared-space namespaces in the tenant's default warehouse and registers per-user Spark identities.
  • BSQAI API, whose data catalog reads table structure with the caller's token.
  • PostgreSQL, which stores catalog state.
  • Rook-Ceph or external S3-compatible storage, which stores table data.

On this page