Data catalog
How BullSequana AI aggregates authorization, table metadata, and enrichment into a single data catalog.
The data catalog is an API for application developers and data engineers. It is distinct from the Data Hub connector catalogue, which lists ingestion connectors rather than governed tables.
The data catalog is the discovery surface for governed tables on BullSequana AI. It combines what a user is allowed to see, what a table's structure and location look like, and optional business context into one view instead of requiring separate lookups against each underlying service.
What the catalog provides
Without a catalog, discovering a table means knowing which catalog and schema it lives in, checking authorization separately, and reading raw metastore metadata to understand its structure. That approach doesn't scale once a tenant has more than a handful of datasets, and it gives data consumers no way to tell whether a table is governed, who owns it, or what it contains before they query it.
The data catalog exists to close that gap. Its goals are to:
- give users a single place to discover the tables they're authorized to see, scoped to their identity
- show table structure (columns, partitions, storage location) without requiring direct access to the metastore or the authorization service
- surface business context, such as descriptions, tags, glossary terms, and lineage, where that context is available
- let table and space owners manage permissions from the same surface where they discover data
By aggregating these concerns behind one API, the catalog keeps authorization, storage metadata, and business metadata as separate systems of record while presenting them to users as one coherent experience.
What the catalog aggregates
The data catalog aggregates data from three sources and merges the results per table.
| Source | What it provides |
|---|---|
| Authorization service | Which tables a user can discover, and direct authorization grants on a table |
| Metastore | Table structure (columns and types), partition layout, and storage location |
| Metadata API | Description, tags, glossary terms, quality score, and upstream and downstream lineage |
The authorization service and the metastore are required dependencies. The Metadata API is an optional enrichment source: when it isn't deployed, the catalog still returns authorization and structure data, just without the enrichment fields.
How it works
Every request to the catalog's API is authorized with respect to the identity of the user, which also scopes the response to only the tables that user is allowed to discover.
Managing table permissions
The catalog also supports the permissions workflow that sits next to discovery:
- Grants: the catalog returns the direct authorization grants stored for a table, so a permissions view can show exactly who has been granted access and at what level.
- Can-manage check: the catalog resolves a table's parent space and checks whether the requesting user owns that space. The result tells the caller whether to show permissions management controls or a request-access action instead.
Graceful degradation
The catalog's availability depends on the authorization service: if it's unreachable, the catalog is not ready, since authorization is required for every catalog operation. A metastore failure is scoped to the affected table rather than failing the whole request. A Metadata API failure, or the Metadata API not being deployed, is treated as unavailable enrichment, never as a request failure.