Data
Connect, govern, process, analyze, and operationalize data on BullSequana AI.
Data is the product domain for moving information from source systems into governed pipelines, workspaces, models, analytics, and AI experiences. It connects data engineering and data science without requiring users to operate the underlying platform components.
Choose an outcome
Ingest data in Studio
Connect sources and destinations and operate managed ingestion.
Discover governed tables
Search the tables you can see and review their grants.
Run Spark workloads
Use Compute clusters and jobs, and connect from Dev environments.
Build a data workflow
Follow the path from a source to an operated data or ML result.
Open a Developer Workspace
Use managed development environments, pipelines, and MLflow.
Build and run pipelines
Upload, execute, schedule, and inspect pipeline runs.
Choose a data path
Compare ingestion, processing, analytics, and AI knowledge paths.
Capability map
Integrate and move data
Data integration connects applications, databases, object stores, and external services to governed platform workflows. Connectors define the source and destination; orchestration controls when work runs; operational signals show whether transfers are healthy.
In the Portal, the Data Ingestion group of Studio configures connections, discovers streams, schedules ingestions and shows runs; see Data ingestion in Studio. The managed-ingestion reference explains the BSQAI API, Temporal and connector runtime, including Meltano/Singer integration.
Engineer and orchestrate data
Lakehouse and pipeline capabilities organize catalog-backed tables, processing jobs, artifacts, schedules, and dependencies. Developer Workspace pipelines provide the practical path for building and operating reusable workflows.
Process data at scale
Apache Spark runs distributed processing in the user's profile namespace. Compute in Studio creates Spark clusters and runs one-off PySpark jobs; dev environments connect to a running cluster with the platform helper. The BSQAI API applies Spark roles and per-user limits to every path. See Run Spark workloads.
Govern and discover data
The data catalog combines authorized table discovery, schema, and optional enrichment metadata. Users search it in Studio under Discover > Data Catalog, and applications call it through the BSQAI API.
Catalog and authorization capabilities describe schemas, ownership, permissions, lineage, and the tenant or team allowed to use an asset. Multi-tenancy and tiered RBAC explains how those boundaries apply across Data. Metastore explains how tables are cataloged and how access is granted.
Develop models and experiments
Developer Workspace provides notebook-first and Python-first environments. MLflow records experiments, artifacts, and model registrations so that development work can move into governed AI serving without losing its history. Notebooks and pipelines reach MLflow through the BSQAI API.
Start with track experiments and register models.
Stream and analyze
Streaming moves events through Kafka-backed paths. Analytics and BI turn analytical data into dashboards and operational views in Superset. These paths share access-control, observability, and storage responsibilities with batch workflows.
Typical data-to-production flow
- Identify the source, owner, classification, and expected refresh behavior.
- Choose a connector or approved ingestion path.
- Develop transformations or models in a managed workspace.
- Package the work as a repeatable pipeline with artifacts and parameters.
- Track experiments and register models when machine learning is involved.
- Apply tenant, team, catalog, and data-access policies before production use.
- Monitor freshness, failures, consumption, and downstream impact.