Reference architecture
A high-level view of how Foundation capabilities fit together.
The Foundation layer is not a single component. It is a platform architecture made of several cooperating domains.
Architecture view
External DNS manages DNS records; it is not a proxy in the request path. The component groups above show service dependencies, not a requirement that every request traverse every service.
Calls to in-cluster models stay inside the cluster. The BSQAI API resolves the model's route and sends a call for an in-cluster model through the internal model gateway, which admits only namespaces enabled for model serving. A route to an external provider connection leaves the cluster from the BSQAI API and does not use the internal gateway. See Inference.
Each tenant gets its own namespace, tenant-<slug>. The BSQAI API records the requested tenant state, Temporal runs the lifecycle workflow, and the BSQAI Tenant Operator reconciles the PlatformTenant resource into the tenant's Keycloak Organization, namespace, storage, databases, GPU queues, and optional services. See Tenant lifecycle control plane.
Secret delivery
The two controller paths are alternatives selected for the deployment. With external-vault, the renderer omits Sealed Secrets and creates a Vault-backed ClusterSecretStore; the Vault and its read token are prerequisites, not resources provisioned by the platform. Kubernetes-generated service-account token Secrets are not externalized. OpenBao is a separate platform component, not a mandatory intermediary between workloads and the external Vault.
See External Vault secrets for bootstrap, refresh and verification.
Capability domains
| Domain | Main role | Example components |
|---|---|---|
| Network | Expose, publish, route, and restrict traffic | Gateway API (Envoy Gateway), internal model gateway, infrastructure load balancer, External DNS, Kubernetes NetworkPolicies |
| Security | Authenticate, authorize, protect secrets, and enforce admission rules | Keycloak, OpenFGA, cert-manager, Sealed Secrets or External Secrets Operator, Kyverno and ValidatingAdmissionPolicies, optional Kubescape; separate OpenBao service |
| Inference | Run production model workloads | KServe, vLLM, Speaches serving runtimes, KAI Scheduler, Envoy AI Gateway, Gateway API Inference Extension |
| Workflow engines | Execute event-driven, pipeline, and durable processes | Argo Events, Argo Workflows, Temporal |
| Data plane | Store operational state, vectors, and artifacts | CloudNativePG, PgAdmin, Milvus, Rook-Ceph or external S3-compatible storage |
| Tenant lifecycle | Reconcile each tenant's Keycloak Organization, namespace, quota, network policy, object storage, databases, optional Milvus database and private MLflow, GPU queues, model-serving admission, and Grafana Organization | BSQAI Tenant Operator, PlatformTenant |
| Observability | Collect, route, authorize, and explore platform signals | Grafana, Prometheus, Mimir, Loki, Tempo, Grafana Alloy, Alloy OTLP gateway, telemetry query gateway |
| Delivery | Move artifacts and desired state into the cluster | Git repository, OCI registry, Argo CD |
Component relationships
Network and exposure
- Gateway API (Envoy Gateway) handles all public HTTP(S) ingress and routing through
HTTPRouteresources. - The internal model gateway connects the BSQAI API to Global Organization and tenant models.
- The infrastructure load balancer makes the shared gateway reachable from the target network.
- External DNS publishes hostnames when the site configures a DNS provider.
- cert-manager issues the gateway's wildcard certificate, and trust-manager and Reflector distribute CA bundles and shared secrets across namespaces.
Access and security
- Keycloak provides identity for the Portal, the BSQAI API, and the platform UIs. Each tenant is a Keycloak Organization.
- OpenFGA stores the relationships behind roles and permissions.
- Sealed Secrets or External Secrets Operator deliver platform secrets to workloads.
- Kyverno and Kubernetes admission policies enforce workload rules, such as the runtimes and resource profiles a tenant model may use.
- Kata Containers, where the site enables the agent sandbox, isolate each sandbox workspace in its own virtual machine.
Execution and state
- KServe reconciles model-serving workloads from
LLMInferenceServiceandInferenceServiceresources. - vLLM runs language models selected through platform runtime profiles.
- KAI Scheduler places GPU workloads in platform and tenant queues, while Envoy AI Gateway provides the private data path.
- Temporal, Argo Workflows, and Argo Events handle durable workflows, pipelines, and event-based automation.
- PostgreSQL stores structured operational state, and Milvus stores vectors.
- Rook-Ceph provides S3-compatible object storage, and block and shared-file storage when the site's storage classes use Ceph.
- The BSQAI Tenant Operator reconciles each tenant's Keycloak Organization, namespace, storage, databases, private MLflow, GPU queues, and Grafana Organization.
Observability
- Prometheus collects metrics from Foundation services and remote-writes them to Mimir, which stores them per tenant.
- Grafana Alloy collects node logs and, through the Alloy OTLP gateway, routes application logs, traces, and metrics by their authorized tenant.
- Loki stores logs, and Tempo stores traces.
- The telemetry query gateway authorizes each Grafana query against the caller's tenant.
- Grafana brings these signals together into dashboards and operational views.
Delivery
- The configured Git service stores versioned source and rendered manifests. GitLab and GitHub are supported options.
- The configured OCI registry stores images and Helm charts. Harbor is the reference customer-operated option, but another compatible registry can be used.
- Argo CD deploys the desired state into the cluster.
Important examples
- PgAdmin -> PostgreSQL PgAdmin exists to operate PostgreSQL more easily.
- Grafana -> Prometheus, Mimir, Loki, Tempo Grafana is the common interface for metrics, logs, and traces.
- Rook-Ceph -> Loki, Mimir, Tempo, MLflow, Milvus, Argo Events, Argo Workflows, KServe, CloudNativePG backups Rook-Ceph provides S3-compatible storage through RGW to several Foundation and higher-layer consumers. A site with external S3-compatible storage serves the same consumers from its own provider.
- Argo CD -> all platform components Argo CD is the main GitOps deployment engine across the stack.
- Git and OCI services -> Argo CD Argo CD reads rendered manifests from the configured Git repository and retrieves Helm charts from the configured OCI registry. Cluster nodes pull workload images from that registry. These services are external delivery dependencies rather than platform-managed workloads.
For component-level details, use the Foundation component catalog.
Design intent
This architecture keeps infrastructure concerns explicit and reusable. Higher layers consume Foundation services without rebuilding the same operational patterns for each product area.