# Reference Architecture (/docs/ai/reference-architecture)



AI combines product-facing interfaces with tenant-aware application services, model serving, retrieval, tools, and observability.

## Architecture view [#architecture-view]

<Mermaid
  chart="flowchart TD
    U[&#x22;Users&#x22;] --> PORTAL[&#x22;AI Web Portal&#x22;]
    ADMIN[&#x22;Tenant and platform administrators&#x22;] --> PORTAL
    CLIENT[&#x22;Applications and developer tools&#x22;] --> API[&#x22;BSQAI API&#x22;]
    PORTAL --> API

    API --> KC[&#x22;Keycloak Organizations&#x22;]
    API --> FGA[&#x22;OpenFGA&#x22;]
    API --> LLM[&#x22;LiteLLM&#x22;]
    API --> VDB[&#x22;Tenant-scoped Milvus collections&#x22;]
    API --> DOC[&#x22;Docling workers&#x22;]
    API --> MLF[&#x22;MLflow&#x22;]
    API --> OBJ[&#x22;S3-compatible object storage&#x22;]
    API --> SQL[&#x22;PostgreSQL&#x22;]
    API --> TMP[&#x22;Temporal&#x22;]
    API --> MCP[&#x22;MCP Gateway&#x22;]

    TMP --> PT[&#x22;PlatformTenant resource&#x22;]
    PT --> TENOP[&#x22;BSQAI Tenant Operator&#x22;]
    TENOP --> KC
    TENOP --> TENRES[&#x22;Namespace, quota, optional storage and database&#x22;]

    API --> MCPCR[&#x22;MCPServer resource&#x22;]
    MCPCR --> MCPOP[&#x22;MCP Lifecycle Operator&#x22;]
    MCPOP --> MANAGED[&#x22;Managed MCP Servers&#x22;]
    MCP --> MANAGED
    MCP --> REMOTE[&#x22;Remote MCP Servers&#x22;]

    LLM --> INF[&#x22;Private inference gateway&#x22;]
    INF --> KS[&#x22;KServe&#x22;]
    KS --> VLLM[&#x22;vLLM runtimes&#x22;]

    DOC --> OBJ
    DOC --> VDB
    API --> OTEL[&#x22;Application OTLP signals&#x22;]
    MCP --> OTEL
    OTEL --> ALLOY[&#x22;Alloy Gateway&#x22;]
    KS -. &#x22;Prometheus metrics&#x22; .-> ALLOY
    ALLOY --> OBS[&#x22;Foundation observability&#x22;]"
/>

## AI domains [#ai-domains]

| Domain                     | Main role                                                    | Example components                      |
| -------------------------- | ------------------------------------------------------------ | --------------------------------------- |
| User experience            | Provide chat, files, Agents, MCP, models, and administration | AI Web Portal                           |
| API and orchestration      | Expose tenant-aware application services and workflows       | BSQAI API, Temporal                     |
| Tenant lifecycle           | Reconcile tenant identity and optional platform resources    | BSQAI Tenant Operator, `PlatformTenant` |
| Model lifecycle            | Import, register, and observe models                         | Model Installer, MLflow                 |
| Model access and inference | Route and execute model requests                             | LiteLLM, KServe, vLLM                   |
| Tools and search           | Connect approved MCP and web-search providers                | MCP Gateway, MCP Lifecycle Operator     |
| Retrieval and documents    | Convert, embed, store, and retrieve content                  | Docling, Milvus                         |
| AI observability           | Trace application and inference operations                   | OpenTelemetry, MLflow, Grafana Alloy    |

## Design intent [#design-intent]

Applications integrate with the portal or BSQAI API. Internal model, storage, workflow, and tool components can evolve without changing those primary product interfaces. Tenant identity and authorization remain attached as requests move through each layer.