# Reference Architecture (/docs/coreai/reference-architecture)



CoreAI is composed of several cooperating services. Some are user-facing, some provide shared AI capabilities, and others support model lifecycle, retrieval, and observability.

Architecture View [#architecture-view]

<Mermaid
  chart="flowchart TB
    U[&#x22;Users and Teams&#x22;] --> PORTAL[&#x22;CoreAI Web Portal&#x22;]

    PORTAL --> API[&#x22;CoreAI API&#x22;]
    PORTAL --> MI[&#x22;Model Installer&#x22;]
    PORTAL --> MLF[&#x22;MLflow&#x22;]
    PORTAL --> LLM[&#x22;LiteLLM&#x22;]
    PORTAL --> AUTH[&#x22;Runtime Identity&#x22;]

    AUTH --> KC[&#x22;Keycloak&#x22;]

    API --> KC
    API --> LLM
    API --> VDB[&#x22;Milvus&#x22;]
    API --> DOC[&#x22;Docling&#x22;]
    API --> MLF
    API --> OBJ[&#x22;MinIO&#x22;]
    API --> SQL[&#x22;PostgreSQL&#x22;]
    API --> OTEL[&#x22;OpenTelemetry&#x22;]
    API --> TH[&#x22;ToolHive&#x22;]

    MI --> KC
    MI --> MLF
    MI --> OBJ
    MI --> LLM
    MI --> OTEL
    MI --> INF[&#x22;Runtime Inference&#x22;]

    INF --> KAI[&#x22;KubeAI&#x22;]
    INF --> VLLM[&#x22;vLLM&#x22;]
    INF --> FW[&#x22;FasterWhisper&#x22;]

    DOC --> OBJ
    DOC --> VDB
    LLM --> VLLM
    LLM --> KAI

    MLF --> OBS[&#x22;Runtime Observability&#x22;]
    OTEL --> OBS"
/>

CoreAI Domains [#coreai-domains]

| Domain                  | Main role                                           | Example components          |
| ----------------------- | --------------------------------------------------- | --------------------------- |
| User experience         | Provide the main product entry point                | CoreAI Web Portal           |
| API and orchestration   | Expose stable application services and AI workflows | CoreAI API, Model Installer |
| Model lifecycle         | Manage tracked and installable models               | MLflow, Model Installer     |
| LLM access and tools    | Standardize model access and MCP integration        | LiteLLM, ToolHive           |
| Retrieval and documents | Parse content and support semantic retrieval        | Docling, Milvus             |
| AI observability        | Trace and instrument AI workloads                   | OpenTelemetry, MLflow       |

Design Intent [#design-intent]

CoreAI keeps AI application capabilities grouped in one layer, while still relying on Runtime for networking, security, storage, inference, and operational services. The goal is to give users and developers stable product-facing entry points even when internal components evolve over time.