Grafana Mimir

Multi-tenant metrics store that keeps each tenant's metric series in its own organization.

Agentic Friendly

Component Category

Monitoring and debugging / metrics storage

Component Description

Grafana Mimir is a horizontally scalable, multi-tenant time-series database for Prometheus metrics. BullSequana AI uses it as the tenant-partitioned metrics store: each series is stored in the Mimir organization of the tenant it belongs to, and every read and write names an organization.

Why It Is Used

In BullSequana AI Foundation, Mimir gives metrics the same tenant boundary that Loki gives logs and Tempo gives traces. Per-organization limits keep one tenant's series volume from stopping ingestion for other tenants.

Learn More

Deployment notes

Mimir deploys from the mimir-distributed chart in the mimir namespace at sync wave 10 in the common tier, and is enabled by default. It uses the ingest-storage architecture: distributors write to a bundled Kafka cluster of three brokers, and ingesters consume from it. Ingesters run in two zones and store-gateways in three. Metric blocks are stored in the configured S3-compatible object storage, either the in-cluster Rook-Ceph RGW service or the site's external S3 service, with 720 hours (30 days) of retention.

By default, required anti-affinity keeps ingester zones, store-gateway zones, Kafka brokers, and tenant router replicas on separate nodes, so the cluster needs at least three schedulable nodes. On a single-node cluster, set MIMIR_PLACEMENT_SPREAD_ACROSS_NODES=false: Mimir runs the same topology on one node, without node-level high availability. To spread zones over cloud availability zones instead of nodes, set MIMIR_PLACEMENT_ZONE_TOPOLOGY_KEY=topology.kubernetes.io/zone, which needs at least three availability zones.

On a site with external S3 storage, create the mimir-blocks bucket and set MIMIR_S3_ACCESS_KEY and MIMIR_S3_SECRET_KEY before the first sync. MIMIR_COMPONENT_ENABLED=false removes Mimir together with the Prometheus remote write, the Mimir Grafana data source, and the Mimir query routes.

Tenant routing

Prometheus keeps its local storage and also remote-writes every series it ingests to the Mimir tenant router. The router routes each series on its own, because one scrape of a shared pod can carry series for several tenants:

tenant_id labelMimir organization
Absent, or platformplatform
Lowercase tenant UUIDThat tenant's organization
Any other valuetelemetry-quarantine, with the label quarantine_reason="malformed_tenant_id" added

Only allowlisted tenant-serving metrics carry tenant_id, and the label stays on the stored series. Every other series goes to the platform organization. The router is the intended writer. The Mimir NetworkPolicies keep tenant workloads away from the distributor: they admit only pods in the Mimir namespace, which include the router, Mimir's own components and its Kafka, and the platform Prometheus, which reaches the distributor's port because it scrapes component metrics there. The policies do not stop those admitted pods from writing to the distributor directly with an organization header of their choosing, so per-tenant routing holds for writes that go through the router. NetworkPolicies are a control only where a policy engine enforces them.

Queries

The telemetry query gateway exposes Mimir under three path prefixes and sets the organization header itself:

PrefixOrganization
/mimir/platformplatform
/mimir/tenantThe tenant organization the telemetry authorizer permits for the caller
/mimir/globalThe organizations the telemetry authorizer permits for the caller

The tenant prefix admits organization.telemetry.read and platform.audit.read, like the Loki and Tempo tenant routes. The platform and global prefixes admit platform.audit.read only. A caller permitted to read one tenant's telemetry can read that tenant's organization and is refused for any other. The Grafana Mimir data source uses /mimir/platform, so it returns platform series only. Platform dashboards keep reading local Prometheus.

Limits and alerts

Mimir enforces per-organization limits:

OrganizationActive seriesIngestion rate
Each tenant organization150,00010,000 samples per second
platform1,000,00050,000 samples per second

The series limit blocks only new series; the ingestion-rate limit throttles volume immediately. An organization that reaches its limit leaves the other organizations ingesting. The [limits] table of the mimir component sets the tenant defaults, and [limits.platform] sets the override for the platform organization.

Alert rules in the mimir namespace report routing and limit problems:

AlertCondition
MimirTenantSeriesMalformedContextSeries are routed to quarantine because their tenant_id is malformed
MimirTenantRouterRejectedRequestsThe router drops whole remote-write requests before routing them
MimirTenantRouterForwardErrorsWrites to one organization keep failing for 15 minutes
MimirTenantNearSeriesLimitAn organization is above 80% of its series limit
MimirTenantSeriesLimitReachedMimir drops an organization's new series at its series limit
MimirTenantIngestionRateLimitedMimir drops an organization's samples at its ingestion-rate limit

Interacts With

  • Prometheus, which remote-writes every series to Mimir through the tenant router and keeps its own local storage.
  • Grafana, which queries the platform organization through the Mimir data source.
  • BSQAI API, whose telemetry authorizer decides which organizations a tenant or global query may read.
  • Grafana Loki and Grafana Tempo, which keep logs and traces in the same tenant organizations.
  • Rook Ceph, whose RGW service retains Mimir metric blocks, or the site's external S3 service when one is configured.

On this page