Grafana Tempo

Distributed tracing component for Foundation observability.

Agentic Friendly

Component Category

Monitoring and debugging / distributed tracing

Component Description

Grafana Tempo is a distributed tracing backend designed to store and query traces at scale.

Why It Is Used

In BullSequana AI Foundation, Tempo helps operators follow requests across multiple services, understand latency paths, and diagnose issues that span system boundaries.

Learn More

Deployment notes

Tempo 2.8 deploys in distributed mode in the tempo namespace at sync wave 11 in the common tier, after the Alloy gateway at sync wave 10. Trace data is stored in the configured S3-compatible object-storage backend, either the in-cluster Rook-Ceph RGW service or the site's external S3 service, and kept for 720 hours (30 days). A memcached layer provides store-level caching.

Tenant organizations

Tempo runs with multi-tenancy on. Each span is stored in the organization of its tenant.id, which the Alloy gateway sets: platform, the tenant's organization, or quarantine for missing or malformed tenant context. Queries go through the telemetry query gateway, which sets the organization itself. The platform Grafana Tempo data source reads the platform organization, and each tenant Grafana Organization reads its tenant's organization through the gateway's tenant routes. See Observability and audit.

The metrics generator writes service-graph and span metrics to the platform Prometheus. These metrics have no tenant dimension and are platform metrics.

Interacts With

  • Grafana, which uses Tempo for trace exploration and trace-to-logs or trace-to-metrics navigation.
  • Prometheus, through Tempo metrics, service monitors, and span-metrics generation.
  • Rook Ceph, which provides S3-compatible object storage for Tempo trace data through RGW, or the site's external S3 service when one is configured.
  • Grafana Alloy, whose gateway routes traces into Tempo by tenant.
  • Instrumented platform services, which send traces into Tempo through the telemetry pipeline.

On this page