Authorization Events Audit
Grant a subject read access to a tenant's authorization audit log and consume it directly from Kafka.
The Data authorization service (Tuples) emits an audit event for each request it completes successfully. This log covers requests made against that service's own API, such as role grants and authorization checks; it doesn't currently record rejected or failed requests. A tenant with Kafka auditing configured can grant a subject, such as a compliance reviewer or a SIEM integration, direct read access to this log, scoped to that permission alone. This page walks through granting that access, minting a Kafka credential, consuming the log, and revoking access again.
Prerequisites
Direct Kafka access to a tenant's authorization audit log requires two things to already be in place:
- The tenant has
kafka_topicconfigured. This is a one-time, platform-operator-owned setting made when the tenant is provisioned, and it's what enables direct Kafka access in the first place; see Kafka and Tuples, the authorization service, for what it enables. - You can grant Keycloak client roles and authorization relations for the tenant, typically as a tenant administrator or platform operator.
How access is gated
Minting a Kafka credential requires two independent checks, and both must pass:
| Check | What it answers | Who grants it |
|---|---|---|
auditor relation | Is this subject allowed to read this tenant's audit log? | A tenant administrator, through the authorization service's API |
tuples-auditor role | Is this principal allowed to trigger the authorization service's Kubernetes-facing credential-minting code path at all? | A platform operator, through Keycloak |
The two checks stay separate because they answer different questions with different
owners. Granting audit visibility to a subject is a product decision a tenant
administrator makes about their own data. Letting a principal trigger the code that
mints credentials, which creates a real Kubernetes resource, is an infrastructure
decision that stays with whoever operates the cluster. Holding only one of the two still
results in a 403 when you try to mint a credential.
Step 1: Grant the tuples-auditor role
A platform operator assigns the tuples-auditor client role to the subject in Keycloak.
- Open the Keycloak Admin Console for the tenant's realm.
- Go to Users, select the subject, and open Role mapping.
- Click Assign role, filter by the tenant's authorization service client (for
example,
tuples-svc), and select tuples-auditor.
This role controls whether the subject can call the audit-token endpoint at all. Pair it
with the auditor relation in the next step to also grant visibility into the tenant's
audit log.
Step 2: Grant the auditor relation
A tenant administrator grants the auditor relation on the subject through the
authorization service's API, from their own Developer Workspace notebook or any
authenticated client.
Use the subject's Keycloak ID, not their email
The authorization service checks the subject against the sub claim of the
subject's own access token, a Keycloak-issued UUID, not their email address.
Granting an email-shaped subject still returns 201, but the later mint
request then fails with a 403, because the granted tuple never matches
what the service checks against. Find the ID in the Keycloak Admin Console
(Users > the subject > ID), or have the subject decode their own
token and read its sub claim.
Example: grant the auditor relation
import bsqai_data
import requests
AUTH_SERVICE_URL = "http://tuples.tuples.svc.cluster.local:8080"
token = bsqai_data.get_token()
response = requests.post(
f"{AUTH_SERVICE_URL}/api/v1/data-platform/roles",
headers={"Authorization": f"Bearer {token}"},
json={"subject": "user:3f9c1a2e-8b4d-4e2a-9c3f-1a2e8b4d4e2a", "relation": "auditor"},
)
print(response.status_code) # 201A 201 response confirms the subject now holds can_audit for this tenant.
Step 3: Mint a Kafka credential
With both checks satisfied, the granted subject mints their own credential, using their own sign-in, not the tenant administrator's from the previous step: minting always issues a credential for whoever's token makes the request.
Example: mint the credential
reviewer_token = bsqai_data.get_token() # signed in as user:reviewer@acme.com
response = requests.post(
f"{AUTH_SERVICE_URL}/api/v1/data-platform/audit-token",
headers={"Authorization": f"Bearer {reviewer_token}"},
)
print(response.status_code) # 200
credential = response.json() # {"username", "password", "brokers", "topic"}{
"username": "auditor-tenant-a-1a2b3c4d5e6f7a8b",
"password": "...",
"brokers": ["kafka:9095"],
"topic": "tenant-a.tuples.audit"
}Keep credential out of anything you print, log, or save: it holds a live Kafka
password. If it's ever exposed by accident, revoke the subject's auditor
relation (see below) to invalidate it.
The response is a SASL/SCRAM-SHA-512 credential for a Kafka user created just for this subject, scoped to read and describe only the tenant's own audit topic. The subject connects to Kafka directly with any standard client library from here on.
Rate limit
A mint request for the same tenant and subject within 5 seconds of the last one
returns 503 with the message "audit token was issued too recently for this
subject, retry shortly." This is a rate limit that bounds how fast a caller can
force credential churn, not an authorization failure. Wait a few seconds and
retry.
Step 4: Consume the audit log
Authenticate with plain SASL/SCRAM-SHA-512 using the username and password from the
mint response, exactly like any other SCRAM login. No token flag or special client option
is needed.
Assign the partition manually
The credential's ACL covers read and describe access to the topic only; it has no consumer-group ACL. If your client library joins a consumer group by default (most do), the broker rejects the group-coordination request and you see an authorization error instead of messages. Assign the topic's partition directly instead, as in the example below, so the client never tries to join a group.
Trust the broker's certificate
The audit listener presents a certificate from Strimzi's own self-signed
cluster CA, which most systems don't trust by default. Connecting with
SASL_SSL and no CA configured fails at the TLS handshake, before
authentication is even attempted. Ask a platform operator for the CA
certificate (the ca.crt key of the <kafka-cluster-name>-cluster-ca-cert
secret in the Kafka namespace) and pass it as ssl_cafile, or, for local
testing only, disable verification.
Example: consume with kafka-python
from kafka import KafkaConsumer, TopicPartition
topic = credential["topic"]
consumer = KafkaConsumer(
bootstrap_servers=credential["brokers"], # from the mint response
security_protocol="SASL_SSL",
ssl_cafile="ca.crt", # the cluster CA cert, from a platform operator
sasl_mechanism="SCRAM-SHA-512",
sasl_plain_username=credential["username"],
sasl_plain_password=credential["password"],
)
partitions = consumer.partitions_for_topic(topic)
consumer.assign([TopicPartition(topic, p) for p in partitions])
for message in consumer:
print(message.value)This uses kafka-python, a
publicly available client library; any Kafka client that supports
SASL/SCRAM-SHA-512 and manual partition assignment works the same way. It
also needs the python-snappy package installed alongside it, since the
audit topic uses Snappy-compressed records; without it, reading raises
UnsupportedCodecError. Read the topic from the mint response rather than
hardcoding a tenant's topic name, since it's configured per tenant. Discover
and assign every partition of the topic rather than hardcoding a partition
number: a tenant's audit topic can have more than one, and a consumer
assigned to only some of them silently misses events published to the rest.
Retry right after minting
A connection attempt in the first moment after minting can occasionally see a stale authentication rejection while the credential finishes propagating to the broker. Retry for a few seconds if this happens; it resolves quickly and doesn't recur once the connection is established.
Each event describes one request to the authorization service's own API, and includes an
id, trace_id, timestamp, tenant_id, user, operation (for example
space.grant_role), outcome, http.method, http.path, and a free-form metadata map
of the resource IDs involved.
Revoke access
Revoke the auditor relation the same way you granted it, with a DELETE request
instead of a POST.
Example: revoke the auditor relation
response = requests.delete(
f"{AUTH_SERVICE_URL}/api/v1/data-platform/roles",
headers={"Authorization": f"Bearer {token}"},
json={"subject": "user:3f9c1a2e-8b4d-4e2a-9c3f-1a2e8b4d4e2a", "relation": "auditor"},
)
print(response.status_code) # 204A 204 response means the subject's auditor relation is gone, so any future
can_audit check, including a new mint attempt, fails from this point on. It doesn't by
itself confirm that direct Kafka access has ended: the authorization service also
attempts to delete the subject's Kafka credential right away, but that delete is
best-effort, and a failed attempt leaves the credential valid until a periodic sweep
removes it. If you need certainty that the credential itself stopped working, confirm
its KafkaUser resource is gone (a platform operator can check this) rather than relying
on the revoke response alone.
Next steps
- Assign the
tuples-auditorrole to any other principal, such as a service account for a SIEM integration, the same way you did in step 1. - See Tuples and Kafka for how the underlying components are deployed.
- See Observability and audit for the platform's general telemetry pipeline, which covers all services and is a separate signal path from the authorization service's own audit log.