Kafka ACLs over Istio overview

Using Streaming Data Manager, you accomplish two things without having to modify your Kafka client libraries or code:

• Authenticate the Kafka clients using their Kubernetes namespaces and service accounts
• Automatically apply the ACLs defined for the <namespace>-<serviceaccount> Kafka user to the client applications

The authentication and authorization flow looks like this under the hood:

1. When a Kafka client application runs inside an Istio mesh where mTLS communication is provided by Istio, Istio generates a certificate for the client application, which is used during the mTLS session. This Istio-generated certificate contains the namespace and the name of the service account of the client application.
2. Streaming Data Manager (or more precisely, the Kafka ACL WASM filter for Envoy in Streaming Data Manager) checks this certificate when traffic from the client application flows towards a broker. It extracts the namespace and service account information, then mutates the traffic by injecting <namespace>-<serviceaccount> into it.
3. The Kafka brokers receive this mutated stream that contains <namespace>-<serviceaccount>. Streaming Data Manager reads the <namespace>-<serviceaccount> from the TCP stream and uses it as the Kafka Principal that represents the client application.

From the client application’s point of view this is the least intrusive solution, as it doesn’t require any changes.

Supported scenarios

Depending on where the client applications run relative to the Istio service mesh of your Kafka deployment, you can configure Streaming Data Manager for the following scenarios:

• Client applications are in the same Istio mesh as the Kafka cluster
• Client applications are in the same Istio mesh and use OAuth
• Client applications are in the same Kubernetes cluster as the Kafka cluster, but outside the Istio mesh
• Client applications are outside the Kubernetes cluster