Koperator

The Koperator (formerly called Banzai Cloud Kafka Operator) is a Kubernetes operator to automate provisioning, management, autoscaling and operations of Apache Kafka clusters deployed to K8s.

Overview

Apache Kafka is an open-source distributed streaming platform, and some of the main features of the Koperator are:

• the provisioning of secure and production-ready Kafka clusters
• fine grained broker configuration support
• advanced and highly configurable External Access via LoadBalancers using Envoy
• graceful Kafka cluster scaling and rebalancing
• monitoring via Prometheus
• encrypted communication using SSL
• automatic reaction and self healing based on alerts (plugin system, with meaningful default alert plugins) using Cruise Control
• graceful rolling upgrade
• advanced topic and user management via CRD

We took a different approach to what’s out there - we believe for a good reason - please read on to understand more about our design motivations and some of the scenarios which drove us to create the Koperator .

Koperator (formerly called Banzai Cloud Kafka operator) is a core part of Cisco Streaming Data Manager that helps you create production-ready Apache Kafka cluster on Kubernetes, with scaling, rebalancing, and alerts based self healing. While the Koperator itself is an open-source project, the Cisco Streaming Data Manager product extends the functionality of the Koperator with commercial features (for example, declarative ACL handling, built-in monitoring, and multiple ways of disaster recovery). Read a detailed comparison of Streaming Data Manager and the Koperator .

Motivation

Apache Kafka predates Kubernetes and was designed mostly for static on-premise environments. State management, node identity, failover, etc all come part and parcel with Kafka, so making it work properly on Kubernetes and on an underlying dynamic environment can be a challenge.

There are already several approaches to operating Apache Kafka on Kubernetes, however, we did not find them appropriate for use in a highly dynamic environment, nor capable of meeting our customers' needs. At the same time, there is substantial interest within the Kafka community for a solution which enables Kafka on Kubernetes, both in the open source and closed source space.

We took a different approach to what’s out there - we believe for a good reason - please read on to understand more about our design motivations and some of the scenarios which were driving us to create the Koperator .

Finally, our motivation is to build an open source solution and a community which drives the innovation and features of this operator. We are long-term contributors and active community members of both Apache Kafka and Kubernetes, and we hope to recreate a similar community around this operator.

Apache Kafka, Kafka, and the Kafka logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.