The Koperator (formerly called Banzai Cloud Kafka Operator) is a Kubernetes operator to automate provisioning, management, autoscaling and operations of Apache Kafka clusters deployed to K8s.
Apache Kafka is an open-source distributed streaming platform, and some of the main features of the Koperator are:
- the provisioning of secure and production-ready Kafka clusters
- fine grained broker configuration support
- advanced and highly configurable External Access via LoadBalancers using Envoy
- graceful Kafka cluster scaling and rebalancing
- monitoring via Prometheus
- encrypted communication using SSL
- automatic reaction and self healing based on alerts (plugin system, with meaningful default alert plugins) using Cruise Control
- graceful rolling upgrade
- advanced topic and user management via CRD
We took a different approach to what’s out there - we believe for a good reason - please read on to understand more about our design motivations and some of the scenarios which drove us to create the Koperator .
Koperator (formerly called Banzai Cloud Kafka operator) is a core part of Cisco Streaming Data Manager that helps you create production-ready Apache Kafka cluster on Kubernetes, with scaling, rebalancing, and alerts based self healing. While the Koperator itself is an open-source project, the Cisco Streaming Data Manager product extends the functionality of the Koperator with commercial features (for example, declarative ACL handling, built-in monitoring, and multiple ways of disaster recovery). Read a detailed comparison of Streaming Data Manager and the Koperator .
Apache Kafka predates Kubernetes and was designed mostly for
static on-premise environments. State management, node identity, failover, etc all come part and parcel with Kafka, so making it work properly on Kubernetes and on an underlying dynamic environment can be a challenge.
There are already several approaches to operating Apache Kafka on Kubernetes, however, we did not find them appropriate for use in a highly dynamic environment, nor capable of meeting our customers' needs. At the same time, there is substantial interest within the Kafka community for a solution which enables Kafka on Kubernetes, both in the open source and closed source space.
We took a different approach to what’s out there - we believe for a good reason - please read on to understand more about our design motivations and some of the scenarios which were driving us to create the Koperator .
Finally, our motivation is to build an open source solution and a community which drives the innovation and features of this operator. We are long-term contributors and active community members of both Apache Kafka and Kubernetes, and we hope to recreate a similar community around this operator.
Apache Kafka, Kafka, and the Kafka logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.