Create KafkaCluster using the CLI

To create a Kafka cluster using the CLI, run the following commands.

  1. Verify that Zookeeper is installed on the cluster.

    kubectl get pods -n zookeeper

    If the Zookeeper is not installed, run:

    smm sdm zookeeper install

    If you want to install Zookeeper using GitOps methods, see Deploy ZooKeeper cluster application.

  2. Create a Kafka cluster.

    smm sdm cluster create

    This command creates the following resources:

    • A service account called kafka-cluster, which is used by the Kafka broker pods.

    • A KafkaCluster custom resource with:

      • 2 Kafka brokers
      • 2 internal listeners (port: 29092, 29093)
        • The name of the service depends on the KafkaCluster.spec.headlessServiceEnabled value. If the boolean value KafkaCluster.spec.headlessServiceEnabled is true then the service is kafka-headless:29092 else kafka-all-broker:29092 (load balancing takes place at this address).
      • 10GB broker log dir storage
      • Zookeeper address: zookeeper-server-client.zookeeper:2181
  3. Verify that the Kafka cluster is running.

    smm sdm cluster get

    The output should look similar to:

    Namespace  Name    State              Image                                    Alerts  Cruise Control Topic Status  Rolling Upgrade Errors  Rolling Upgrade Last Success
    kafka      kafka    ClusterRunning     0       CruiseControlTopicReady       0                       2023-01-31 20:17:25