How to manage multiple Apache Kafka clusters using the dashboard

Using the multi-cluster feature of Streaming Data Manager, you can view the resources of Apache Kafka clusters that reside on remote Kubernetes clusters.

Kafka Cluster Selector

To select the Kafka cluster to view, click the Kafka cluster selector at the top of the page, then select a Kafka cluster. Note that for Kafka clusters that reside on remote Kubernetes clusters, the following limitations apply, but you can easily access the Streaming Data Manager dashboard of the remote cluster to avoid the limitations.

Kafka cluster selector Kafka cluster selector

Kubernetes clusters are indicated by the Kubernetes logo

Kubernetes namespaces are indicated by the three-dot icon

Kafka clusters are indicated by the Apache Kafka logo

Click on the desired Kafka cluster to open that cluster.

Limitations

Due to technical limitations, selecting a Kafka cluster that resides on a remote Kubernetes cluster disables some features of the Streaming Data Manager dashboard. To avoid these limitations, see Redirect to remote Kubernetes cluster.

The dashboard will also only operate in read-only mode, meaning the following operations are not supported on a remote cluster:

  • Topic Creation
  • Topic Deletion
  • Topic Configuration modification
  • Kafka User Creation
  • Kafka User Deletion
  • Kafka User Certificate download

Currently unsupported pages/properties include:

  • Brokers page

  • Consumers page

  • ACL/ACLs page

  • Topic details page including:

    • Topic partitions
    • Topic consumer groups
    • Topic configuration
    • Topic ACLs

  • Topics page

    • Number of all topics on cluster
    • Number of partitions on cluster
    • Number of under replicated partitions
    • Total number of partitions without a leader
    • Partitions under minimum in-sync replication number
    • Topic message count
    • Topic in/out traffic in bytes
    • Topic consumers
    • Topic last message date stamp
    • Topic spread

  • While the dashboard is in remote cluster mode, a warning notification is displayed at the top of the page which contains a Redirect button to the remote Kubernetes cluster’s dashboard. Redirecting to a remote cluster Redirecting to a remote cluster

  • The following pages also display a Redirect button to navigate to the dashboard of the remote cluster: MENU > BROKERS, MENU > CONSUMERS, MENU > ACL > ACLs, and the details of a topic on the MENU > TOPICS page.

    Redirecting to a remote cluster Redirecting to a remote cluster