Synchronizing SDM resources across multiple Kubernetes Clusters

Streaming Data Manager (SDM) leverages the Cluster Registry Controller to form a group of Kubernetes clusters and synchronize resources between them. This reduces the operational overhead of administering SDM on multiple Kubernetes clusters because it allows you to view SDM resources spanning the whole cluster by querying any single one of them.

The Cluster Registry controller works in a fully distributed topology, there is no leader or single point of failure in the system. The Cluster custom resource represents a Kubernetes cluster. The Cluster Registry controller fills the status of the Cluster CR with cluster-related metadata. Once cluster-registry is installed and Kubernetes clusters have been attached to each other, SDM synchronizes SDM and Koperator specific custom resources (ApplicationManifest, KafkaCluster, KafkaTopic, KafkaUser, KafkaACL, KafkaRole, and KafkaResourceSelector) between the attached clusters by default. These default synchronization settings are defined in a series of ResourceSyncRule custom resources which can be edited in order to customize the behavior to suit your needs.