Cluster Registry Custom Resources

Cluster Registry introduces the Cluster, ClusterFeature, and ResourceSyncRule Kubernetes resources which are described here.

Cluster

The Cluster custom resource represents a Kubernetes cluster. The Cluster Registry controller fills the status of the Cluster CR with cluster related metadata.

The following is an example of what a Cluster could look like:

apiVersion: clusterregistry.k8s.cisco.com/v1alpha1
kind: Cluster
metadata:
  labels:
    banzaicloud.io/managed-by: supertubes
  name: cluster1
spec:
  authInfo:
    secretRef:
      name: cluster1
      namespace: cluster-registry
  clusterID: 02818b21-95e6-4b7e-aaf8-6763c009c97b
  kubernetesApiEndpoints:
    - serverAddress: https://3.135.85.19:6443
status:
  conditions:
    - lastHeartbeatTime: "2022-03-23T00:29:36Z"
      lastTransitionTime: "2022-03-23T00:29:30Z"
      message: cluster metadata is set
      reason: ClusterMetadataSet
      status: "True"
      type: ClusterMetadataSet
  distribution: PKE
  kubeProxyVersions:
    - v1.19.10
  kubeletVersions:
    - v1.19.10
  locality:
    region: us-east-2
    regions:
      - us-east-2
    zones:
      - us-east-2a
  provider: amazon
  state: Ready
  type: Local
  version: v1.19.10

As you can see, the status has been filled with the provider, locality, zone, distribution, and version info. Additionally, it contains a clusterID which is set to be the ID of the kube-system namespace of the Kubernetes cluster it represents.

Resource Sync Rule

The ResourceSyncRule custom resource represents a synchronization rule for a particular type of resource to be carried out amongst a group of attached Kubernetes clusters.

The following shows a resource sync rule that syncs KafkaCluster resources across a group of attached Kubernetes clusters:

apiVersion: clusterregistry.k8s.cisco.com/v1alpha1
kind: ResourceSyncRule
metadata:
  annotations:
    cluster-registry.k8s.cisco.com/resource-sync-disabled: "true"
  labels:
    banzaicloud.io/managed-by: supertubes
    cluster-registry-controller.k8s.cisco.com/core-sync-resource: "true"
    supertubes.banzaicloud.io/cli-version: add-missing-rsr
  name: sdm-core-resources-kafka-clusters-sink
spec:
  clusterFeatureMatch:
  - featureName: sdm-core-resources-source
  groupVersionKind:
    group: kafka.banzaicloud.io
    kind: KafkaCluster
    version: v1beta1
  rules:
  - mutations:
      overrides:
      - path: /metadata/name
        type: replace
        value: '{{ printf "%s-%s-%s" .Object.GetName .Cluster.GetName (trunc 4 (sha256sum
          .Cluster.GetName))}}'
      syncStatus: true

In the spec groupVersionKind specifies KafkaCluster objects from api group kafka.banzaicloud.io can be synced to the local Kubernetes clusters from attached Kubernetes clusters. The target namespace of the objects remains the same as it was in the original Kubernetes cluster. In order to avoid naming collisions, the name of the source Kubernetes cluster and a four digit hash of the source Kubernetes cluster is appended to the original name of the KafkaCluster before it is synchronized to attached Kubernetes clusters.

Cluster Feature

The ClusterFeature custom resource represents a feature (like a software feature such as “Service Mesh Manager” or “Streaming Data Manager”) and can be used to group resources together that are needed to support a specific feature. This is useful in case there is more than one feature running in a group of Kubernetes clusters that use cluster registry to synchronize resources. For example, if SMM is running in one Kubernetes cluster but not another, SMM resources should not be synchronized to the cluster where SMM is not running. This is where Cluster Feature can come in handy because it can act as a kind of filtering mechanism. Resources specified in the resource sync rule only get copied from clusters which have the ClusterFeature object with the feature-specific name defined. If you look again at the example resource sync rule above, you will see that in the spec there is a clusterFeatureMatch section specified with featureName sdm-core-resources-source.

apiVersion: clusterregistry.k8s.cisco.com/v1alpha1
kind: ClusterFeature
metadata:
  labels:
    banzaicloud.io/managed-by: supertubes
    cluster-registry-controller.k8s.cisco.com/core-sync-resource: "true"
  name: sdm-core-resources
spec:
  featureName: sdm-core-resources-source