Manage Kafka Connectors
Streaming Data Manager automates the deployment of Kafka Connect clusters and the creation of connectors through KafkaConnect custom resource instances and KafkaConnector custom resource instances, respectively.
Imperative Kafka Connect connector management
supertubes install -a
command which will set up a Kafka Connect cluster with default settings, linked to the provisioned demo Apache Kafka cluster and the deployed Schema Registry instance.
supertubes cluster kafka-connector [command]
update sub-commands require KafkaConnector custom resource specifications as descriptors. For details on these commands, see smm sdm cluster kafka-connector.
Declarative Kafka Connect connector management
The pattern for managing connectors is similar to that of Kafka Connect. There is a KafkaConnector custom resource that represents a connector type which is used by Streaming Data Manager to instantiate connectors on Kafka Connect clusters.
apiVersion: kafka.banzaicloud.io/v1beta1 kind: KafkaConnector metadata: name: my-connector spec: # The connector class name (e.g. io.confluent.connect.s3.S3SinkConnector) class: # The maximum number of tasks the Kafka Connector can create tasksMax: # References the Kafka Connect cluster where this connector is to run clusterRef: name: my-kafka-connect # Defines the config values for Kafka Connector in key - value pairs format config: # Whether to pause or resume the connector execution pause:
- [Kafka Connect and Connector examples](/sdm/kafka-connect/examples/