Running ksqlDB in headless mode

One of the main features of ksqlDB is the capability of running in headless mode(non-interactive mode).

Streaming Data Manager has built-in support for managing ksqlDB servers running in headless mode.

Prerequisites

  • You must create a file containing the set of queries that is assigned to the ksqlDB server(s). Currently Streaming Data Manager only supports providing the queries.sql file as a ConfigMap object.

    Provide the name of the configmap in the spec.queryConfigMapName field of the KsqlDB custom resource. This setting defaults to <name of the KsqlDB CR>-ksql-queries-configmap. The file is supplied as the KSQL_KSQL_QUERIES_FILE environment variable to the server (this cannot be modified).

  • In order to enable headless mode, set the spec.headless field of the KsqlDB custom resource to true.

Behavior

  • In headless mode the ksqlDB REST API is disabled, therefore authorization policies will not take effect.

  • If Kafka ACLs are enabled, configure the input and output topics configurations (spec.inputTopics and spec.outputTopics fields in KsqlDB custom resource) aligned with your queries to enable ksqlDB to access those topics.

Architecture

The following diagram shows how ksqlDB integrates with Streaming Data Manager.

Streaming Data Manager integrating with ksqlDB in headless mode

For more details on how Streaming Data Manager manages the ksqlDB servers, read our Managing ksqlDB with Streaming Data Manager blog post.