OpenShift integration

Red Hat OpenShift provides scalable and reliable solutions to monitor microservices and offers full-fledged container security. With a Red Hat OpenShift Service on AWS (ROSA) setup, you can seamlessly install Calisti on Red Hat OpenShift clusters. Calisti avoids vendor lock-in so that you can mix-n-match your cluster cloud providers and transition or continue using mixed multi-clusters (GKE, EKS, AKS, OpenShift).

Supported OpenShift Versions

Cloud Managed Services/Platforms	Calisti 1.12.x
Red Hat OpenShift Service on AWS (ROSA)	4.11

Prerequisites

Prepare a ROSA cluster.
Install the oc command-line tool.
Check the supported ROSA versions before installing Calisti.

Install Calisti on OpenShift

To install Calisti using the CLI, run the Calisti installation command with --platform=openshift flag:
```
smm install --platform=openshift
```
To install Calisti as a Kubernetes operator, set the spec.platform field to openshift in the Calisti control plane manifest:
```
spec:
  platform: openshift
```

Istio Container Network Interface and NetworkAttachmentDefinition

Calisti performs the following two steps to have istio-cni work well in OpenShift.

Install Istio CNI pods

The Container Network Interface (CNI) on OpenShift is managed by Multus and istio-cni is not enabled in OpenShift by default. Calisti can invoke istio-cni with ease. During the Calisti installation process, the following CNI configuration is automatically patched to the existing Istio control plane manifest:

cni:
  binDir: /var/lib/cni/bin
  chained: false
  confDir: /etc/cni/multus/net.d
  confFileName: istio-cni.conf
  daemonset:
  image: registry.eticloud.io/smm/istio-install-cni:v1.15.3-bzc.0
    securityContext:
      privileged: true
  enabled: true

The meaning of the fields is:

binDir is the directory path to store CNI binaries.
chained determines whether to install CNI plugin as chained or standalone.
confDir is the directory path on the host where CNI network plugins are installed.
confFileName specifies the name of the CNI configuration file.

With the CNI configuration pathched, the istio-operator invokes istio-cni on OpenShift. After the Istio control plane is updated, the istio-operator reconciles and deploys istio-cni DaemonSets which create related pods on every node in the cluster.

For more details on how istio-cni works, see the Istio CNI plugin documentation.

Install NetworkAttachmentDefinition objects

The network Redirect method via the istio-cni is also required for Openshift support. The NetworkAttachmentDefinition custom resource objects are required for the istio-cni to be invoked. Calisti automatically deploys NetworkAttachmentDefinition into the data plane namespaces with Istio sidecar enabled (for example, smm-system, supertubes-system).

For details, see the Istio documentation.

SecurityContextConstraints

The Calisti container security configurations align with OpenShift’s pod and container security standards.

Note: Calisti components that are installed on Openshift adhere to the SecurityContextConstraints set forth by the Red Hat SCC standards.

UID/PID `1337` for Istio sidecars

The Istio network redirect layer assumes the istio-proxy is running in the pod with UID/GID 1337. This is not configurable in Istio. The Istio injection templates for sidecars and gateways enable the istio-proxy containers' securityContext field with:

runAsGroup: 1337
runAsNonRoot: true
runAsUser: 1337

The default OpenShift SecurityContextConstraints reject the UID/GID of 1337 as it’s not in the [1001190000, 1001199999] range.

To allow pods with istio-proxy sidecars/gateways to come up, the service account for the pod needs to be added to the nonroot-v2 policy. One or more of the following SecurityContextConstraints approaches need to be implemented to attach to nonroot-v2 SCC policy.

RBAC bindings to SecurityContextConstraints

The Istio control-plane component service accounts require RBAC bindings to SecurityContextConstraints (SCC). Here is the list of SCC bindings Calisti deploys for Istio:

istiod and istio meshgateway: nonroot-v2 SCC binding for UID/GID 1337
istio-cni-node pods: privileged SCC binding for NET_ADMIN to allow iptables setup and hostnetwork SCC binding for access to the node’s network namespaces (netns)

Calisti deploys Prometheus node-exporter. To have node-exporters functioning properly, Calisti also deploys the following SCC bindings for node-exporters:

node-exporter SCC binding for privileged accesses to the node resources
hostwork SCC binding to allow node-exporter having configuration hostNetwork=true

For more details regarding SCC policies, see Important OpenShift changes to Pod Security Standards.

Calisti Prometheus OpenShift integration

Calisti leverages its own Prometheus chart to provide the functionality to monitor all kinds of different metrics. Given that the memory requirement varies in the OpenShift environment, the memory limits of k8sproxy container deployed by the Prometheus operator are increased when installing Calisti on OpenShift.

resources:
      limits:
        cpu: 100m
        memory: 1000Mi
      requests:
        cpu: 50m
        memory: 500Mi