Multi-cluster upgrade from 1.10.0 to 1.11.0

This document shows you how to upgrade Service Mesh Manager in a multi-cluster mesh scenario. For details on how to set up a multi-cluster mesh, see the multi-cluster installation guide. To access the latest binary files, see Accessing the Service Mesh Manager binaries.

Upgrade from 1.10.0 to 1.11.0

To upgrade Service Mesh Manager from 1.10.0 to 1.11.0 for a multi-cluster setup, complete the following steps.

Before upgrading Service Mesh Manager 1.10 to 1.11, apply the following patch to your Service Mesh Manager v1.10 cluster to modify the spec field of a job that cleans up the cert-manager-startupapicheck job after 100sec when completed. If you skip this step, you might see a “cert-manager-startupapicheck” related error during the upgrade. The error is non-blocking and doesn’t stop the upgrade process. Alternatively, you can apply the patch after you have upgraded the cluster.
```
kubectl patch jobs.batch -n cert-manager cert-manager-startupapicheck -p '{"spec":{"ttlSecondsAfterFinished":100}}' --type=merge
```
Download the Service Mesh Manager command-line tool for version 1.11.0. The archive contains the smm and supertubes binaries. Extract these binaries and update your local copy on your machine. For details, see Accessing the Service Mesh Manager binaries.
```
smm --version
```
The output should be similar to:
```
Service Mesh Manager CLI version  1.11.0 (6ba681d83) built on 2022-11-15T21:43:27Z
```
Deploy a new version of Service Mesh Manager.

The following command upgrades the Service Mesh Manager control plane. It also installs the new Istio control plane (version 1.15.x), but the applications keep using the old control plane until you restart your workloads.

In the following examples, smm refers to version 1.11.0 of the binary.
- If you want to upgrade only Service Mesh Manager:
```
smm install -a
```
  - In case you want to have custom settings for your Istio control plane, you can provide that during the installation:
```
smm install -a --istio-cr-file <custom-istio-cr-file.yaml>
```
Restart the Prometheus instance so that it starts using the cp-v115x.istio-system Istio control plane:
```
kubectl rollout restart statefulset prometheus-smm-prometheus --namespace smm-system
```
Rerun the attach command with --force flag to upgrade Service Mesh Manager on the peer cluster:
```
smm istio cluster attach <PEER_CLUSTER_KUBECONFIG_FILE> --force
```

Select the 1.15.x version Istio control plane as that’s the Istio version supported by Service Mesh Manager version 1.11.0:

✓ validate-kubeconfig ❯ checking cluster reachability...
✓ validate-version ❯ checking K8s API server version...
✓ validate-version ❯ detected K8s API server version: 1.21.14
Multiple Istio control planes were found. Which one would you like this cluster to attach to?
?  cp-v115x

The upgrade process is completed.

Note: If you see the following error message, rerun the attach command:

could not apply k8s resources: could not update resource: Internal error occurred: failed calling webhook "cluster-validator.clusterregistry.k8s.cisco.com": Post "https://cluster-registry-controller.cluster-registry.svc:443/validate-cluster?timeout=30s": x509: certificate is not valid for any names, but wanted to matchcluster-registry-controller.cluster-registry.svc

Upgrade existing workloads

Now the Service Mesh Manager control plane is upgraded and is using the new -1.15.x Istio control plane. But, the workloads are still using the old Istio control plane and data plane. Complete the following steps to upgrade the Istio sidecar proxy in your application pods.

On your primary Service Mesh Manager cluster, add the cp-v115x.istio-system label to your application namespaces (for example, the smm-demo namespace). This label is automatically synchronized to the peer clusters.
```
kubectl label ns <name-of-your-namespace> istio.io/rev=cp-v115x.istio-system --overwrite
```
Run a kubectl rollout command on the primary and peer clusters to ensure that the pods use the latest Istio sidecar proxy:
```
# Run this command on ALL clusters
kubectl rollout restart deployment --namespace <name-of-your-namespace> 
```
To verify whether the upgrade process has succeeded on peer clusters, you can check the workload pods and see if they’re using the 1.15.x Istio proxies.