Deploy a new Alertmanager

SLO-based alerting requires you to configure Service Mesh Manager to use an Alertmanager deployment. This procedure describes how to deploy a new Alertmanager for Service Mesh Manager.

Note: If you want to use an existing Alertmanager deployment instead, see Use an existing Alertmanager.

Prerequisites

In this article an example configuration is provided that is suitable for use with Slack alerts. To configure this notification, a Slack account and a Slack Incoming Webhook is required.

To configure additional notification targets, see the Prometheus Alertmanager documentation.

Steps

Set up Alertmanager. You can find a basic alerting setup in this example. Download the file and replace the %SLACK_API_URL% string with your Slack Incoming Webhook URL.

Apply the resulting file by running this command:

kubectl apply -f alertmanager.yaml -n smm-system

Verify that Alertmanager is running. Run the following command:

kubectl get pods -n smm-system -l app=alertmanager
NAME                                    READY   STATUS    RESTARTS   AGE
alertmanager-smm-alertmanager-0   3/3     Running   0          62s
alertmanager-smm-alertmanager-1   3/3     Running   0          62s

You should see two instances of Alertmanager in Running state.

Configure Prometheus to use this Alertmanager. Service Mesh Manager is controlled by a ControlPlane custom resource found in the Service Mesh Manager’s namespace (default: smm-system) named smm.

The following command changes the spec.smm.prometheus.alertmanager value to connect to the Alertmanagers started by the YAMLs specified in the previous steps. Run the following commands:

cat > enable-new-alertmanager.yaml <<EOF
spec:
  smm:
    prometheus:
      enabled: true
      alertmanager:
      static:
        - host: alertmanager-smm-alertmanager-0.alertmanager-operated.smm-system.svc.cluster.local
          port: 9093
        - host: alertmanager-smm-alertmanager-1.alertmanager-operated.smm-system.svc.cluster.local
          port: 9093
EOF

kubectl patch controlplane --type=merge --patch "$(cat enable-new-alertmanager.yaml)" smm

If you are using Service Mesh Manager in operator mode, skip this step.

Otherwise, execute a reconciliation so Service Mesh Manager updates your Kubernetes cluster to the desired state described by the ControlPlane Custom Resource. Run the following command:
```
smm operator reconcile
```

Example Alertmanager configuration

The following is a basic alerting setup for Alertmanager. Download the file and replace the %SLACK_API_URL% string with the your Slack Incoming Webhook URL.