Set up Persistent Volumes for Prometheus
Each Prometheus instance needs a persistent volume (PV) to store the recorded metrics.
To set up such persistent volumes, modify the Service Mesh Manager control plane custom resource to allow for acquiring such volumes. Complete the following steps.
Find which storage classes does your Kubernetes cluster support. Log in to your cluster, then run the following command:
kubectl get storageclass (kubernetes-admin@turip-host/smm-system) NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 7h17m
Select the appropriate storage class from the output list.
The previous example shows the default AWS settings for EKS or PKE clusters, in this case it is safe to use gp2 volumes.
Set up the Persistent Volume.
To enable Persistent Volumes for Prometheus, first customize this YAML patch file:
Save this patch file as
Enter the Storage Class name obtained in the previous step into the
storageClassNameproperty, and set the storage space required by updating the value of
Service Mesh Manager is controlled by a
ControlPlanecustom resource found in the Service Mesh Manager’s namespace (default:
smm. To update the Custom Resource with the patch, run the following command.
kubectl patch controlplane --type=merge --patch "$(cat persistent-storage.patch.yaml)" smm controlplane.smm.cisco.com/smm patched
If you are using Service Mesh Manager in operator mode, skip this step.
Otherwise, execute a reconciliation so Service Mesh Manager updates your Kubernetes cluster to the desired state described by the ControlPlane Custom Resource. Run the following command:
smm operator reconcile
Verify that Persistent Volumes exist. To verify that Prometheus has the physical volumes attached, run the following command:
kubectl get pv -n smm-system NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-79cc1a86-bcfa-46a9-a0e7-24e38d001d29 50Gi RWO Delete Bound smm-system/prometheus-smm-prometheus-db-prometheus-smm-prometheus-0 gp2 7m10s
The list shows that a Physical Volume (named
smm-system/prometheus-smm-prometheus-db-prometheus-smm-prometheus-0in the example) has been created and attached (
If you haven’t done yet, Set up High Availability for the Monitoring Stack.