Istio resources
When adding an external workload to the mesh, there are two crucial Istio resources that are used.
WorkloadGroup
needs to be created in the namespace where the machine will be attached to. This object represents a group of machines serving the same service. This is analogous to the Kubernetes concept of aDeployment
.- Each virtual machine attached to the mesh will be represented by a
WorkloadEntry
object in the workload’s namespace. This is analogous to thePod
concept of Kubernetes.
The VM attachment flow used in Service Mesh Manager relies on the PILOT_ENABLE_WORKLOAD_ENTRY_AUTOREGISTRATION
and PILOT_ENABLE_WORKLOAD_ENTRY_HEALTHCHECKS
features.
Autoregistration
To understand the autoregistration feature first take a look at a WorkloadGroup
resource:
apiVersion: networking.istio.io/v1alpha3
kind: WorkloadGroup
metadata:
labels:
app: analytics
version: v1
name: analytics-v1
namespace: smm-demo
spec:
metadata:
labels:
app: analytics
version: v1
template:
ports:
http: 8080
serviceAccount: default
If autoregistration is enabled, the Istio pilot-agent running on the virtual machine connects to the istio-meshexpansion-gateway
in the istio-system
namespace and presents the specified ServiceAccount
’s bearer token (and some registration details that Service Mesh Manager sets automatically) to authenticate itself to the Istio control plane. If the authentication is successful the Istio control plane creates a WorkloadEntry
in the cluster, like this:
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
annotations:
istio.io/autoRegistrationGroup: analytics-v1
istio.io/connectedAt: "2022-03-31T06:52:14.739292073Z"
istio.io/workloadController: istiod-cp-v115x-df9f5d556-9kvqs
labels:
app: analytics
hostname: ip-172-31-22-226
istio.io/rev: cp-v115x.istio-system
service.istio.io/canonical-name: analytics
service.istio.io/canonical-revision: v1
topology.istio.io/network: vm-network-1
name: analytics-v1-3.67.91.181-vm-network-1
namespace: smm-demo
ownerReferences:
- apiVersion: networking.istio.io/v1alpha3
controller: true
kind: WorkloadGroup
name: analytics-v1
uid: d01777d5-4294-44e7-a311-3596c2f63bb1
spec:
address: 1.2.3.4
labels:
app: analytics
hostname: ip-172-31-22-226
istio.io/rev: cp-v115x.istio-system
service.istio.io/canonical-name: analytics
service.istio.io/canonical-revision: v1
topology.istio.io/network: vm-network-1
locality: eu-central-1/eu-central-1a
network: vm-network-1
serviceAccount: default
Any attached machine that has a corresponding WorkloadEntry
resource behaves as a Kubernetes workload, and has a set of labels assigned that could be used by Services
to match the machine.
For example, the following Service
will route traffic to the virtual machine due to the .spec.selector
matching the WorkloadEntry
’s labels (.metadata.labels
):
apiVersion: v1
kind: Service
metadata:
name: analytics
namespace: smm-demo
spec:
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: analytics
sessionAffinity: None
type: ClusterIP
Autoregistration is also crucial when removing Workloads from the mesh. In case the istio sidecar process is stopped on the host, the Istio control plane automatically removes the related WorkloadEntry
custom resource. This can be used to temporarily remove a VM for maintenance or troubleshooting purposes from the mesh, but it also ensures that if Istio is uninstalled from the node, it automatically de-registers itself without needing to manually update any Kubernetes resources.
Health checks
The PILOT_ENABLE_WORKLOAD_ENTRY_HEALTHCHECKS
setting provided by Istio allows health checks to be defined for VMs. If the health check fails, Istio will not route any traffic to the workload.
In case of Service Mesh Manager, the health checks are defined in the WorkloadGroup
resource and our agent running on the VM ensures that Istio uses that setting. For example, the following WorkloadGroup
defines an HTTP health check:
apiVersion: networking.istio.io/v1alpha3
kind: WorkloadGroup
metadata:
labels:
app: analytics
version: v1
name: analytics-v1
namespace: smm-demo
spec:
metadata:
labels:
app: analytics
version: v1
probe:
httpGet:
host: 127.0.0.1
path: /
port: 8080
scheme: HTTP
template:
network: vm-network-1
serviceAccount: default
The .spec.probe
’s definition is the same as the Probe
object of the official Kubernetes API. The defined probe is analogous to the liveliness probe of the Pod
: it is checked constantly while Istio is running on the machine. The only difference is that Istio will not restart the VM if the probe fails, instead it stops routing any traffic to the WorkloadEntry
.
You can query the status of the health checks from Kubernetes by checking the machine’s WorkloadEntry
:
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
annotations:
istio.io/autoRegistrationGroup: analytics-v1
istio.io/connectedAt: "2022-03-31T06:52:14.739292073Z"
istio.io/workloadController: istiod-cp-v115x-df9f5d556-9kvqs
labels:
app: analytics
hostname: ip-172-31-22-226
istio.io/rev: cp-v115x.istio-system
service.istio.io/canonical-name: analytics
service.istio.io/canonical-revision: v1
topology.istio.io/network: vm-network-1
name: analytics-v1-3.67.91.181-vm-network-1
namespace: smm-demo
ownerReferences:
- apiVersion: networking.istio.io/v1alpha3
controller: true
kind: WorkloadGroup
name: analytics-v1
uid: d01777d5-4294-44e7-a311-3596c2f63bb1
spec:
address: 1.2.3.4
labels:
app: analytics
hostname: ip-172-31-22-226
istio.io/rev: cp-v115x.istio-system
service.istio.io/canonical-name: analytics
service.istio.io/canonical-revision: v1
topology.istio.io/network: vm-network-1
locality: eu-central-1/eu-central-1a
network: vm-network-1
serviceAccount: default
status:
conditions:
- lastProbeTime: "2022-03-31T07:23:07.236758604Z"
lastTransitionTime: "2022-03-31T07:23:07.236759090Z"
status: "True"
type: Healthy
In the status
field of the custom resource, the conditions
array contains an entry with the type
field set to Healthy
. If the same objects status
is set to True
, then the machine is considered healthy and will receive traffic.