Let's Encrypt support for gateways

Service Mesh Manager provides a streamlined way to use Let’s Encrypt with Istio-native Gateways. Compared to relying on Istio’s Ingress, this solution gives you greater flexibility when defining your own Ingress gateway.

Background

Cert Manager supports Let’s Encrypt’s http01 challenge mechanism for verifying the authenticity of a service via HTTP requests, but it relies on Kubernetes Ingress to ensure that the http01’s required verification endpoints are working as expected.
Istio provides a Kubernetes Ingress implementation IngressClass, but it is more limited than the Gateway resource.

Service Mesh Manager solves this issue by allowing Cert Manager to issue TLS certificates directly to an Istio Gateway resource.

Prerequisites

To set up TLS certificates based on Let’s Encrypt for an Istio Gateway, the following requirements must be met:

cert-manager must be enabled for the deployment. Verify that the cert-manager namespace exits.
The deployment must be able to allocate a LoadBalancer type service, in addition to the existing Gateway
The deployment must expose an http endpoint on port 80
The DNS must be set up properly to point to the existing MeshGateway’s Service (we recommend using external-dns).

For example, if you have a namespace called my-app, where the traffic is exposed over TCP port 80 using http with the following MeshGateway and Gateway setup:

apiVersion: servicemesh.cisco.com/v1alpha1
kind: IstioMeshGateway
metadata:
    labels:
        app.kubernetes.io/instance: app
        app.kubernetes.io/name: app-ingress
    name: app-ingress
    namespace: my-app
spec:
    istioControlPlane:
        name: cp-v115x
        namespace: istio-system
    deployment:
       metadata:
          labels:
              app.kubernetes.io/instance: app
              app.kubernetes.io/name: app-ingress
              gateway-name: app-ingress
              gateway-type: ingress
       replicas:
          min: 1
          max: 1
          count: 1
    service:
      metadata:
        annotations:
          external-dns.alpha.kubernetes.io/hostname: my-app.example.org
      ports:
      - name: http2
        port: 80
        protocol: TCP
        targetPort: 8080
      type: LoadBalancer
    runAsRoot: true
    type: ingress
---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  labels:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: app-ingress
  name: app-ingress
  namespace: my-app
spec:
  selector:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: app-ingress
  servers:
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP

Note: The VirtualService and other Istio resources are omitted from this example, but those are also required for the application to work properly.

Add TLS endpoints to the Gateway

Reconfigure the Gateway and the MeshGateway to support SSL.

Add a new port needs to the MeshGateway resource to accept incoming traffic on port 443 (see the .spec.ports.[name=https] part):

apiVersion: servicemesh.cisco.com/v1alpha1
kind: IstioMeshGateway
metadata:
  labels:
     app.kubernetes.io/instance: app
     app.kubernetes.io/name: app-ingress
  name: app-ingress
  namespace: my-app
spec:
  istioControlPlane:
     name: cp-v115x
     namespace: istio-system
  deployment:
     metadata:
        labels:
           app.kubernetes.io/instance: app
           app.kubernetes.io/name: app-ingress
           gateway-name: app-ingress
           gateway-type: ingress
     replicas:
        min: 1
        max: 1
        count: 1
  service:
     metadata:
        annotations:
           external-dns.alpha.kubernetes.io/hostname: my-app.example.org
     ports:
     - name: http2
       port: 80
       protocol: TCP
       targetPort: 8080
     - name: https
       port: 443
       protocol: TCP
       targetPort: 8443
     type: LoadBalancer
  runAsRoot: true
  type: ingress

Configure the Gateway to accept HTTPS traffic. You have to:

Instruct Istio to redirect any non-https traffic to the https endpoint (for details, see Known limitations):
```
    tls:
      httpsRedirect: true
```

And set up a new https endpoint:

  - hosts:
    - '*'
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      credentialName: app-ingressgateway-tls
      mode: SIMPLE

The modified Gateway resource should look something like this:

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  labels:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: app-ingress
  name: app-ingress
  namespace: my-app
spec:
  selector:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: app-ingress
  servers:
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP
    tls:
      httpsRedirect: true
  - hosts:
    - '*'
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      credentialName: app-ingressgateway-tls
      mode: SIMPLE

Note: The secret in the example does not exist yet. This does not cause a configuration issue, but the TLS endpoint is not yet operational.

Configure cert-manager to provide TLS certificates

Configure Cert Manager to issue the TLS credentials into the app-ingressgateway-tls Secret.

Create an Issuer that represents a connection to the Let’s Encrypt service:

cat > issuer.yaml <<EOF
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  labels:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: app-issuer
  name: app-issuer
  namespace: my-app
spec:
  acme:
    email: noreply@cisco.com
    preferredChain: ""
    privateKeySecretRef:
      name: app-letsencrypt-issuer
    server: https://acme-v02.api.letsencrypt.org/directory
    solvers:
    - http01:
        ingress:
          class: nginx
EOF
kubectl apply -f issuer.yaml

Note: You can also use a cluster-wide issuer called ClusterIssuer if multiple Certificates are required for multiple namespaces.

Check that the Issuer is working properly. Run the kubectl get issuer command. The Issuer should be in ready state:
```
NAME          READY   AGE
app-issuer    True    19h
```

Create the Certificate object that instructs Cert Manager to maintain a certificate for a given domain name, and automatically refresh it if needed. You can use the following example, but note the following points:

The dnsNames section must contain all the host names for which TLS certificates must be present in the resulting secret. For each domain name listed here, there must be a properly set up DNS record pointing to the LoadBalancer’s external IP for the MeshGateway.
```
  dnsNames:
  - my-app.example.org
```
For Service Mesh Manager to know which Gateway to attach this certificate to, the following annotation must be put on the Certificate. This is a label matcher, that the Let’s Encrypt Operator uses to find the Gateways needed to be configured for TLS.
```
metadata:
  annotations:
    acme.smm.cisco.com/gateway-selector: |
      {
        "app.kubernetes.io/instance": "app",
        "app.kubernetes.io/name": "app-ingress"
      }      
```
The secretName field must match the value of the previously configured TLS secret in the Gateway resource.

The following yaml includes all three requirements:

cat > certificate.yaml <<EOF
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  annotations:
    acme.smm.cisco.com/gateway-selector: |
      {
        "app.kubernetes.io/instance": "app",
        "app.kubernetes.io/name": "app-ingress"
      }
  labels:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: app-certificate
  name: app-certificate
  namespace: my-app
spec:
  dnsNames:
  - my-app.example.org
  duration: 2160h0m0s
  issuerRef:
    group: cert-manager.io
    kind: Issuer
    name: app-issuer
  privateKey:
    algorithm: RSA
    encoding: PKCS1
    size: 2048
  renewBefore: 360h0m0s
  secretName: app-ingressgateway-tls
  usages:
  - server auth
  - client auth
EOF
kubectl apply -f issuer.yaml

Verify that the Certificate has been issued, by running the kubectl get certificate command. The secret should be in ready state:

NAME                          READY   SECRET                   AGE
app-certificate               True    app-ingressgateway-tls   19h

Known limitations

Istio’s http to https redirection allows for little configurability. As a result, the Let’s Encrypt challenge verification fails in the following case:

The gateway has named hosts (for example, '*' is not in the hosts list of the Gateway resource) AND
Automatic redirection to HTTPS is enabled in the tls settings of the http endpoint

In this case, try to use one of the following workarounds:

Use a Gateway that has '*' specified in its hosts list (this might mean that you need external load balancers).
Use the Kubernetes Ingress implementation IngressClass of Istio.
Disable the automatic https redirection and use application logic to do the redirect.