Setting up Kubernetes Ingress

The Ingress resource serves to expose single- or multi-node deployments in Kubernetes, acting as a reverse proxy to one or more Service resources.

This guide will introduce you to some scenarios for setting up Ingress, assuming you’ve installed Document Engine with Helm into a namespace named document-engine and your Helm values file is named document-engine.values.yaml.

Ingress-nginx

Ingress-nginx is the most common ingress controller.

To expose Document Engine at http://de.example.com, set the /ingress section of document-engine.values.yaml in the following way:

---
ingress:
  enabled: true
  className: nginx
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/proxy-body-size: "16m"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "180"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "180"
    nginx.ingress.kubernetes.io/large-client-header-buffers: "8 16k"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "128k"
  hosts:
    - host: de.example.com
      paths:
        - path: /
          pathType: ImplementationSpecific

Note that de.example.com must be resolved by DNS to the address Ingress is responding at. On most platforms, this implies a CNAME record. To determine the hostname it has to point to, use the following command:

kubectl get ingress -n document-engine \
  document-engine \
  -o=jsonpath='{.status.loadBalancer.ingress}'

It’ll give an output similar to the following:

[{"hostname":"k8s-ingressn-ingressn-7531d67379.amazonaws.com"}]

Ingress-nginx with HTTPS

If you have a TLS certificate for de.example.com with the following code, de.example.com.key is your private key, and de.example.com.cert is the certificate file, and both should be in PEM format:

kubectl create secret -n document-engine \
  tls de-ingress-tls \
  --key de.example.com.key --cert de.example.com.cert

Incorporating the secret into the Ingress definition is done by adding the /ingress/tls section to the values file:

---
ingress:
  enabled: true
  className: nginx
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/proxy-body-size: "16m"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "180"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "180"
    nginx.ingress.kubernetes.io/large-client-header-buffers: "8 16k"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "128k"
  hosts:
    - host: de.example.com
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls:
    - hosts:
        - de.example.com
      secretName: de-ingress-tls

Automatic TLS certificates with Ingress-nginx and cert-manager

A more sustainable approach than that of manual secret creation is automatic TLS certificate management.

If you have cert-manager installed in the cluster with a global issuer named my-tls-issuer, secrets will be created and rotated automatically by cert-manager.

To enable this functionality, use the cert-manager.io/issuer annotation:

ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: my-tls-issuer
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/proxy-body-size: "16m"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "180"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "180"
    nginx.ingress.kubernetes.io/large-client-header-buffers: "8 16k"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "128k"
  hosts:
    - host: de.example.com
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls:
    - hosts:
        - de.example.com
      secretName: de-ingress-tls

AWS ALB integration

For deployments on AWS, you can integrate Document Engine with an AWS Application Load Balancer (ALB) using the AWS Load Balancer Controller.

When using an Application Load Balancer in front of Document Engine, it needs to have the pod lifecycle aligned with Document Engine.

Specifically:

A pod needs to stay alive longer than the ALB Target Group deregistration delay. This can be achieved using lifecycle and terminationGracePeriodSeconds values.
As in any other case for Document Engine, all internal timeouts (e.g. config.requestTimeoutSeconds) should be smaller than terminationGracePeriodSeconds.
As is common for ALB, the ALB load balancer idle timeout should be greater than the target group deregistration delay.

Here’s an example of a configuration subset from your document-engine.values.yaml file to use with the AWS Load Balancer Controller, passing platform service parameters as ingress annotations:

# Application-specific timeouts
config:
  requestTimeoutSeconds: 180
  urlFetchTimeoutSeconds: 120
  generationTimeoutSeconds: 120
  workerTimeoutSeconds: 150 # Must be less than terminationGracePeriodSeconds
  readAnnotationBatchTimeoutSeconds: 120

# Pod lifecycle settings
terminationGracePeriodSeconds: 330 # Total time Kubernetes allows for graceful shutdown

lifecycle:
  preStop:
    exec:
      command:
        # This sleep should be aligned with ALB's deregistration_delay.timeout_seconds (e.g., 300s)
        # and must be less than terminationGracePeriodSeconds.
        - sleep
        - "305"

ingress:
  enabled: true
  className: alb
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/healthcheck-path: /healthcheck
    alb.ingress.kubernetes.io/success-codes: "200"
    alb.ingress.kubernetes.io/healthy-threshold-count: "2"
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: "5"
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "2"
    alb.ingress.kubernetes.io/load-balancer-attributes: >-
      routing.http2.enabled=true,
      idle_timeout.timeout_seconds=600,
      routing.http.desync_mitigation_mode=defensive
    alb.ingress.kubernetes.io/target-group-attributes: >-
      deregistration_delay.timeout_seconds=300,
      load_balancing.algorithm.type=least_outstanding_requests,
      load_balancing.algorithm.anomaly_mitigation=off