Deploying and Using Envoy in Kubernetes: A Step-by-Step Guide // First Mission

🚪 Introduction

Envoy is a popular open-source edge and service proxy that provides advanced load balancing, routing, and observability features. If you have ever used a service mesh like Istio or Linkerd, you have already met Envoy, since it is the data plane underneath several of them: the smart proxy that sits next to every service and handles all the traffic in and out. Kubernetes is a container orchestration platform that simplifies the deployment and management of containerized applications.

Deploying Envoy in Kubernetes can provide a highly scalable and reliable service mesh infrastructure for your microservices-based application. In this tutorial, we will go through the process of deploying and using Envoy in Kubernetes, from a bare deployment all the way to routing traffic between real services.

✅ Prerequisites

Before we begin, ensure that you have the following prerequisites:

A Kubernetes cluster up and running
kubectl command-line tool installed and configured
Basic knowledge of Kubernetes concepts such as pods, deployments, and services
Basic knowledge of Envoy concepts such as listeners, clusters, and routes

If any of the Kubernetes concepts are fuzzy, do not worry: you can follow along even with a local cluster such as kind or minikube.

📦 Deploying Envoy in Kubernetes

The easiest way to deploy Envoy in Kubernetes is to use the official Envoy Docker image and deploy it as a Kubernetes deployment. Here are the steps to do this:

1. Create the Deployment Manifest

Create a file named envoy-deployment.yaml with the following contents:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: envoy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: envoy
  template:
    metadata:
      labels:
        app: envoy
    spec:
      containers:
      - name: envoy
        image: envoyproxy/envoy:v1.18.3
        ports:
        - containerPort: 8080
        - containerPort: 9901

This YAML file defines a Kubernetes deployment that will create a single replica of the Envoy container. The two ports matter: 8080 is where Envoy listens for traffic, and 9901 is its admin endpoint, where you can query stats and dump configuration.

2. Apply the Deployment

Deploy the Envoy deployment by running the following command:

kubectl apply -f envoy-deployment.yaml

This command will create the Envoy deployment in your Kubernetes cluster.

3. Create the Service Manifest

Create a file named envoy-service.yaml with the following contents:

apiVersion: v1
kind: Service
metadata:
  name: envoy
spec:
  selector:
    app: envoy
  ports:
  - name: http
    port: 80
    targetPort: 8080
  - name: admin
    port: 9901
    targetPort: 9901

This YAML file defines a Kubernetes service that will expose the Envoy container to the outside world. Note how the service maps port 80 to Envoy’s 8080, while exposing the admin port under its own name, so both are reachable without confusion.

4. Apply the Service

Deploy the Envoy service by running the following command:

kubectl apply -f envoy-service.yaml

This command will create the Envoy service in your Kubernetes cluster.

🔀 Using Envoy in Kubernetes

Now that Envoy is deployed in your Kubernetes cluster, you can start using it to route traffic to your microservices. Here are the steps to do this:

1. Create the Envoy Configuration

Create a file named envoy-config.yaml with the following contents:

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        address: [IP_ADDRESS]
        port_value: 80
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/service-a"
                route:
                  cluster: service-a
              - match:
                  prefix: "/service-b"
                route:
                  cluster: service-b

The envoy-config.yaml file defines the Envoy listener and the routing configuration. In this example, we have defined a listener on port 80 that will forward traffic to two microservices named service-a and service-b. Requests starting with /service-a go to the first cluster, and requests under /service-b go to the second, Envoy’s prefix matching does the routing for us.

2. Deploy the Configuration

Deploy the Envoy configuration by running the following command:

kubectl create configmap envoy-config --from-file=envoy-config.yaml

This command will create a Kubernetes configmap with the Envoy configuration. Using a ConfigMap means you can change Envoy’s routing rules and roll them out without rebuilding the container image.

3. Mount the Configuration

Modify the envoy-deployment.yaml file to mount the Envoy configuration as a volume in the container. Update the file with the following lines:

spec:
  containers:
  - name: envoy
    image: envoyproxy/envoy:v1.18.3
    ports:
    - containerPort: 8080
    - containerPort: 9901
    volumeMounts:
    - name: config-volume
      mountPath: /etc/envoy
  volumes:
  - name: config-volume
    configMap:
      name: envoy-config

This will mount the Envoy configuration as a volume in the container at /etc/envoy, which is exactly where the Envoy image expects to find its configuration file.

4. Apply the Updated Deployment

Deploy the updated Envoy deployment by running the following command:

kubectl apply -f envoy-deployment.yaml

This will update the Envoy deployment to mount the Envoy configuration as a volume.

5. Verify Everything Works

Verify that Envoy is running correctly by accessing the Envoy admin endpoint at http://<envoy-service-ip>:9901/stats. If you see a wall of counters and gauges, the proxy is alive, configured, and ready to route.

💡 Best Practices

Here are some best practices to keep in mind when deploying and using Envoy in Kubernetes:

Use a separate Envoy container for each microservice. This provides better isolation and makes it easier to scale the microservices independently.
Use Kubernetes ConfigMaps to manage the Envoy configuration. This makes it easier to update the configuration without redeploying the Envoy container.
Use Kubernetes Services to expose the Envoy container to the outside world. This provides load balancing and high availability for the Envoy container.
Use Kubernetes Ingress resources to define the Envoy listener and routing configuration. This provides a more flexible and scalable way to manage the Envoy configuration.

🧠 Final Thoughts

Deploying and using Envoy in Kubernetes can provide a highly scalable and reliable service mesh infrastructure for your microservices-based application. In this tutorial, we went through the process of deploying and using Envoy in Kubernetes, including configuring Envoy for routing traffic to microservices, and we discussed some best practices along the way.

The setup is deliberately small, but the architecture scales: swap the static YAML for dynamic discovery, add more clusters, wire in observability, and you have the same foundation that powers production meshes at massive scale. Thanks for reading!