If you run Kubernetes on bare metal or on a VM instead of a managed cloud, you have probably met the classic puzzle: the cluster works perfectly, but nothing from the outside can reach it, and the culprit is almost always the host firewall. Firewalld and Kubernetes do not always see eye to eye, since Kubernetes expects to manage its own networking while the firewall is busy enforcing its own rules.
Kubernetes is an open-source container orchestration platform that provides a way to deploy, manage, and scale containerized applications. It makes use of a networking component called Ingress to allow external traffic to reach the services running inside the Kubernetes cluster. Ingress routes traffic from the outside world to services running within the Kubernetes cluster. To allow Ingress traffic while keeping Firewalld enabled, follow the steps below. The good news: you do not have to disable the firewall, which would be the easy and wrong answer.
1. 📃 Enable Deny Logging in Firewalld
The first step is to enable deny logging in Firewalld. This step is important because it helps you to monitor all the denied connections in the system, so you can see exactly what the firewall is blocking and why. To do this, run the command below:
sudo firewall-cmd --set-log-denied=allWith logging on, the next steps become visible: every dropped packet shows up in the logs, and you can confirm each fix as you make it.
2. 🔌 Create a New Zone Containing All the Kubernetes veth Interfaces
The second step is to create a new zone that contains all the Kubernetes Virtual Ethernet (veth) interfaces. These interfaces are used to connect the Kubernetes pods to the Kubernetes network. To create a new zone containing all the veth interfaces, run the commands below:
sudo firewall-cmd --permanent --new-zone=k8s-veth
sudo firewall-cmd --reload
sudo firewall-cmd --permanent --zone=k8s-veth --add-interface=<veth1> --add-interface=<veth2> <...>Note: Replace <veth1>, <veth2>, and <...> with the actual names of the veth interfaces on your system. You can find them with ip link and look for the interfaces whose names look like veth....
3. 🔗 Create a New Zone Containing the CNI Interface of Kubernetes
The third step is to create a new zone that contains the Container Network Interface (CNI) interface of Kubernetes. This interface is used to manage the network of Kubernetes containers. To create a new zone containing the CNI interface, run the commands below:
sudo firewall-cmd --permanent --new-zone=k8s-cni
sudo firewall-cmd --reload
sudo firewall-cmd --permanent --zone=k8s-cni --add-interface=<cni0>Note: Replace <cni0> with the actual name of the CNI interface on your system, usually cni0 or cbr0 depending on your CNI plugin.
4. ⏩ Allow Forward to the Zone k8s-cni
The fourth step is to allow forward to the zone k8s-cni. This step is important because it allows traffic to be forwarded from the Kubernetes pods to the Kubernetes network. To allow forward to the k8s-cni zone, run the command below:
sudo firewall-cmd --permanent --zone=k8s-cni --add-forwardWithout forwarding, even a perfectly opened port is useless, since the firewall will refuse to route packets between the pod network and the outside world.
5. 🛡️ Create a New Policy Allowing Ingress from the k8s-cni Zone to the k8s-veth Zone
The fifth step is to create a new policy that allows ingress from the k8s-cni zone to the k8s-veth zone. This policy is important because it allows traffic to flow from the Kubernetes network to the Kubernetes pods. To create a new policy that allows ingress from the k8s-cni zone to the k8s-veth zone, run the commands below:
sudo firewall-cmd --permanent --new-policy=k8s-forward-network
sudo firewall-cmd --permanent --policy=k8s-forward-network --add-ingress-zone=k8s-cni --add-egress-zone=k8s-veth
sudo firewall-cmd --permanent --policy=k8s-forward-network --set-target=ACCEPT
sudo firewall-cmd --permanent --policy=k8s-forward-network --add-service=http --add-service=https
sudo firewall-cmd --reloadThe policy ties the two zones together: traffic entering through the CNI zone is allowed to reach the veth zone, and only HTTP and HTTPS services are let through. Everything else stays blocked, which is exactly the security posture you want.
6. 📜 Monitor All the Logs
The sixth step is to monitor all the logs using the command below:
sudo journalctl -xefThis command will display all the logs in real-time, including the allowed and denied connections. Watch the denial lines as you test, they will tell you instantly whether a rule is missing or misconfigured.
7. 🧪 Test the Configuration
The seventh step is to test the configuration and check for any issues with the added policies. You can do this by trying to access the Kubernetes service from a different network. If the connection is successful, then the configuration is correct. Try both HTTP and HTTPS if you have TLS set up, and check the logs for any new denial messages.
8. 🧹 Disable the Firewalld Deny Logging
The final step is to disable the Firewalld deny logging to reduce the log size. To do this, run the command below:
sudo firewall-cmd --set-log-denied=offLeave it on while you are debugging, and turn it off once everything is confirmed working, so your logs do not fill up with routine noise.
🧠 Final Thoughts
In conclusion, allowing Kubernetes ingress traffic with Firewalld enabled requires creating new zones for the Kubernetes veth and CNI interfaces, allowing forward to the CNI zone, creating a policy that allows ingress from the CNI zone to the veth zone, and monitoring all the logs. By following these steps, you can ensure that your Kubernetes cluster is secure while allowing external traffic to access the services running inside the cluster.
The beauty of this setup is that the firewall stays fully armed for everything else. You are not punching a hole in the wall; you are building a door for the traffic that is supposed to get through. Thanks for reading!