How To Install HAProxy for Kubernetes Load Balancer

When you run Kubernetes in the cloud, the provider hands you a load balancer whenever you need one. On-premise, you get nothing: if you want a highly available control plane with three master nodes, something in front of them must accept traffic on one stable address and spread it across the masters. That something is exactly what HAProxy does well.

This tutorial shows you how to build a highly available layer 4 load balancer for a Kubernetes cluster using HAProxy and keepalived. Two small VMs share a single virtual IP: one actively forwards traffic to your master nodes, and if it dies, the other takes over the IP within a couple of seconds. Your kubectl, your worker nodes, and your users only ever see one address.

This guide is for sysadmins and DevOps engineers deploying Kubernetes on bare metal or VMs. The same setup also works for K3s, OKD, and any other distribution that exposes an API server on a TCP port.

How the Pieces Fit Together

Three components are involved, and it helps to be clear about the job of each:

HAProxy is the load balancer. We run it in layer 4 (TCP) mode, which means it forwards raw TCP connections without looking inside them. This matters for the Kubernetes API server: the traffic is TLS with client certificate authentication, and the API server itself must see and validate those certificates. A layer 7 (HTTP) proxy would have to terminate TLS and would break that. In TCP mode, HAProxy is invisible to the TLS handshake.

keepalived provides the high availability. It implements VRRP (Virtual Router Redundancy Protocol): both load balancer VMs agree on a shared virtual IP (VIP), the one with the higher priority holds it, and the other continuously listens for VRRP advertisements. If the active node stops advertising (machine down) or its HAProxy process dies (detected by a tracking script), the backup claims the VIP by sending a gratuitous ARP, and traffic follows.

Health checks are what make this a real load balancer rather than a dumb packet forwarder. HAProxy probes each master node’s API port every few seconds and stops sending traffic to any master that fails the check, so a dead master never receives connections.

The target architecture:

                          VIP: 10.0.0.10
                         _______|_______
                        |               |
                      lb-1            lb-2
                   10.0.0.11       10.0.0.12
                  (HAProxy +      (HAProxy +
                   keepalived)     keepalived)
                        |               |
        ________________|_______________|________________
       |                        |                        |
   master-1                 master-2                 master-3
   10.0.0.20:6443           10.0.0.21:6443           10.0.0.22:6443

Everything (kubectl, worker nodes, kubeadm join) points at 10.0.0.10:6443.

Prerequisites

Two VMs for the load balancers running Ubuntu 20.04, 22.04, or 24.04 (1 CPU and 1 GB RAM each is plenty for a small cluster)
One or more Kubernetes master nodes, or a plan to build them (if you have not built the control plane yet, do this load balancer first, then follow How to Install Kubernetes with a Single Master using the VIP as the control plane endpoint)
One free IP address in the same subnet as the load balancers and masters, to use as the VIP
A user with sudo privileges on both load balancer VMs
Basic Linux command line skills

The IP plan used throughout this tutorial:

lb-1:     10.0.0.11/24
lb-2:     10.0.0.12/24
vip:      10.0.0.10/24   (currently unused address in the subnet)
master-1: 10.0.0.20/24
master-2: 10.0.0.21/24
master-3: 10.0.0.22/24

The VIP does not belong to any machine’s netplan configuration. It is not configured anywhere except inside keepalived, which assigns it dynamically to whichever node is active. Just pick an address in the subnet that nothing else uses and, if you run DHCP, exclude it from the pool.

Step 1: Install HAProxy and keepalived

On both load balancer VMs:

sudo apt update
sudo apt install haproxy keepalived -y

Check the HAProxy version, anything 2.x or newer is fine:

haproxy -v

HAProxy version 2.8.5-1ubuntu3 2024/04/01 - https://haproxy.org/

Step 2: Configure HAProxy

The HAProxy configuration is identical on both VMs, which is one of the nice properties of this design: the load balancers are interchangeable.

Open the config file:

sudo nano /etc/haproxy/haproxy.cfg

Keep the existing global and defaults sections that ship with the package, and append this at the bottom:

frontend k8s-master-frontend
    bind *:6443
    mode tcp
    option tcplog
    default_backend k8s-master-backend

backend k8s-master-backend
    mode tcp
    option tcp-check
    balance roundrobin
    default-server inter 10s downinter 5s rise 2 fall 2
    server master-1 10.0.0.20:6443 check
    server master-2 10.0.0.21:6443 check
    server master-3 10.0.0.22:6443 check

Let’s walk through what each part does, because this is the heart of the setup:

frontend defines where HAProxy listens: every address on the VM (*), port 6443, the standard Kubernetes API port. Binding to * rather than a specific IP is what lets the same config work on both VMs and on the VIP, wherever it currently lives.
mode tcp puts both sections in layer 4 mode. Do not use mode http here, the API server traffic is TLS end to end.
option tcplog logs connections with useful TCP-level details (source, backend chosen, timings) to /var/log/haproxy.log.
option tcp-check enables active health checking by opening a TCP connection to each server’s port 6443.
balance roundrobin rotates new connections across healthy masters. balance leastconn is a good alternative that favors the master with the fewest open connections, useful because API server connections (watches) are long-lived.
default-server inter 10s downinter 5s rise 2 fall 2 sets the health check rhythm: probe every 10 seconds while up, every 5 seconds while down, require 2 consecutive successes to mark a server up and 2 consecutive failures to mark it down.
Each server line is one master node, with check enabling the health check for it.

If you have only one master today, list just that one server. You can add the others later with a reload and no downtime.

Validate and restart:

sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl restart haproxy
sudo systemctl enable haproxy

The -c flag checks the configuration file without starting anything, the same habit as nginx -t.

Repeat this step on the second VM.

Step 3: Configure keepalived on lb-1 (MASTER)

Create the keepalived configuration on lb-1:

sudo nano /etc/keepalived/keepalived.conf

vrrp_script check_haproxy {
    script "killall -0 haproxy"
    interval 2
    weight -101
}

vrrp_instance haproxy-vip {
    interface ens18
    state MASTER
    priority 200
    virtual_router_id 101

    virtual_ipaddress {
        10.0.0.10/24
    }

    track_script {
        check_haproxy
    }
}

What this means:

vrrp_script check_haproxy runs killall -0 haproxy every 2 seconds. The -0 signal does not kill anything, it only tests whether a process named haproxy exists. If the check fails, weight -101 subtracts 101 from this node’s priority, dropping it below the backup so the VIP moves. This is the crucial part: without it, keepalived would happily hold the VIP on a node whose HAProxy has crashed.
interface ens18 is the network interface that owns the subnet. Check yours with ip link show and adjust, it may be ens3, enp0s3, or similar.
state MASTER and priority 200 make this node the preferred holder of the VIP.
virtual_router_id 101 identifies this VRRP group. Both load balancers must use the same value, and it must not collide with any other VRRP group in the same network segment.
virtual_ipaddress is the VIP that will float between the two nodes.

Start it:

sudo systemctl restart keepalived
sudo systemctl enable keepalived

Step 4: Configure keepalived on lb-2 (BACKUP)

Same file on lb-2, with only two differences: state and priority.

vrrp_script check_haproxy {
    script "killall -0 haproxy"
    interval 2
    weight -101
}

vrrp_instance haproxy-vip {
    interface ens18
    state BACKUP
    priority 100
    virtual_router_id 101

    virtual_ipaddress {
        10.0.0.10/24
    }

    track_script {
        check_haproxy
    }
}

sudo systemctl restart keepalived
sudo systemctl enable keepalived

If you want a deeper understanding of VRRP states, priorities, and preemption behavior, I wrote a dedicated tutorial: Install and Configure keepalived on Ubuntu.

Step 5: Verify the Setup

First, confirm the VIP lives on lb-1:

ip addr show ens18

2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 ...
    inet 10.0.0.11/24 brd 10.0.0.255 scope global ens18
    inet 10.0.0.10/24 scope global secondary ens18

The VIP appears as a secondary address. On lb-2, the same command should show only 10.0.0.12.

Next, test that the API server answers through the VIP (from any machine in the subnet):

curl -k https://10.0.0.10:6443/version

{
  "major": "1",
  "minor": "33",
  "gitVersion": "v1.33.2",
  ...
}

The -k flag skips certificate validation, which is fine for this connectivity test. Getting a JSON version response means the whole chain works: VIP, HAProxy, health-checked backend, API server.

Now test the failover. Stop HAProxy on lb-1:

sudo systemctl stop haproxy

Within a few seconds, the tracking script fails, lb-1 drops its priority, and lb-2 claims the VIP. Run ip addr show ens18 on lb-2 to confirm, then run the curl again: it should still work. Start HAProxy back up on lb-1 and the VIP returns to it (because lb-1 has the higher base priority and preemption is on by default).

Finally, watch the health checks do their job. Shut down one master node and check the HAProxy log on the active load balancer:

sudo tail -f /var/log/haproxy.log

[WARNING] Server k8s-master-backend/master-2 is DOWN, reason: Layer4 timeout ...

New connections now go only to the remaining healthy masters, and kubectl keeps working without you touching anything.

Using the VIP in Your Cluster

For a new cluster, initialize the control plane with the VIP as its endpoint so every kubeconfig and join command uses it:

sudo kubeadm init --control-plane-endpoint "10.0.0.10:6443" --upload-certs

Worker and additional master nodes then join through the VIP as well, which I cover in How to Join a Node to a Kubernetes Cluster.

One thing this load balancer does not do is expose your applications. That is a separate concern handled inside the cluster, either by MetalLB for LoadBalancer services or by an Nginx Ingress Controller for HTTP traffic. You can even add another HAProxy frontend on ports 80/443 pointing at your ingress NodePorts, using exactly the same pattern you just learned.

Common Mistakes and Troubleshooting

Both nodes hold the VIP at the same time (split brain). The two keepalived instances cannot hear each other. Usual causes: a firewall blocking VRRP (IP protocol 112, multicast address 224.0.0.18), or mismatched virtual_router_id values. If you use UFW, allow VRRP between the two load balancers.

HAProxy fails to start with “cannot bind socket”. Something else is listening on 6443 on the load balancer VM, or you are trying to run the load balancer on a master node itself. Check with sudo ss -tlnp | grep 6443. Co-locating HAProxy on the masters requires a different port for the frontend (for example 8443), it cannot share 6443 with the local API server.

All backends marked DOWN. From the load balancer, test reachability directly: curl -k https://10.0.0.20:6443/version. If that fails, the problem is network or firewall between load balancer and masters, not HAProxy.

Failover works, failback does not. Check that lb-1’s priority is genuinely higher and that HAProxy is actually running again. The weight -101 must be large enough that a failed check drops the MASTER below the BACKUP; with priorities 200 and 100, a subtraction of 101 does exactly that.

kubectl hangs after failover. Long-lived watch connections were attached to the failed node and take a moment to time out and reconnect. This is normal and self-heals; new connections work immediately.

Best Practices

Keep the HAProxy config identical on both nodes, and update both every time you add or remove a master. Configuration drift between the pair is the most common operational failure of this design.
Use balance leastconn for API traffic once your cluster gets busy. API server connections are long-lived, so connection counts matter more than request counts.
Monitor the pair. Enable the HAProxy stats socket or scrape it with Prometheus, and alert on backend DOWN events, not just on total outage. See Setup Prometheus and Grafana on Ubuntu for the monitoring stack.
Test failover on purpose, regularly. A high availability setup you have never failed over is a hope, not a design.
Restrict who can reach port 6443 on the VIP with firewall rules. The Kubernetes API is a sensitive surface; only admins and cluster nodes need it.

Conclusion

You built a highly available entry point for your Kubernetes control plane: two HAProxy nodes in TCP mode with active health checks against your masters, and keepalived floating a single virtual IP between them. You verified both failure modes, a dead load balancer and a dead master, and neither one interrupts kubectl.

From here, complete the picture for application traffic with MetalLB and the Nginx Ingress Controller, or grow the control plane itself by joining more nodes to the cluster.