Share One IP Across Servers with Keepalived

As you might have picked up, I run baremetal servers. They provide the best performance-to-cost ratio. A problem I’ve been trying to solve for some time now has been to securely expose a performant service, which is load-balanced across a bunch of servers.

I first tried using Cloudflare Tunnels, a tunnel per server. These tunnels then get load balanced using Cloudflare Loadbalancer. Turns out, it’s quite slow. Tunnels just aren’t fast. Passing any production traffic via them just doesn’t make the cut.

I then exposed the servers with their public IP addresses, putting them behind a Cloudflare Loadbalancer. This was faster than using tunnels but still fell short of expectations. A simple ping from a Digital Ocean droplet to my servers takes 100-200 ms. Pinging the server directly only takes 20ms.

Surely, I could directly alias a subdomain over to the server’s public IP address, but the challenge is — How do you alias one IP address when you have multiple NGINX servers running?

Meet Keepalived

Keepalived can be used to assign a single IP address across servers. If all servers are healthy, Keepalived would assign this IP address to the highest priority server. But in case that server crashes or stops responding, Keepalived would automatically reassign that IP address to another healthy server.

That is a very useful property. With Keepalived, you can expose multiple servers all behind a virtual load balancer of sorts to ensure you achieve high-availability.

This is how you can set this up:

sudo apt update
sudo apt install keepalived
sudo ifconfig # Or, sudo ip addr to get the primary network interface name.

# Modify and add the following config to /etc/keepalived/keepalived.conf.
vrrp_instance virt0 {
    state MASTER

    # Your network interface. Can be a bonded interface too.
    interface eth0

    virtual_router_id 101
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
      <IP-ADDR-TO-ASSIGN> label eth0:v0
    }
}
sudo vim /etc/keepalived/keepalived.conf

sudo service keepalived restart

# This should now expose a new virtual interface, with the IP address.

sudo ifconfig eth0:v0
eth0:v0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet xxxxxxxxxxxxxx  netmask 255.255.255.255  broadcast 0.0.0.0
        ether xxxxxxxxxxxxxxxxx  txqueuelen 1000  (Ethernet)
        device memory 0x51100000-511fffff

# Remember to set the firewall to allow connections between servers.
sudo ufw allow from xxx.xxx.xxx.xxx/xx

Remember to update:

• the interface name
• the priority. Set the highest priority to the primary server.
• the IP address that you want to use and expose.

Once the first server is set up, you can then just re-do these operations on the other server(s).

Once you do, you should see something like this:

Nov 19 06:33:48 xxxxxxxxxxxx Keepalived[373008]: Starting VRRP child process, pid=373009
Nov 19 06:33:48 xxxxxxxxxxxx Keepalived[373008]: Startup complete
Nov 19 06:33:48 xxxxxxxxxxxx Keepalived_vrrp[373009]: (virt0) Entering BACKUP STATE (init)
Nov 19 06:33:51 xxxxxxxxxxxx Keepalived_vrrp[373009]: (virt0) Entering MASTER STATE
Nov 19 06:47:56 xxxxxxxxxxxx Keepalived_vrrp[373009]: (virt0) Master received advert from xxxxxxxxxxxxxx with higher priority 101, ours 100
Nov 19 06:47:56 xxxxxxxxxxxx Keepalived_vrrp[373009]: (virt0) Entering BACKUP STATE

Congrats! You now have one IP address to expose to the world.

If you want to learn more about setting up baremetal servers, see my posts with baremetal tag.

---

---

---

---