How to Build a Kubernetes Cluster on Bare Metal Ubuntu 24.04

Learn how to deploy a multi-node Kubernetes cluster on Ubuntu 24.04 LTS bare-metal servers using containerd and Cilium CNI. Step-by-step guide.

Deploying Kubernetes on bare metal gives you total control over your hardware, network, and storage. However, without a cloud provider's managed control plane, the responsibility for security and stability falls entirely on you.

Many online guides rely on outdated repositories or encourage terrible security practices (like turning off your firewall). In this tutorial, we will build a secure, stable, multi-node Kubernetes v1.31 cluster on Ubuntu 24.04 LTS using containerd and the eBPF-powered Cilium Container Network Interface (CNI).

Tutorial Contents

Prerequisites

Phase 1: Prepare the Nodes & Harden Security

Phase 2: Install and Secure containerd

Phase 3: Install Kubernetes Components

Phase 4: Cluster Initialization & Worker Nodes

Prerequisites

Before installing Kubernetes, ensure your bare-metal servers meet the following requirements:

Hardware & OS

  • At least two bare-metal servers (1 Control Plane, 1+ Worker Nodes)
  • Fresh installation of Ubuntu 24.04 LTS
  • Control Plane Min: 2 CPUs and 4GB RAM
  • Worker Node Min: 2 CPUs and 2GB RAM

Network & Privileges

  • Root or sudo access on all nodes
  • Static Private IPs assigned to all nodes
  • Unique hostnames across the cluster

Phase 1: Prepare the Nodes & Harden Security

Note: Perform these steps on ALL nodes (Control Plane and Workers).

1
Configure Hostnames and Hosts File

Ensure each node has a unique hostname and can resolve the others.

BASH 
# On the Control Plane node:
sudo hostnamectl set-hostname k8s-control-plane

# On the Worker node:
sudo hostnamectl set-hostname k8s-worker-01

Edit /etc/hosts on all nodes to include the IPs and hostnames of your cluster:

PLAINTEXT 
192.168.1.10 k8s-control-plane
192.168.1.11 k8s-worker-01

2
Disable Swap

Kubernetes requires swap to be disabled to ensure the kubelet can properly enforce resource limits.

BASH 
sudo swapoff -a

To make this permanent, comment out the swap entry in your filesystem table:

BASH 
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

3
Load Required Kernel Modules

containerd and Kubernetes require specific kernel modules for networking.

BASH 
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

4
Configure Sysctl for Kubernetes Networking

Ensure IP forwarding and iptables bridge traffic are enabled.

BASH 
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

5
Configure the Firewall (UFW)

Security Best Practice
Do not disable your firewall. Instead, open only the ports required for Kubernetes and the Cilium overlay network to function, and explicitly allow Pod routing to avoid the "UFW Routing Trap."

On the Control Plane Node:

BASH 
sudo ufw allow 6443/tcp      # Kubernetes API server
sudo ufw allow 2379:2380/tcp # etcd server client API
sudo ufw allow 10250/tcp     # Kubelet API
sudo ufw allow 10259/tcp     # kube-scheduler
sudo ufw allow 10257/tcp     # kube-controller-manager
sudo ufw allow 4240/tcp      # Cilium health checks
sudo ufw allow 8472/udp      # Cilium VXLAN overlay network

# Allow Pod-to-Pod routed traffic (Prevents the UFW Routing Trap)
sudo ufw route allow from 10.244.0.0/16

sudo ufw enable

On the Worker Node(s):

BASH 
sudo ufw allow 10250/tcp       # Kubelet API
sudo ufw allow 30000:32767/tcp # NodePort Services
sudo ufw allow 4240/tcp        # Cilium health checks
sudo ufw allow 8472/udp        # Cilium VXLAN overlay network

# Allow Pod-to-Pod routed traffic
sudo ufw route allow from 10.244.0.0/16

sudo ufw enable

Phase 2: Install and Secure containerd

Note: Perform these steps on ALL nodes.

6
Install containerd.io from Docker Repository

BASH 
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg lsb-release

# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add the repository to Apt sources
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install -y containerd.io

7
Configure the Systemd Cgroup Driver

CRITICAL WARNING
We must configure containerd to use systemd. Failing to do this causes the kubelet and containerd to use two different cgroup managers, which will fight for system resources and crash your node under pressure.
BASH 
# Generate the default configuration
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml > /dev/null

# Edit the file to set SystemdCgroup = true
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

# Restart and enable containerd
sudo systemctl restart containerd
sudo systemctl enable containerd

Phase 3: Install Kubernetes Components

8
Install Kubelet, Kubeadm, and Kubectl

Note: Perform these steps on ALL nodes.

Using the modern community-owned pkgs.k8s.io repositories for Kubernetes v1.31:

BASH 
sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl

# Lock the versions to prevent unattended upgrades from breaking your cluster:
sudo apt-mark hold kubelet kubeadm kubectl

Phase 4: Cluster Initialization & Worker Nodes

9
Initialize the Control Plane

Note: Perform this step ONLY on the Control Plane node.

Initialize the cluster. Be sure to replace <YOUR_PRIVATE_IP> with your server's actual internal IP address to avoid the "IP Roulette" risk of exposing your API server to the public internet.

BASH 
sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=<YOUR_PRIVATE_IP>

Once the initialization completes, save the kubeadm join command output; you will need it for your worker nodes.

Configure kubectl access for your regular user:

BASH 
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

10
Install the Cilium CNI

Note: Perform this step ONLY on the Control Plane node.

Install the Cilium CLI:

BASH 
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz

Install Cilium into the cluster:

BASH 
cilium install

Wait a few moments, then verify the installation:

BASH 
cilium status
kubectl get pods -n kube-system

Ensure the CoreDNS pods transition from Pending to Running.

11
Join the Worker Nodes

Note: Perform this step ONLY on your Worker Node(s).

Paste the kubeadm join command you saved from Step 9. It will look similar to this:

BASH 
sudo kubeadm join <YOUR_PRIVATE_IP>:6443 --token <your-token> \
        --discovery-token-ca-cert-hash sha256:<your-hash>

Back on your Control Plane, verify that the nodes have joined and are in the Ready state:

BASH 
kubectl get nodes

Congratulations!

You now have a secure, bare-metal Kubernetes cluster running with a properly configured containerd runtime and a robust eBPF-based networking layer.

Ready to Scale Your Infrastructure?

Building a bare-metal Kubernetes cluster from scratch requires reliable hardware and absolute network stability. At Leo Servers, we provide high-performance, strictly isolated bare-metal servers perfect for your containerized workloads.

Stop wrestling with noisy neighbors and cloud provider lock-in. Explore Leo Servers Bare Metal Solutions Today and deploy your Kubernetes clusters on hardware that works as hard as you do.

Explore Dedicated Servers →

Discover Leo Servers High-Performance Locations

Leo Servers operates premium bare-metal environments worldwide, offering diverse hosting options. Check out our specialized offerings to choose the setup that best suits your intensive workload needs.