How to Install NVIDIA CUDA Toolkit on Ubuntu 24.04

Tutorial Contents

Prerequisites

Before installing CUDA, verify your dedicated GPU server meets the necessary software and hardware requirements:

Verify Your GPU Model

Before proceeding, verify that your system recognizes the installed NVIDIA GPU using the PCI lookup tool:

lspci | grep -i nvidia

Expected Output: You should see your specific GPU model listed (e.g., NVIDIA Corporation AD102 [GeForce RTX 4090]).

Phase 1: System Preparation

1

Update Your OS

It is crucial to start with a fully updated system to prevent dependency conflicts during installation:

sudo apt update
sudo apt upgrade -y

2

Install Essential Build Tools

The CUDA installer relies on GNU Compiler Collection (GCC) and other tools to compile drivers and libraries. Install them via apt:

sudo apt install build-essential dkms software-properties-common -y

3

Remove Old NVIDIA/CUDA Installations

If you have previously attempted an installation or upgraded from an older Ubuntu version, purge old packages to avoid conflicts:

sudo apt purge "nvidia-*" "cuda-*" -y
sudo apt autoremove -y

Phase 2: Install NVIDIA Drivers

4

Identify Recommended Drivers

Ubuntu includes a handy utility to identify the best proprietary drivers for your exact GPU hardware.

ubuntu-drivers devices

Look for the line that says recommended next to a driver version (e.g., nvidia-driver-550 - distro non-free recommended).

5

Install the Driver

You can let Ubuntu automatically install the recommended version:

sudo ubuntu-drivers autoinstall

Alternatively, if you need a specific version, install it manually:

sudo apt install nvidia-driver-550 -y

6

Reboot System

Drivers interact directly with the kernel. A system reboot is required for the new drivers to initialize.

sudo reboot

7

Verify Driver Installation

Once back online, run the NVIDIA System Management Interface to confirm the drivers are active:

nvidia-smi

You should see a table displaying your GPU name, memory usage, and driver version. Note the CUDA Version shown in the top right—this indicates the maximum CUDA toolkit version your driver supports.

Phase 3: Install CUDA Toolkit

8

Add NVIDIA Developer Repository

While Ubuntu maintains its own CUDA packages, installing directly from NVIDIA's repository ensures you get the latest features and toolkit versions tailored for Ubuntu 24.04.

Fetch the repository pin and GPG key:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb

9

Install the CUDA Toolkit

Update your apt cache to register the new NVIDIA repository:

sudo apt update

To install the absolute latest CUDA toolkit available in the repository:

sudo apt install cuda-toolkit -y

This process will download several gigabytes of data and may take some time depending on your server's connection speed.

Phase 4: Environment Setup & Verification

10

Update Environment Variables

For your system and compilers to locate the newly installed CUDA libraries, you must add them to your shell's environment variables.

Open your bash profile (or zshrc if you use Zsh):

nano ~/.bashrc

Scroll to the bottom of the file and paste the following lines (assuming you installed CUDA 12; adjust the path if necessary):

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Save and exit (CTRL+X, Y, Enter).

11

Reload the Environment

Apply the changes immediately to your current session:

source ~/.bashrc

12

Verify CUDA Compiler

Finally, confirm that the CUDA compiler (nvcc) is active and returns the correct version:

nvcc --version

Expected Output: It will output the Cuda compilation tools, release details, and build version. Congratulations, CUDA is now fully installed!

Best Practices for AI & ML Environments

To maximize the performance and stability of your deep learning server, consider the following best practices:

• cuDNN Installation: If you are running deep neural networks, install NVIDIA cuDNN right after CUDA. It contains highly optimized routines for standard deep learning operations.
• Python Virtual Environments: Avoid installing pip packages globally. Use Conda, venv, or Docker containers to isolate your PyTorch or TensorFlow projects and avoid library conflicts.
• Docker Integration: Install the nvidia-container-toolkit to easily pass GPU access directly into Docker containers. This is heavily utilized in enterprise AI environments.
• Temperature Monitoring: Under heavy ML loads, GPUs run hot. Regularly check thermals using watch -n 1 nvidia-smi.
• Version Pinning: When configuring production environments, lock your apt repository versions so an accidental `apt upgrade` doesn't break framework dependencies.