Ollama + CUDA on Fedora: Prerequisites

The Problem

Running ollama run on a fresh Fedora install does nothing useful on the GPU — it falls back to CPU inference. The Ollama installer doesn’t set up NVIDIA drivers or CUDA for you, and on Fedora the driver installation has an extra step that’s easy to miss.

The Solution

You need working NVIDIA proprietary drivers with CUDA support before installing Ollama. On Fedora, drivers come via RPM Fusion and are built as kernel modules by akmods.

1. Enable RPM Fusion (if not already):

sudo dnf install \
  https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \
  https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

2. Install NVIDIA drivers and CUDA toolkit:

sudo dnf install akmod-nvidia xorg-x11-drv-nvidia-cuda

3. Wait for the kernel module to build — this is the step people skip:

sudo akmods --force
sudo dracut --force

4. Reboot:

sudo reboot

5. Verify CUDA is working:

nvidia-smi

You should see your GPU listed with driver version and CUDA version. If nvidia-smi works, the GPU is ready.

6. Install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

Ollama detects CUDA automatically at install time. Verify it’s using the GPU after running a model:

ollama run llama3
# In another terminal:
nvidia-smi  # GPU memory should be in use

An RTX 3090 with 24 GB VRAM comfortably fits most 7B–13B models at full precision, and 70B models at 4-bit quantization.