Installing PyTorch with CUDA on a GPU Instance

Install PyTorch from the official selector, verify the driver and CUDA runtime, and diagnose common import and device errors.

By Hostnot GPU · 9/6/2026

A working CUDA PyTorch installation requires both a compatible host driver and a GPU-enabled framework build. Installing a Python wheel does not update the host kernel driver. Start from the actual image and GPU, then select the package combination; do not copy a CUDA version from an unrelated tutorial.

Inventory the target environment

nvidia-smi -L
nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv
python3 --version
uname -m
df -h .

Confirm OS, CPU architecture, Python support, GPU architecture, and the driver's compatibility with the proposed wheel. The CUDA value in nvidia-smi is driver support, not the installed compiler version. If the host utility fails, fix allocation/driver visibility through the owner first. In a container, ensure the GPU is exposed before installing Python packages.

Select a release deliberately

Use PyTorch's official install selector for a supported release, or its previous-version matrix when reproducing a known workload. Choose a CUDA wheel index documented for that exact release. Check GPU architecture support too: a wheel built before a GPU generation may not contain its kernels. Record the full command and source index. A version from a historical model tutorial is not automatically appropriate for newly released hardware.

Create a new environment in a writable project directory. If venv support is missing, request the matching distribution package through the supported image workflow; do not use system-wide sudo pip.

python3 -m venv .venv-torch
. .venv-torch/bin/activate
python -m pip --version
# Set these to the version and CUDA index from the same official release entry.
: "${TORCH_VERSION:?Set the selected exact PyTorch version}"
: "${TORCH_INDEX_URL:?Set its official download.pytorch.org CUDA wheel index}"
python -m pip install "torch==$TORCH_VERSION" --index-url "$TORCH_INDEX_URL"
python -m pip check

Install torchvision or torchaudio only if the application needs them, using the matching versions in the same official release matrix. Prebuilt CUDA wheels provide their runtime dependencies; a full CUDA Toolkit is normally needed only for documented compilation requirements. Do not preinstall an unrelated CUDA stack into this environment.

Verify actual device execution

Save this as torch_smoke.py and run python torch_smoke.py inside the activated environment. It checks a tiny matrix product rather than just returning an availability flag.

import sys
import torch

print("python:", sys.executable)
print("torch:", torch.__version__, "build CUDA:", torch.version.cuda)
assert torch.cuda.is_available(), "No usable CUDA device"
print("device:", torch.cuda.get_device_name(0))
print("capability:", torch.cuda.get_device_capability(0))
print("build architectures:", torch.cuda.get_arch_list())
with torch.inference_mode():
    x = torch.ones((32, 32), device="cuda")
    y = x @ x
    torch.cuda.synchronize()
    assert torch.allclose(y, torch.full_like(y, 32))
print("CUDA arithmetic passed")

The expected final line is CUDA arithmetic passed. This establishes basic execution, not performance, BF16/FP8 support, or compatibility with every extension. Next test one real application step with its intended dtype and memory demand.

Resolve common failures by layer

Import missing: compare sys.executable, the activated environment, and notebook kernel. Restart/select the correct notebook kernel after installation.
Build CUDA is None: inspect whether a CPU wheel was selected. Recreate a clean environment with the intended official index.
CUDA unavailable with a GPU wheel: inspect allocation, container GPU flags, visibility variables, and driver compatibility.
No kernel image / invalid device function: check GPU architecture support for both PyTorch and custom compiled extensions.
Undefined symbol: investigate mixed native packages; a clean environment is safer than random library symlinks.

Record and reuse the accepted environment

After the real smoke test, python -m pip freeze records resolved versions. Review its output for private package URLs before saving or sharing; it is a dependency snapshot, not a complete cross-platform reproducibility lock. Retain Python/OS, wheel source, driver, GPU, and image digest alongside it. Preparing this before a long rental reduces idle setup cost. Save results and release unused compute through instance controls.

Next: DDP setup or driver/toolkit compatibility.