Tensor Parallelism Explained for Multi-GPU Inference
Learn what tensor parallelism splits, when it helps memory fit, and why topology and communication matter.
Hostnot GPU engineering
GPU, AI, and infrastructure guides
Practical notes for choosing hardware, connecting safely, and building dependable AI workloads. Every guide reflects what the platform actually does today.
Showing 1–5 of 5 published articles
FP16 vs BF16 vs FP8 for AI Workloads
Choose a reduced-precision format by hardware support, model stability, framework support, and measured workload behavior.
GPU Performance Degradation Troubleshooting
A measurement-first workflow for slower inference, lower utilization, memory pressure, and changing throughput.
GPU Benchmarking Basics: Synthetic and Workload Benchmarks
Design repeatable GPU benchmarks that separate hardware capability from model, framework, input, and operational effects.
Multi-GPU Setup with PyTorch: Data Parallel and Distributed
Understand when to use one process per GPU, how to launch distributed jobs, and how to verify that work is actually parallelized.