Tensor Parallelism Explained for Multi-GPU Inference
Learn what tensor parallelism splits, when it helps memory fit, and why topology and communication matter.
Hostnot GPU engineering
GPU, AI, and infrastructure guides
Practical notes for choosing hardware, connecting safely, and building dependable AI workloads. Every guide reflects what the platform actually does today.
Showing 1–9 of 9 published articles
Open-Source LLM Models: Size, Capabilities, and GPU Matching
A practical model-selection framework covering license, capability, memory, context, serving support, and evaluation.
Mistral 7B Local Inference: GPU Requirements
Plan a Mistral 7B deployment by checkpoint, precision, context, KV cache, runtime support, and test workload.
LLM Inference Quantization: 4-bit and 8-bit Guide
Understand weight quantization, formats, accuracy tradeoffs, KV cache memory, and how to validate a quantized deployment.
LLaMA 2 Local Inference (Historical): Memory and Runtime Planning
A historical, reproducible planning guide for Llama 2 inference, with explicit license, memory, context, and runtime caveats.
A100 40GB vs A100 80GB: What's the Difference?
Why the memory capacity in an A100 configuration changes model fit, context headroom, and concurrency planning.
How to Choose a GPU for LLM Inference
A repeatable selection process for model fit, context policy, concurrency, framework support, and cost control.
How Much GPU VRAM Do You Need for LLM Inference?
A memory-planning checklist for weights, quantization, context windows, KV cache, and concurrency.
NVIDIA H100 vs H200: Which GPU Should You Choose?
A workload-first framework for choosing between accelerator classes with different memory and platform characteristics.