Local Ollama or serverless inference for background agents?
For queued agent work, choose between local control and hosted inference by measuring utilization, end-to-end latency and the cost of operating the runtime.
A single-instance DDP run can avoid multi-host networking, but GPU fit, SSH setup and launch details still decide whether training scales cleanly.
Distributed Data Parallel (DDP) is a practical way to fine-tune a model across several GPUs on one cloud instance. It starts a process for each GPU, splits training data across processes and synchronizes gradients. That can increase throughput when the model, batch size and GPU memory fit the chosen instance. It can also add communication overhead, so more GPUs do not guarantee a faster run.
Hostnot GPU offers on-demand GPU instances, with live capacity shown across 18 GPU types, 8 regions and 80 catalog offerings. The catalog is the place to confirm what is currently available. Do not assume that a GPU name means a particular number of devices, memory size or network arrangement. Choose an offering that explicitly meets the run’s GPU-count and memory needs.
Start with the simplest topology: one instance containing all the GPUs for the run. That keeps DDP’s processes on one host and avoids the extra coordination and network requirements of multi-node training. Check the offering details for GPU count, memory, region, capacity class and displayed rate. Match memory to the model, optimizer state, context or sequence length, and per-GPU batch size—not just the model’s parameter count.
If a deploy template fits the runtime you need, it can give repeated runs a more consistent starting environment. Otherwise, record the operating system, Python and PyTorch versions, CUDA-related dependencies, and data and model revisions you use. Confirm that the installed PyTorch build can use the instance’s GPUs before committing to a long training run.
Review the displayed rate and wallet authorization before requesting compute. Rates are calculated server-side, and availability can change. The available wallet balance may exclude active reservations. For a deployment pipeline, the practical checks are covered in the guide to checking wallet funds before requesting GPU capacity.
Register a public SSH key with Hostnot GPU before connecting. Keep its private key on your own device; do not copy it onto the instance or share it. After provisioning, use the displayed host, user and port, and verify the displayed host fingerprint. This gives you a direct way to install dependencies, inspect logs and manage training without exposing your private credential.
On the instance, check that the expected GPUs are visible to the runtime and that each can allocate memory. Then test the training environment with a short run. The point is to catch a mismatched package, missing data path or device error before a multi-GPU job starts consuming time and capacity.
In the training script, initialize a process group using the NCCL backend for GPU communication. Read each process’s local rank, select that GPU with the CUDA device API, and move the model to it before wrapping the model in PyTorch’s DistributedDataParallel. Use a distributed sampler for the training dataset, and set its epoch each epoch so that data shuffling stays coordinated across processes. Each process should write only the checkpoint or logs it owns, commonly the process with rank zero.
Launch with torchrun --standalone --nnodes=1 --nproc-per-node=<GPU_COUNT> train.py, replacing the placeholder with the number of GPUs assigned to the job. The process count should match the visible GPUs you intend to use. With a standalone, single-node launch, torchrun handles the local rendezvous; you do not need to set up a public rendezvous server or open ports between separate instances.
That boundary matters. A multi-node DDP job needs a reachable rendezvous address, consistent node and rank settings, and working communication between hosts. Do not treat two separately provisioned GPU instances as a cluster unless Hostnot GPU documents the required networking and connectivity for that configuration. The known product details establish on-demand instances, not a particular multi-instance network topology.
Run the same representative training step on one GPU and on the full instance. Compare examples or tokens processed per second, step time, GPU memory use and any data-loading stalls. Keep the global batch size and evaluation conditions clear when comparing results. DDP introduces gradient communication, and small workloads may spend too much time synchronizing to benefit from extra GPUs. If throughput does not improve enough to justify the added capacity, reduce the GPU count or reconsider the workload.
Save the checkpoints and other outputs you need before terminating the instance. Ending an SSH session does not stop or release an instance; use the workspace controls and check the resulting status. Review usage and transactions after the run. This single-host recipe is deliberately narrow: it gets a multi-GPU fine-tuning job running without assuming that separate instances can communicate, while leaving the actual GPU choice and rate to Hostnot GPU’s current customer-visible catalog.
For queued agent work, choose between local control and hosted inference by measuring utilization, end-to-end latency and the cost of operating the runtime.
No verified B200 or B300 rates are available here; capacity, workload fit and billing terms matter more than a headline rental price.
Give each backend only the API access it needs, and keep GPU spending under one centrally authorized wallet.