Open-Source LLM Models: Size, Capabilities, and GPU Matching

Select an open model as a versioned software and data artifact, not as a parameter-count ranking. Publicly downloadable weights can carry different licenses, use restrictions and runtime requirements. “Open weights” alone does not establish unrestricted redistribution, commercial permission or reproducible training.

Create an artifact record before testing

Record the publisher, repository, exact revision, license file, model card, tokenizer revision, chat template and weight representation. A community quantization, adapter or merged checkpoint is a distinct artifact that needs its own provenance and compatibility review. The Hugging Face model-card documentation describes intended use, limitations, training and evaluation information, plus metadata linking derivatives to base models.

Read the actual license and associated conditions, not only a repository tag. Check whether your intended distribution, hosted access and use case are covered. Keep the decision with the artifact record so a later model upgrade does not silently inherit an earlier release's approval.

Match the model type to the task

A base causal model and an instruction-tuned chat model are not interchangeable prompt targets. Embeddings need an embedding-specific interface and evaluation, while image or multimodal work needs the appropriate input pipeline. A familiar family name may span many of these variants. Avoid assuming that a larger model is more useful for a constrained extraction task.

For self-hosted chat, apply the selected tokenizer's documented template. The chat-template guide explains the role delimiters and generation prompt. If formatting text before tokenizing separately, avoid duplicating special tokens. Incorrect delimiters can produce poor output even when the model loads and the GPU is healthy.

Budget the complete working set

Estimate weight memory from the actual representation, then add quantization metadata, KV cache, temporary buffers and concurrency headroom. A mixture-of-experts model's active-parameter count is not automatically the total weight residency requirement. CPU offload and quantization can change latency and supported kernels, so treat them as measured tradeoffs.

Use VRAM planning and quantization guidance to turn the artifact into a hardware requirement. Confirm the precise GPU and serving framework support before selecting an offer; this article makes no unmeasured throughput or memory-fit guarantee for a named model.

Evaluate quality and operations together

• Build representative tasks with expected outputs or an explicit scoring rubric, including difficult and long-input cases.
• Track task accuracy, output-format validity and required human corrections, not just plausible-looking prose.
• Measure full-response latency, useful throughput, memory and failures at target context and concurrency.
• Repeat after changing model revision, template, quantization or runtime; change one factor at a time.

Use licensed, appropriately de-identified evaluation data. Treat downloaded model code and extensions as executable software requiring review; isolate tests and avoid exposing credentials to untrusted loading paths.

Choose a delivery path that exists

Hostnot GPU serverless exposes its current public catalog, not arbitrary model uploads or every capability in a model card. Check live discovery and the strict public schema. For a custom artifact, use a compatible dedicated environment and own the serving, access control and storage lifecycle. Preserve a known-good artifact and evaluation record for rollback.