OpenAI-Compatible APIs for Self-Hosted and Open Models

“OpenAI-compatible” should be tested as a list of endpoint and schema contracts. Similar messages JSON can simplify an adapter, but it does not establish SDK drop-in compatibility, tools, streaming or identical response objects. Hostnot GPU's current public inference interface is a concrete example of why that distinction matters.

Compare paths, not just base URLs

The documented Hostnot GPU base is https://serverless.hostnotgpu.ae/v1, with model discovery at GET /models and chat generation at POST /inference/chat. Do not assume that changing an OpenAI SDK's base URL will rewrite its chat-completions or Responses API path. Either implement the documented HTTP request directly or write an explicit adapter whose path and shape are tested.

Discovery requires pricing:read; generation requires inference:write. Send the public model ID from discovery, a server-side bearer key and an Idempotency-Key for generation. Unknown models are a configuration error, not a reason to guess an upstream name. See the public API guide.

Intersect the accepted request fields

Chat accepts model, messages, max_tokens, optional temperature and optional stream: false. Messages contain system, user or assistant roles and string content. Strict validation rejects unknown fields: tool calls, response-format schemas, multimodal content arrays or other vendor-specific options cannot be presumed supported.

There can be at most 100 messages, each no longer than 50,000 characters. The output limit is 1–8192 tokens, default 1024; temperature is 0–2 when provided. These request bounds do not enlarge a model's context window. Catalog modality and context metadata are not feature flags for tools or streaming.

Translate successful responses deliberately

Normalized chat returns an inference.result containing output.type: "text" and output.text. It also supplies usage, billing and latency fields. An adapter that reads choices[0].message.content will miss this public result. Image output has a different normalized structure; video currently returns a different response path and should not share an assumed universal decoder.

Define your internal result type and validate before conversion. Preserve money as decimal strings and request IDs as diagnostic metadata. Keep model text untrusted even when the HTTP request succeeds; a prompt asking for JSON does not guarantee schema-valid output.

Audit retries and errors before adopting an SDK

Many clients retry automatically. Check which statuses, how many attempts, whether headers survive and whether one logical operation keeps its key. Hostnot GPU uses the key for operation references but does not promise cached-response replay for every inference retry. A disconnect can leave delivery uncertain, so disable blind duplicate submissions and reconcile first.

Fixture tests should cover the real success envelope, strict unknown-field rejection, no streaming, 429 with missing Retry-After, non-JSON gateway failures and a timeout after delivery. Use the error decision tree and the non-streaming UI guide.

Keep model formatting distinct from HTTP compatibility

For self-hosted open models, the tokenizer's chat template still matters even behind a familiar API. Hugging Face's chat-template guide explains why control tokens differ across instruction-tuned models. Correct transport cannot repair a wrong template, mismatched tokenizer or incompatible model artifact.