Models & routing

Every request names a model; Heimdallm picks one of the endpoints serving it. This page covers how models are identified, how endpoints are chosen, and every control you have over that choice.

Model ids#

A model is identified as author/slug, where the author is always the creator of the weights, never the party serving them. Ids are matched case-insensitively; the catalogue stores them in lower case. A modified rebuild — a fine-tune, an abliteration — is its own model with a distinguishing suffix under the original author:

deepseek/deepseek-v4-flash-0731-abliterated

— an abliterated rebuild of DeepSeek V4 Flash, listed under deepseek because DeepSeek created the base weights, whoever serves the rebuild. Two independent rebuilds of one base are two models, each with a slug naming the rebuild rather than the base: zai-org/glm-5.3-flash-abliterated-dealign beside zai-org/glm-5.3-flash-abliterated-jangq, both under the base's author and both grouped into its family. When several providers serve the same id, that is one model with several endpoints; Heimdallm routes each request to one of them. The model list shows every model with at least one live endpoint; a model page lists each endpoint with its price, context window and measured performance.

Choosing a provider in the model id#

To send a request to one provider, name the provider in place of the author:

{ "model": "impcc/minimax-m3" }

provider/slug resolves to the catalogue model that provider serves under that slug — minimaxai/minimax-m3 here — and routes only to that provider's endpoints. The response's model field names the catalogue id. Where one provider serves two models sharing a slug under different authors, the short form is refused with 400 naming both; write the author too, impcc/minimaxai/minimax-m3, which is always unambiguous. Routing suffixes apply as usual (impcc/minimax-m3:nitro), and so does every field of provider: a provider the id names but provider.ignore lists has no endpoint left to serve the request, which is 503.

A catalogue id always wins. When an author's name is also a provider's slug, author/slug is the author's model, and that provider's endpoints for it are reached with provider: { "only": [...] } or the three-part form. The same choice for every model goes in provider preferences; the model id form suits a single request, and a fallback list can mix both: "models": ["impcc/minimax-m3", "minimaxai/minimax-m3"] tries impcc first and then any provider.

Families#

Variants of one base model group into a family named after the base — the rebuild above belongs to the family deepseek/deepseek-v4-flash-0731. Families exist for browsing and filtering — a request must name a served model, not a family. The base model itself may not be served at all (true of this family: only the abliterated rebuild is live); sending such a family id fails with 404 and the error names the served variants, both in the message and in metadata.family_variants. GET /families lists every family, and GET /models?family=<id> filters the catalogue to one; see the catalogue reference.

Routing suffixes#

Append a suffix to a model id to change how the endpoint is chosen:

{ "model": "deepseek/deepseek-v4-flash-0731-abliterated:floor" }

The four supported suffixes are routing variants. :online is recognized and refused with The :online variant requires a web search plugin, which this API does not support. Other : segments remain part of the model id and must match the catalogue verbatim.

An explicit provider.sort beats the suffix's ordering. :exacto also holds every explicit optional setting to an endpoint declaring it, as require_parameters: true does. Use sort: "exacto" to request its ordering alone. Measured tool-call success sorts highest first, and unknown readings follow measured ones. Capacity, explicit provider order and the other routing constraints still apply.

Reported capacity comes first, whatever the suffix: endpoints whose provider says it has free request slots are tried before endpoints reporting nothing, and endpoints reporting themselves full are tried last — deprioritised, never excluded, so a request still routes when everything is busy. Providers report capacity in their model list and Heimdallm re-reads it every few seconds; a report older than a minute counts as none.

Within a capacity group and without a suffix, endpoints are grouped by health (healthy endpoints first) and load-balanced with a strong preference for lower prompt price — an endpoint at half the price is tried four times as often. The order is deterministic for a given request, so retrying an identical request walks the same list.

Image and video requests compare each endpoint's total reserved quote for the effective request, including its supported defaults and conditional prices. This total supplies both sort: "price" and the default price weighting. A per-image rate and a megapixel rate are compared using the requested count and size; when the request leaves the output size or duration open, the quote uses the endpoint's declared ceiling. Capacity, health and explicit provider order still apply. The final charge uses validated delivered quantities and can be lower than the reserved quote.

Speech and transcription compare a per-request total too, over the units each rail meters, and that total supplies both sort: "price" and the default price weighting. A synthesis is priced on the characters of the text it was handed, which are known in full before routing, so the figure the ranking compares is the exact charge. A transcription is priced on the reserve it is held against, since nothing knows how many seconds an upload holds until an endpoint has decoded it; the final charge follows the seconds or the tokens the endpoint reported. An endpoint publishing no rate in any unit its own rail meters is tried last rather than ranked as free.

Fallback models#

Pass models (in addition to model) to fall back across models when the first choice cannot serve — every provider down, the model withdrawn, or the request rejected upstream. Up to 20 fallbacks are accepted; they are tried in order after the primary model's endpoints are exhausted:

{
  "model": "deepseek/deepseek-v4-flash-0731-abliterated",
  "models": ["huggingfacetb/smolvlm2-2.2b-instruct"],
  "messages": [{ "role": "user", "content": "Hello" }]
}

The response's model field names the model that actually served the request, and provider names the provider. Failures that cannot succeed elsewhere — an invalid request body, a payload too large, a client-side abort — stop the walk immediately instead of burning through the list.

Reading request history#

Request history shows the model that served each response. When its reference differs from the primary requested reference, the row shows both. Recorded routing details include the full requested model list in supplied order and whether model/provider fallbacks were enabled. Open the request to inspect that list and the actual endpoint attempts. Repeated references across model and models appear once in the display and model count, preserving their first occurrence.

A model in the list need not receive a call: endpoint eligibility, routing preferences and conversation affinity can affect selection before dispatch. A different served reference does not by itself prove that an earlier model failed; routing suffixes can also make the requested and served references differ. If the full list or settings were not retained, request details state that they are unavailable.

Provider preferences#

The optional provider object narrows which endpoints may serve a request and how the eligible ones are ordered. Unknown keys inside provider are rejected with 400.

{
  "model": "deepseek/deepseek-v4-flash-0731-abliterated",
  "provider": {
    "order": ["impcc"],
    "allow_fallbacks": true,
    "require_parameters": false,
    "data_collection": "deny",
    "zdr": true,
    "only": ["impcc", "borealis"],
    "ignore": ["example-provider"],
    "quantizations": ["fp8", "bf16"],
    "sort": "throughput",
    "max_price": { "prompt": 1, "completion": 4 }
  },
  "messages": [{ "role": "user", "content": "Hello" }]
}

Defaults#

A workspace can store the same choices once instead of repeating them on every request. Open the workspace from your account page and use its Routing section for the basic provider preferences and a default suffix. Advanced partition and performance values set through the workspace API remain intact when editing those controls.

Stored defaults fill only the gaps. Precedence, highest first:

1. What the request states explicitly — a provider field the request sets wins outright.
2. The model id's suffix — :nitro, :floor or :exacto on each requested model beats a stored sort, including fallback models.
3. The workspace's stored defaults — every remaining field.

Filling is field by field, so a request that sends provider with nothing but zdr still inherits the workspace's sort, only and the rest. A stored default suffix applies to any requested model that does not carry one of its own. Keys authenticate to exactly one workspace, so the defaults in force are those of the workspace the key belongs to.

Preferences that ask for a guarantee are evidence-based: zdr: true and data_collection: "deny" exclude endpoints whose policy is unknown, not just endpoints whose policy fails. zdr: true goes one step further — a zero-data-retention declaration also has to pass marketplace review before it counts. The declared policies appear per endpoint in the endpoints listing.

Sorting across models and performance preferences#

provider.sort accepts "price", "throughput", "latency", "exacto", or an object such as { "by": "price", "partition": "none" }. The default partition is "model": each requested model's endpoints stay ahead of fallback models. "none" orders eligible endpoints across all requested models, retaining capacity and health partitions. An explicit by wins over model suffixes and workspace defaults. Without an explicit sort, global ordering uses the first resolved model's suffix, then the workspace sort, then ordinary health and price balancing.

The deprecated top-level route alias accepts "fallback" for model grouping and "sort" for global ordering. Explicit provider.sort.partition wins over route, which wins over the workspace partition. Other route values receive 400 naming route. allow_fallbacks: false still considers only the first routable requested model.

{
  "models": ["deepseek/deepseek-v4-flash-0731-abliterated", "qwen/qwen3.8-27b"],
  "messages": [{ "role": "user", "content": "Hello" }],
  "provider": {
    "sort": { "by": "price", "partition": "none" },
    "preferred_min_throughput": { "p50": 50, "p90": 20 },
    "preferred_max_latency": 2
  }
}

preferred_min_throughput uses generation tokens per second; preferred_max_latency uses seconds to first token. Each accepts a nonnegative number for p50, or an object with p50, p75, p90 and p99 cutoffs. All named cutoffs apply. These preferences use gated 30-minute observations, with throughput percentiles describing performance floors: p90 is met or exceeded by 90% of qualifying streams. The ordinary throughput sort continues to use the daily median.

A measured failure moves behind measured passes within the same capacity and health partition. An endpoint missing requested measurements, with no known failure, retains its position. Endpoints remain available as fallbacks even if every measured endpoint misses a cutoff. Explicit provider order and session affinity remain authoritative preferences; these cutoffs do not promise latency or throughput.

enforce_distillable_text: true requires verified model-author permission for text distillation. The catalogue has no verified permission field, so no endpoint qualifies: the response is 503 with no_route and rejection reason distillable_text_unverified. Omit the field or use false to route without that guarantee.

These controls apply to chat completions, text completions, Messages and Responses. The latter two accept models beside their required primary model; the same routing plan selects the provider.

Session affinity#

An agent harness sends hundreds of requests in one conversation, and a provider's prompt cache only pays when they land on the same engine. Routing therefore remembers which endpoint served a conversation and moves it to the front of the plan on the next request, for an hour past the last one served.

The conversation is identified by an opaque id, up to 256 characters, sent the same on every request that belongs to one conversation and different for an unrelated one. Chat completions take it as the session_id body member; chat completions and Responses also accept prompt_cache_key, and every route takes it as the X-Session-Id header. A chat session_id wins over prompt_cache_key, which wins over the header; on Responses, prompt_cache_key wins over the header. A proxy or an SDK can put a header on a request it did not compose, and a body member is where the author of the turn supplies the affinity hint. The pin is scoped to the workspace and the requested model as well as this id, so the same id sent by two workspaces — or for two different models — never shares a pin. A request naming neither routes with no affinity: a workspace-wide preference would concentrate a fleet of parallel agents onto one engine for no cache gain, so only requests that name their conversation take part.

The resolved id is forwarded to the serving endpoint as x-session-id. An engine that keys its own prefix cache on the conversation can only do so if it is told which conversation this is, and the marketplace is the party that sees both ends. The id is carried and never recorded.

Affinity biases order and never bars fallback. Every routing filter still applies — a remembered endpoint that lost its health standing, its capacity or a required capability is simply not preferred, and the request routes as if never pinned. A provider failure mid-plan falls through to the next endpoint exactly as always; the conversation re-pins to whichever endpoint serves.

Middle-out prompt compression#

A conversation that has outgrown an endpoint's context window can be shortened instead of refused. transforms: ["middle-out"] asks for that. plugins: [{ "id": "context-compression" }] is the same ask written the other way, and either one present asks for compression once.

{
  "model": "deepseek/deepseek-v4-flash-0731-abliterated",
  "messages": [{ "role": "user", "content": "…" }],
  "transforms": ["middle-out"]
}

Compression runs against the endpoint routing has already chosen, because the window that has to be fitted is that endpoint's own. Whole messages are dropped from the middle of the conversation. The system message stays because it carries the instructions, the earliest user turn stays because it carries the task, and the latest turns stay because they are what is happening now. Between those two ends, the oldest turns go first and the fewest that make the prompt fit go at all, so every message dropped is older than every message kept. A tool call is never separated from its result, so an assistant turn carrying tool_calls and the tool messages answering it are kept together or dropped together; an endpoint refuses a result whose call it never saw.

usage reports what was actually sent, so a compressed turn is billed on the shortened prompt.

The instructions, the earliest user turn and the latest turn are what compression can never drop, and that is the size a compressible request is routed on: an endpoint whose window holds them beside the reply is a candidate, and one too small for that is eliminated the way any other endpoint that cannot serve the request is, with the 503 below. The reply counts in full here, unlike on a request that asked for no compression: asking for compression is asking for the history to give way to the answer, so an endpoint that could only fit the conversation by shortening the answer is the wrong home for it. The reply is your max_tokens where you sent one, and the marketplace's own default where you did not.

The credit hold follows the same reading. A compressed turn is held for the conversation the endpoint is sent, not the one you wrote, so a balance that covers the shortened request is enough to make it.

An absent transforms and transforms: [] both mean no compression. The empty list states the refusal rather than leaving it to omission. context-compression is the only served plugin id; any other, web among them, receives 400 naming the id.

Neither spelling reaches the engine. Compression is the marketplace's own work against a window it already knows, so transforms and plugins are answered here and forwarded nowhere.

What reaches the engine#

Each rail forwards its own core members — the reference contract every engine serving that rail must accept, such as messages, temperature and tools on chat completions or input and encoding_format on embeddings — to every endpoint, declared or not. Everything else a request carries, whether a named optional member like top_k or a member only one engine has ever offered, reaches the engine only where the serving endpoint names it in supported_parameters; an endpoint that never claimed to read a knob is not handed one, since the strict engines answer an unknown member with a 400 and the rest ignore it silently. What was withheld is stated rather than left to be inferred: a non-streaming reply carries honoured, one entry per such member the request sent, true where the serving endpoint's declaration carried it up and false where it did not. A member the request never sent is absent rather than false, and a request asking only for core members carries no honoured object at all. POST /audio/speech answers with the recording and no document, so it states the same verdict in the X-Heimdall-Unhonoured response header, naming the withheld members comma-separated and sending no header where nothing was withheld. A streamed reply states the same verdict on its final frame, beside that frame's usage.

Advisory settings#

Routing first looks for an endpoint declaring every optional member a chat or completions request names. Where the model has one, that endpoint serves. Where it has none, the request is planned a second time without its advisory settings — members whose absence leaves the reply the same kind of answer:

• the sampling and bookkeeping members temperature, top_p, top_k, top_a, min_p, frequency_penalty, presence_penalty, repetition_penalty, seed, logit_bias, parallel_tool_calls, reasoning, reasoning_effort, include_reasoning, verbosity, service_tier, store and safety_identifier;
• the values that restate a default: tool_choice: "auto", n: 1, logprobs: false, top_logprobs: 0 and response_format: { "type": "text" };
• the output ceiling, which is lowered to the endpoint's own rather than filtering it, unless the request asked for compression.

An advisory member the serving endpoint does not declare is reported in honoured, as described under what reaches the engine. Nothing else relaxes. tools, structured_outputs, a JSON response_format, a demanding tool_choice, logprobs: true, n above 1, stop, a member only one engine offers, a value outside a declared range, a reasoning level an endpoint's latest check refuted, input and output modalities, the context window and every provider preference hold on both plans. provider.require_parameters: true and the :exacto variant keep the first plan alone, so every member the request names must be declared.

When nothing routes#

Two distinct failures:

• 404 — no requested model exists in the catalogue. If the id names a family, the error lists the served variants.
• 503 — the model exists, but no live endpoint satisfies the request. The message counts the eliminations per rule, and metadata.rejections itemises up to 20 of them:

{
  "error": {
    "code": 503,
    "message": "No endpoints for deepseek/deepseek-v4-flash-0731-abliterated satisfy the request (zdr_required: 2, max_price_exceeded: 1).",
    "metadata": {
      "error_type": "provider_overloaded",
      "rejections": [
        {
          "modelId": "deepseek/deepseek-v4-flash-0731-abliterated",
          "endpointId": "…",
          "providerSlug": "impcc",
          "reason": "zdr_required"
        }
      ]
    }
  }
}