Operating a source

Listing your models defines the contract; this page is about living with it — keeping a listing healthy, changing it safely, and diagnosing the pull findings the dashboard reports.

Pulls, precisely#

Heimdallm fetches your model-list document on a schedule (default every 60 seconds, configurable per source from 1 minute to 24 hours) and on demand. Every run is recorded with its outcome, and each outcome has fixed consequences:

The design rule underneath: an outage of your models URL never delists you. Only a pull that read your document withdraws anything, and what it withdraws is every endpoint no valid entry of that document lists — omitted or failing validation alike (history and stats kept). A document whose entries all fail validation therefore takes the source's endpoints out of the listing until the next pull that validates, and that pull brings them back on its own: the same endpoint re-enters pending health, with no manual step. An unreachable source costs you updates, not presence.

Withdrawal follows the document, not the endpoint's health: to take a model down deliberately, remove it from the document (or set is_ready: false to keep it listed but out of routing — the right tool for maintenance windows).

A live endpoint leaves routing on the second consecutive pull declaring is_ready: false, so a flag that flickers between pulls costs you nothing and a maintenance window takes hold one pull interval after you declare it. Declaring true again at any point restarts the count. A pull that omits a live endpoint counts the same as one declaring false: a live entry missing from one read stays live, and is withdrawn by the second consecutive pull that omits it or declares it not ready. An endpoint that did leave returns through pending health rather than straight to live, since live is earned from measured attempts; the probes it already answered while live are what promote it, so the wait is one health-gate pass and not a fresh round of checks.

Health and the lifecycle#

A new entry starts pending health and enters probing: Heimdallm calls it through the same inference path customers use. Once two of its last three probes succeed, the endpoint goes live and enters routing — no manual step. A failing endpoint is retried on a growing backoff (1, 5, 15, then every 60 minutes) and never forgotten, so a recovering endpoint re-enters the market on its own.

From routing's side, an endpoint's health tier is computed from the newest 20 attempts routed to it rather than from a clock window, so it recovers exactly as fast as new evidence arrives: healthy at 90% of them or better, degraded at 60%, down below. A run of consecutive probe failures short-circuits it downwards without waiting for the sample to turn over — two in a row is degraded, four is down — so a real outage shows within minutes. Healthier tiers are preferred.

One isolated failure moves none of this. A request that fails for a reason that predicts nothing about the next one — a socket that died in flight, an unreadable reply, a 5xx — is asked once more a quarter-second later, and only its second failure is recorded against you at all. Health is never what you claim; it is what the traffic shows, and a blip is not traffic.

Declarations, checked#

supported_parameters decides who receives which traffic, so the entries routing acts on are asked about directly rather than taken on trust. Tool support can come from your declaration or be inferred from the linked checkpoint. An endpoint with tools is asked to call one; an endpoint declaring a reasoning_effort enum is asked at each declared level. An absent or unknown level domain does not claim all API values and schedules no per-level checks. Each declared level carries its own verdict, so a level your engine refuses is withheld on its own and every level it serves keeps its traffic. none is the one level judged on the reply rather than on the status: it passes when the reply's reasoning channel carries nothing, because a caller who asked for none is otherwise billed for deliberation out of the budget that had to hold their answer.

A prose reply under tool_choice: "auto" is allowed and does not refute tool support. A reply cut short by the probe’s token budget is inconclusive too. Actual refusals and malformed calls remain failures.

A failed check is not downtime and never touches your uptime. The endpoint serves everything else that day perfectly, and marking it down would take away the traffic it can serve to punish the traffic it cannot. What it does is withhold the promise: requests needing that capability, or that level, go elsewhere until a check passes again. Checks repeat every six hours while passing and every half hour while failing, so a fixed engine wins its traffic back within half an hour with nothing to file. A check that failed without your engine refusing the request — a timeout, an overload, an outage — reaches no verdict at all and withholds nothing. Your dashboard names any declaration currently withheld, and the words your engine refused it with.

Reporting capacity#

heimdall.live_capacity in your document (current of total concurrent request slots, across all replicas) is read on a fast poll of its own, separately from the full pull. Fresh reports steer routing: endpoints with free slots are tried before endpoints reporting nothing, saturated ones last — never excluded, so requests still route when everything is busy. A report older than about a minute counts as no report.

The honest implementation is a live counter around your serving path — requests in flight over slots actually provisioned. A constant current: 0 is detectable and worth nothing; a racing counter is tolerated (current above total is clamped, not rejected). If you cannot report capacity, omit the field: an absent report is neutral, a fake one is not.

Changing things safely#

• Price changes apply on the next pull that carries them, and land in the public price history as steps. See Pricing & demand.
• Credential rotation — update the credential on the source; the next pull presents it. Keep the old credential valid until the change is saved, or the pull answers unauthorized.
• Moving infrastructure — bring the new origin up, point the source's base URL at it, and let the pull reconcile. Withdrawals happen only on a pull that read the document, for every endpoint no valid entry lists, so keep the document complete and valid during the move.
• Multiple sources are allowed (each a distinct base URL) — a chat cluster here, a transcription service there, one catalogue. Endpoints are keyed by source and model id.
• Deprecating — set deprecation_date in the document and keep serving; give customers a runway measured in weeks, and list the replacement under its own id next to it. The catalogue keeps your history; a vanished id is a broken integration for everyone who pinned it.

Anthropic-dialect sources#

A source whose dialect is Anthropic-compatible is for providers whose origin speaks the Anthropic Messages API. Heimdallm posts to <base URL>/v1/messages with its own credentials and translates on the way through: OpenAI-style messages and tools become Anthropic shapes, and the usage that comes back is reassembled into the OpenAI fields (input, output, cached tokens).

Boundaries worth knowing:

• Chat only, streaming included. Completions, embeddings and transcriptions are not served by an Anthropic-dialect source — those requests fail rather than half-translate.
• A completion ceiling is required. A request that sends no max_tokens is sent upstream with a default ceiling, since the dialect demands one.
• The model list is still yours. The dialect changes how inference calls are shaped, not the model list document or the health probes.

If your origin serves both dialects, prefer OpenAI-compatible: it is the dialect the whole marketplace's surface (embeddings, transcriptions, parameter forwarding) is built around.

Common findings, mapped to fixes#

Everything the dashboard reports is also in the pull run itself — outcome, HTTP status, the entry and field for each finding — so a fix can be verified with one Pull now.