Working with embeddings
Embeddings turn text into vectors for search, clustering, deduplication and retrieval. On
Heimdallm they go through the OpenAI-compatible POST /embeddings
endpoint, against any model whose output modality is embeddings.
Choosing a model
Embeddings models are their own models, not a mode of a chat model. Look for
"output_modalities": ["embeddings"] in the catalogue or filter
the model list by output. Two properties matter when choosing:
- Dimensionality — every vector a model produces has the same length; vectors from different models (or different dimensionalities) are not comparable. Index and query with the same model.
- Context window — input longer than the model's
context_lengthcannot be embedded in one call; chunk your documents below it. - Input basis — a multimodal model can embed a plain string and a content item in two different bases, and vectors from the two are not comparable. See Input forms and bases.
First request
curl https://api-staging.impossiblecarrot.cc/api/v1/embeddings \ -H "Authorization: Bearer $HEIMDALL_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "baai/bge-m3", "input": "The quick brown fox" }'Batching
input takes a single string or an array of up to 2048 strings. One batched call is one request
— one round-trip, one routing decision, one usage accounting — and each vector in data maps
back to its input by index:
{
"model": "baai/bge-m3",
"input": ["first passage", "second passage", "third passage"]
}
Batching is the difference between embedding a corpus in minutes and in hours, and it also spends your rate limit on data instead of round-trips. Pre-tokenized input (arrays of token ids) is accepted where the serving model supports it.
Input forms and bases
input takes text, as a string or pre-tokenized integers, and on a multimodal model it also takes
content items: { "content": [...] } objects whose parts are text, images and the other media the
model declares (see multimodal input). A multimodal model can
embed these two kinds of input in different bases, each a vector space of its own. Where it does, the
shape of the input decides the basis:
| Input shape | How the model reads it | Basis |
|---|---|---|
| A string, an array of strings, or pre-tokenized integers | The text as it stands, with no role turns | Text |
An array of { "content": [...] } items, an item holding only a text part included | Each item wrapped in the model's chat template: role turns and the <embedding> token | Content item |
Vectors from different bases have the same length and compare without error, so the mismatch is
silent: nothing in the response says which basis produced a vector. Measured on
tencent/wemm-embedding-2b, the text a red square sent as "a red square" and as
{ "content": [{ "type": "text", "text": "a red square" }] } embeds 0.1189 apart by cosine distance,
so the same words land close to, but not on, themselves across the two, and the offset is not
the same for every text.
Keep an index and the queries run against it in one basis:
- An index of images, or of items that mix text and images, can only be built from content items. Query it with content items as well, including for a query that is plain text.
- An index of text alone can use either shape, provided its documents and its queries use the same one.
- Changing the shape an index was built from means embedding its documents again.
The shape of the item chooses the basis; no parameter does. A model's page states a second basis in
its description where the provider has declared one, as the page of tencent/wemm-embedding-2b does;
where a multimodal model's page is silent, treat it as having two.
Options
| Field | Meaning |
|---|---|
encoding_format | float (default) returns number arrays; base64 returns each vector as a base64 string (packed float32, the OpenAI convention) — noticeably smaller on the wire. An explicitly requested format requires an endpoint that declares it. |
dimensions | Truncate output vectors to the first N dimensions, where the model supports it (Matryoshka-style models). Vectors of different lengths do not compare. |
provider | The same provider preferences as chat. |
Billing and routing
Embeddings bill input tokens only, at the endpoint's prompt price — there are no completion
tokens, so usage.total_tokens equals usage.prompt_tokens and usage.cost is the exact USD
charge. Routing works as on chat: an endpoint that turns out not to serve embeddings answers as a
provider failure and routing moves on to one that does. Streaming does not apply to embeddings.