VoyageAI by MongoDB: voyage-4-large

voyage-4-large is Voyage AI's flagship text embedding model in the Voyage 4 family. It represents source passages and queries as numerical vectors for general-purpose and multilingual retrieval. Its two distinguishing features are a mixture-of-experts model design and the shared embedding space used by voyage-4-large, voyage-4, voyage-4-lite and voyage-4-nano.

Shared space: separate corpus and query choices

The manufacturer describes asymmetric retrieval: documents can be embedded with voyage-4-large while a different Voyage 4 family model prepares the query vectors. For example, a corpus represented by Large can be searched with voyage-4 queries. This lets the two sides of search have different model choices while remaining in the documented compatible space.

A practical collection can therefore use Large to prepare stable document passages and choose the query-side model independently. Documents are embedded when the corpus is created or updated, while queries are vectorized as users search. The family announcement describes changing the query model within the compatible family without re-vectorizing the document corpus solely for that change. Keep compared vectors at matching dimensions and in representations the search system can interpret together.

Mixture-of-experts embedding design

A dense feed-forward layer processes every token through its full parameter set. The MoE approach described for voyage-4-large uses a router and expert feed-forward networks, selecting the experts that process each token. This separates total model capacity from the parameters active for a particular token. Voyage reports an activation ratio of 1/10 for the model's MoE design, using top-k routing.

The maker's technical explanation also describes two development choices. Expert capacity affects whether excess tokens bypass an expert layer through a residual path; the team chose the largest feasible capacity factor to avoid retrieval-quality loss. In the final training stage, router parameters are frozen before model merging so interpolation of the source models does not change the learned expert-selection routing. These are characteristics of model preparation, distinct from the text-length or truncation settings used when preparing an input.

Dimensions and output representation

The official model table specifies a 32,000-token context window and selectable 256-, 512-, 1024- and 2048-dimensional embeddings, with 1024 as the default. The family uses Matryoshka learning for nested representation sizes and supports quantization choices: 32-bit floating point, signed or unsigned 8-bit integers, and packed binary output.

With binary output, the returned byte-list length is one eighth of the selected dimension because each byte contains eight embedding bits. Preserve the dimension and data-type choices with the index, and use a comparison method that supports that representation. A smaller or compressed vector changes the representation used for search; assess the relevant passages returned on actual questions before choosing it for a collection.

Prepare document and query inputs

Voyage recommends using the document role for corpus passages and the query role for search wording. Its role option adds the corresponding retrieval instruction before embedding. Preserve the passage's useful heading, conditions and source location, then store the returned vector with the original text and a stable identifier.

For a technical knowledge collection, a question about a failure mode should lead to the passage describing that mode and its relevant conditions. For a document collection spanning languages, retain the source language and evaluate the questions and terminology users will search with. The release covers retrieval material including technical documents, conversations, long documents, code, law, finance and web text. The embedding locates candidate material; a later reranker or answer stage uses the original text.

Context, batching and truncation limits

The maker's current guide allows up to 1,000 strings and 120,000 total input tokens in a voyage-4-large request. This differs from the batch allowance listed for voyage-4, while both have a 32,000-token per-text window. Default truncation shortens an over-length input to fit; disabling truncation returns an error for an over-length text.

Choose answer-bearing passage boundaries deliberately so a long source does not lose a relevant qualification at a truncated tail. Keep the source association when returning search results, and inspect the actual candidate text rather than treating vector proximity as verification of its contents. Shared-space compatibility applies to the documented Voyage 4 text family; contextualized chunk processing, image/video embeddings and later model editions remain their own model-specific workflows.