NVIDIA opens Nemotron 3.5 Embed and Rerank for visual RAG

NVIDIA published its Nemotron 3.5 Retriever release on October 21, adding multimodal embedding and reranking checkpoints for agentic search. The collection addresses a familiar retrieval-augmented generation failure: agents spend inference tokens examining documents that are similar to the question but do not contain the answer. This article focuses on the newly released Embed and Rerank weights; Nemotron 3.5 Parse document extraction was covered separately in the site's previous edition.

Nemotron 3.5 Embed 8B BF16 maps text and document-page images into vectors for nearest-neighbor search. A distilled Embed 1B BF16 provides a smaller footprint, with an NVFP4 variant offered for its own deployment profile. The Rerank 8B BF16 model is a cross-encoder: it jointly examines a query and each retrieved candidate to produce a relevance score. The practical workflow is retrieve broadly using embeddings, then spend heavier inference on only a shortlist.

NVIDIA reports 61.17 nDCG@10 for Embed 8B and 58.38 for Embed 1B on ViDoRe v3.1. It reports 60.49 nDCG@10 for Rerank 8B using combined image-and-text inputs. The company also describes lower total agent inference tokens when reranking is added to a controlled search configuration. These are vendor-published evaluations tied to specific datasets and agent settings, not universal results for enterprise PDFs, languages or private corpora.

The governing download terms in the model cards are OpenMDW License Agreement 1.1; the Apache-2.0 license of a base model should not be confused with the released checkpoint terms. Our assessment: multimodal retrieval is becoming a core open-weight agent component, but operators should test document ingestion, index migration, rerank latency, prompt-injection exposure and rights to deploy. Quantization and model size alone cannot prove lower end-to-end cost.