Zilliz Open-Sources Bilingual Semantic Highlighting Model

REDWOOD CITY, Calif., Jan. 31, 2026 /PRNewswire/ -- Zilliz, the company behind the leading open-source vector database Milvus, today announced the open-source release of its Bilingual Semantic Highlighting Model, an industry-first AI model designed to dramatically reduce token usage and improve answer quality in production RAG-powered AI applications.

This highlighting model introduces sentence-level relevance filtering, enabling AI developers to remove low-signal context before sending prompts to large language models. This approach directly addresses rising inference costs and accuracy issues caused by oversized context windows in enterprise RAG and RAG-powered AI deployments.

"As RAG systems move into production, teams are running into very real cost and quality limits," said James Luan, VP of Engineering at Zilliz. "This model gives developers a practical way to reduce prompt size and improve answer accuracy without reworking their existing pipelines."

Key Innovations and Technical Breakthroughs

• Bilingual relevance by design: Optimized for both English and Chinese, the model addresses cross-lingual relevance challenges common in global RAG deployments. It is built on the MiniCPM-2B architecture, enabling low-latency, production-ready performance.
• Sentence-level context filtering: Rather than scoring entire document chunks, the model evaluates relevance at the sentence level and retains only content that directly supports a user query before sending it to the LLM.
• Lower token usage, higher answer quality: Zilliz reports that sentence-level filtering significantly compresses prompt size while improving downstream response quality, helping teams reduce inference costs and improve generation speed in production environments.

Availability

The Bilingual Semantic Highlighting Model is available today as an open-source release. To learn more about the training methodology and performance benchmarks, visit the Zilliz Technical Blog.

Download: : zilliz/semantic-highlight-bilingual-v1