0.35% 훈련으로 100% 경쟁력 확보: jina-embeddings-v5-omni의 프로즌 타워 아키텍처

The latest jina embeddings model generates multimodal embeddings for text, images, video and audio, competing with models nearly 6x its size on vector search while training just 0. jina-embeddings-v5-omni is our latest multimodal embedding model. It generates embeddings for text, image, video, and audio. Among open-weight models that support those modalities, it’s the best-performing under 2 billion parameters. The notable part is how little of it we actually trained. Every encoder tower stayed frozen, and only about 0.

35% of the model's weights (projectors and a handful of delimiter tokens) were ever updated during training. We call this architecture pattern G eometry-preserving E mbeddings via L ocked A ligned TO wers (GELATO). Let's break down each letter of this acronym: Geometry-preserving Embeddings: jina-embeddings-v5-omni sits atop the foundation laid by jina-embeddings-v5-text . That original text embedding space is completely unchanged, with its geometry left wholly intact. Locked: Synonymous with "frozen. " All of the towers in this architecture have their weights locked.

Aligned: Aligning the other modalities with the text model's vector space, allowing for cross-modal comparison. TOwers: Modality component that converts one type of raw input into vectors. The model comes in two variants: small and nano . The former has more parameters (1. 57 billion) than the latter (0. 95 billion), but functionally their architectures are nearly identical. For the sake of brevity, we mostly focus on jina-embeddings-v5-omni-small in this article. jina-embeddings-v5-omni relies on three core machine learning (ML) concepts: vectors, towers, and frozen weights.

Here's what each means; feel free to skip ahead if you're already familiar with them. How does AI understand abstract concepts? Can a machine comprehend what "ice cream" is? Does it understand that "chocolate fudge" and "rocky road" have more in common with each other than "sorbet"? The answer, surprisingly, is yes. The mechanism that makes it possible is the vector. If you tried visualizing a way to organize all ice cream flavors, you may end up with something like this: a simplex (triangle) with three vertices, each representing a base flavor.