Automatic calibration at merge time picks vector quantization parameters for each segment by predicting recall from a small sample. Here's how we built it into Elasticsearch's merge path. Elasticsearch's DiskBBQ format (IVF clustering plus binary quantization, built for on-disk ANN search at scale) offers several knobs to shape the recall/cost tradeoff of an index. Automatic calibration seeks to optimize those knobs to achieve optimal performance.
In our previous blog , we laid out the statistical model behind that calibration: a manifold model for how nearest-neighbor distances scale with index size, a Gaussian error model for quantization noise, and a closed-form way to combine the two into an expected recall@k for a given rerank depth. If you haven't read it, the one thing you need going in is this: given a candidate quantization encoding and a rerank depth, we can predict recall@k without building an index and benchmarking it, by fitting two small models to a sample of the corpus.
In this post we’ll go through how to score candidate configurations cheaply and how to leverage that to make merge-time decisions that are themselves cheap, correct, and consistent across real, constantly-merging indexes. This led to some pretty impressive improvements: we see almost 17% average improvement in QPS across a broad range of datasets all while increasing recall (in one case by a factor of 3). What’s more you get this immediately by adding one line to your index options, "auto_calibrate": true , and our plan is to make this our default once it has had the chance to bake a bit.
bbq_disk exposes several knobs: quantization bits for documents (1, 2, 4 or 7), a separate bit width for queries, an oversampling factor for reranking, and whether to precondition vectors before quantizing. None of these act independently, and their effect on recall depends on the data: a 4-bit/1-bit encoding might be plenty for one embedding model and clearly insufficient for another. A single index is also built out of many segments, merged over time, each with an eventually different vector distribution.