Eight Megabytes. All of Wikipedia. Seven Minutes on a Laptop.
If you have built a retrieval pipeline recently, you already know the pain. You pick a transformer-based embedding model because that is what the benchmarks recommend. You spin up a GPU instance, or you sign for an embedding API, and you watch the bill climb while your corpus indexes overnight. For a long time, that was just the cost of doing RAG.
Then Erik Kaunismaki dropped a model called Lattice that scores 0.4749 NDCG@10 on decontaminated BEIR, compresses to 7.94 MB at int4 quantization, and embeds all 6.4 million articles of English Wikipedia in 7 minutes and 26 seconds. On an 8-core Apple M2 MacBook Air. Not a server farm. A laptop.
Why does this matter? Because it changes the math on who can build retrieval systems, and how.
What Lattice actually is
Lattice is a static embedding model. That word, “static,” is doing a lot of work here. It means there is no transformer. No attention mechanism. No contextualization step where the model looks at surrounding words and adjusts meaning. The entire model is a learned lookup table: one fixed vector per token, averaged together via mean pooling, then normalized. That is the whole architecture.
The previous best static model, static-retrieval-mrl-en-v1, scored 0.4334 on decontaminated BEIR. Lattice hits 0.4749. That is not a marginal improvement. It is the difference between a static model being a curiosity and a static model being something you would actually deploy.
Kaunismaki trained it on 660 million curated query-document pairs pulled from 34 sources, filtered with a cross-encoder. The weights are open under the Sentence Transformers layout at erikkaum/lattice-retrieval on Hugging Face.
The throughput number that should make you reconsider your stack
Here is the part I keep coming back to. A custom Rust runtime embeds the entire English Wikipedia in under eight minutes on a consumer laptop. No GPU. No API calls. No per-token pricing. Just a binary and a corpus.
For comparison, a transformer-based embedding model doing the same job takes orders of magnitude longer and requires a GPU. Even with a fast model on a decent card, you are talking hours, not minutes, and you are paying for the compute the entire time.
The compression story is just as striking. At int4-row quantization, Lattice shrinks to 7.94 MB with effectively zero quality loss. The quantized version scores 0.4697 on decontaminated BEIR versus 0.4749 for the full model. You give up less than a single point of retrieval quality to fit the entire model in a file smaller than most mobile app assets.
Where static models still lose
I do not want to oversell this. Static models have real limits, and Lattice inherits them.
Because there is no attention, the model does not capture word order, polysemy, or compositional meaning the way a transformer does. “Bank” next to “river” and “bank” next to “loan” get the same vector. A sentence where word order flips the meaning will confuse it. For maximum-quality retrieval on hard queries, a transformer-based model still wins.
Most retrieval pipelines do not need maximum quality at the first stage, though. They need fast candidate generation that pulls the top 50 or 100 documents, which then get re-ranked by a heavier model. That is exactly where Lattice shines. You use the cheap, tiny, static model to cast a wide net, then you spend your compute budget on a cross-encoder reranker that only has to look at a small slice of the corpus.
Practical use cases
If you are building any of the following, Lattice is worth a serious look:
- First-stage retrieval in a two-stage RAG pipeline. Embed your whole corpus in minutes, pull candidates, hand off to a reranker.
- Corpus-scale deduplication. Need to find near-duplicate documents across millions of files? Static embeddings are fast enough to actually finish the job.
- Hard-negative mining. Generating training data for a bigger model? Lattice can churn through candidate negatives at a speed transformer models cannot touch.
- On-device and in-browser retrieval. 7.94 MB fits in a mobile app. It fits in a browser bundle. You can ship retrieval that runs entirely client-side with no server round-trip.
- Edge and offline systems. No network, no GPU, no problem. If you are building for environments where a cloud embedding API is not an option, this model exists for you.
What this means for the retrieval market
The retrieval stack is getting commoditized, fast. Two years ago, building a production RAG system meant picking a hosted embedding provider, budgeting for indexing time, and accepting that your corpus would live in someone else’s database. Now you can download a model smaller than a high-resolution photo, run it on your laptop, and index Wikipedia before your coffee gets cold.
That does not kill hosted embedding APIs. Quality still matters, and transformer models still win on the hard cases. But it raises the floor. The question stops being “can I afford to build retrieval?” and becomes “do I actually need the expensive option, or will the cheap one do?”
For a lot of teams, the answer is going to be the cheap one. And that changes how you architect things. You stop designing around API rate limits and start designing around local throughput. You stop worrying about vendor lock-in for your index and start treating embeddings as a build artifact you control.
How to try it
The model lives at erikkaum/lattice-retrieval on Hugging Face. Kaunismaki published a full writeup on the Hugging Face community blog walking through the training process, the quantization results, and the Rust runtime. If you want to benchmark it against your current setup, the BEIR numbers in the writeup give you a clean comparison point.
My suggestion: pull the int4 quantized version, point it at a slice of your actual corpus, and measure. The whole test takes maybe 15 minutes to set up. If the retrieval quality holds up on your data, you just cut your embedding bill to zero for first-stage retrieval.
That is not a small thing.
Based on coverage from the Hugging Face Community Blog. Lattice is open-weight under the Sentence Transformers layout.
