Most search teams wonder whether they matched the query. Instacart's latest research says that's the wrong question

On July 29, Instacart's search team posted a paper on discovery-augmented search to arXiv, built around a question most search systems never really ask: are we showing shoppers what matches their query, or what they'd actually want to buy?

The problem with precision-only search

Grocery search has always leaned hard on precision: match the query, move on. But satisfaction (and revenue) depend just as much on surfacing substitutes, complements, and related items shoppers didn't think to search for. Instacart built a two-stage setup for this: a large closed-weight LLM generates implicit intents for head queries, and a distilled small model (LoRA plus teacher-student training) carries that same behavior down to the long tail, which is where most real query volume actually lives.

Old way vs. new way

Old way: optimize for exact query match, and accept that tail queries get thin results because running an expensive LLM on every rare query doesn't scale. New way: spend the expensive model where it counts, then distill that behavior into a cheap model that can run everywhere else.

The result

The approach pushed discovery coverage from roughly 60% to 80% of query traffic, at about 30% of the teacher model's inference cost. Credit to the authors for not hiding the rough edges either: they flag hallucinated brand names, decoding-temperature quirks that break carousel coherence, and offline eval bias baked in from training on logged sessions that already reflect the old ranking.

The open question

Is a 20-point jump in discovery coverage a real satisfaction win, or does surfacing more "related" items just pad the funnel without moving purchase intent? The paper's own end-to-end F1 sits around 0.17-0.18, a reminder that "technically relevant" and "what the shopper actually wanted" are still two different things.

Our take

We build custom ML systems for marketplace and e-commerce clients, and this tradeoff, where the expensive model earns its cost and where a distilled one should carry the load instead, shows up in almost every personalization and forecasting system we ship.