Google Research Unveils 53.9M AI Search Model, Cutting Latency 12-20x
Updated
Updated · Google Research · Sep 15
Google Research Unveils 53.9M AI Search Model, Cutting Latency 12-20x
1 articles · Updated · Google Research · Sep 15
Summary
Retrieve-for-Train shifts heavy computation from inference to training, using offline reinforcement learning to teach a 53.9 million-parameter diffusion retriever to generate search fan-outs in a single pass.
Google says that bypasses autoregressive LLM “thinking budgets” that can push fan-out retrieval latency toward 50 seconds under large context batches, while the new system stays in sub-second to low-single-digit seconds.
The framework trains 4B open-source models including Gemma3-4B and Qwen3-4B to produce 10 sub-queries per prompt, then distills those reward-aligned behaviors into the lighter diffusion model.
Across fashion and music retrieval tests, the system outperformed single-query search, zero-shot expansion and Best-of-N baselines by producing more diverse, grounded and complementary result sets.
Google positions the approach as a scalable way to improve multimodal and specialized search where expert-labeled training pairs are scarce and online LLM inference is too costly.