Neos:
Learning to Interleave
Search and Visual Reasoning
1 Accio Team, Alibaba Group
2 Shanghai Innovation Institute 3 Fudan University
* Equal contribution ‡ Project leader † Corresponding author
Read the abstract01 Results
Across ten benchmarks.
Neos-9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents.
02 Overview
Abstract
Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance for subsequent search, limiting their effectiveness in long-horizon settings. To address this gap, we present Neos, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions.
To make this coupling learnable, Neos constructs training tasks with intermediate visual dependencies while filtering out textual shortcuts, and introduces a lightweight, trainable Multimodal Evidence Interface (MEI) that transforms retrieved content into goal-conditioned observations. At its core, an agent–environment co-evolution procedure treats MEI as a co-adaptive component of this loop. Specifically, reference-guided credit assignment uses verified successful trajectories to localize critical policy errors while preserving credit for useful decisions in failed searches, whereas rollout feedback selects informative local evidence-processing interactions for MEI supervised fine-tuning (SFT).
By alternating policy reinforcement learning (RL) with MEI SFT, Neos allows the policy's evolving evidence requests to shape MEI training and the adapted MEI to reshape subsequent observations, closing the loop between evidence acquisition and evidence-conditioned action. Across ten benchmarks, Neos-9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents.
03 Method
An evolving evidence–decision loop.
The policy learns to acquire evidence. The Multimodal Evidence Interface learns to make that evidence useful for the next decision.
04 In action
Watch the search unfold.
Start with an image and a question. Follow each tool call as new visual evidence changes what the agent searches for next.