Neos: Learning to Interleave
Search and Visual Reasoning

Ekko He1,* Jack Hong3,*,‡ Shilin Yan1,*,‡ Xintong Zhang1 Yujie Zhang2
Wenjie Li2 Haoran Sun3 Kaixun Jiang3 Jiyuan Fu3 Guannan Zhang1 Hongwei Xue1,†

1 Accio Team, Alibaba Group

2 Shanghai Innovation Institute 3 Fudan University

* Equal contribution ‡ Project leader † Corresponding author

arXiv GitHub
Read the abstract

01 Results

Across ten benchmarks.

Neos-9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents.

Benchmark performance of Neos-9B and Neos-35B. Four representative deep-search benchmarks: MMSearch, MMSearch+, BC-VL, and MM-BC.

02 Overview

Abstract

Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance for subsequent search, limiting their effectiveness in long-horizon settings. To address this gap, we present Neos, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions.

To make this coupling learnable, Neos constructs training tasks with intermediate visual dependencies while filtering out textual shortcuts, and introduces a lightweight, trainable Multimodal Evidence Interface (MEI) that transforms retrieved content into goal-conditioned observations. At its core, an agent–environment co-evolution procedure treats MEI as a co-adaptive component of this loop. Specifically, reference-guided credit assignment uses verified successful trajectories to localize critical policy errors while preserving credit for useful decisions in failed searches, whereas rollout feedback selects informative local evidence-processing interactions for MEI supervised fine-tuning (SFT).

By alternating policy reinforcement learning (RL) with MEI SFT, Neos allows the policy's evolving evidence requests to shape MEI training and the adapted MEI to reshape subsequent observations, closing the loop between evidence acquisition and evidence-conditioned action. Across ten benchmarks, Neos-9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents.

03 Method

An evolving evidence–decision loop.

The policy learns to acquire evidence. The Multimodal Evidence Interface learns to make that evidence useful for the next decision.

Agent–environment co-evolution. Policy RL alternates with MEI SFT on outcome- and process-filtered online interactions. Reference-guided credit supports policy optimization and identifies useful local interactions for MEI updates.

04 In action

Watch the search unfold.

Start with an image and a question. Follow each tool call as new visual evidence changes what the agent searches for next.

From a concert to six controls

MMBC · 10 turns
Ready

What changes?