Logo DeepEyes

Incentivizing “Thinking with Images” via Reinforcement Learning

Ziwei Zheng1,2,*, Michael Yang1,*, Jack Hong1,*, Chenxiao Zhao1,*,†,
Guohai Xu1,‡, Le Yang2,‡, Chao Shen2, XingYu1
*Equal contribution. Random Order. †Main Code Contributor ‡Corresponding Author
1Xiaohongshu Inc. 2Xi'an Jiaotong University
Paper arXiv Code

🔍

 Model

🤗

 Dataset

Abstract

Large Vision-Language Models (VLMs) have shown strong capabilities in multimodal understanding and reasoning, yet they are primarily constrained by text-based reasoning processes. However, achieving seamless integration of visual and textual reasoning which mirrors human cognitive processes remains a significant challenge. In particular, effectively incorporating advanced visual input processing into reasoning mechanisms is still an open question.

Thus, in this paper, we explore the interleaved multimodal reasoning paradigm and introduce DeepEyes, a model with "thinking with images" capabilities incentivized through end-to-end reinforcement learning without the need for cold-start SFT. Notably, this ability emerges natively within the model itself, leveraging its inherent grounding ability as a tool instead of depending on separate specialized models. Specifically, we propose a tool-use-oriented data selection mechanism and a reward strategy to encourage successful tool-assisted reasoning trajectories.

DeepEyes achieves significant performance gains on fine-grained perception and reasoning benchmarks and also demonstrates improvement in grounding, hallucination, and mathematical reasoning tasks. Interestingly, we observe the distinct evolution of tool-calling behavior from initial exploration to efficient and accurate exploitation, and diverse thinking patterns that closely mirror human visual reasoning processes.

DeepEyes

Method

Overview of DeepEyes.

Our model itself decides whether to perform a second perception via zoom-in by generating
grounding coordinates and cropping relevant regions, or to answer directly.

Results

Visual Search Benchmarks

overall_performance

DeepEyes achieves better performance on visual search benchmarks, such as V* and HR-Bench.

Grounding and Hallucination Benchmarks

overall_performance

Our visual reasoning can further reduce the hallucinations.

Multimodal Reasoning Benchmarks

overall_performance

DeepEyes exhibits strong reasoning ability.

Key Findings

Training Dynamics

overall_performance

Tool-calling dynamics evolve through three distinct stages during RL training:
s1: initial exploration with limited effectiveness
s2: aggressive yet successful usage,
s3: selective and efficient exploitation with high performance.

Thinking Patterns

Visual Search

Visual Comparison

Visual Confirmation

Hallucination Mitigation

Citation


        @article{zheng2025deepeyesincentivizingthinkingimages,
          title={DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning}, 
          author={Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, Xing Yu},
          year={2025},
          eprint={2505.14362},
          archivePrefix={arXiv},
          primaryClass={cs.CV},
          url={https://arxiv.org/abs/2505.14362}, 
        }