A visual puzzle benchmark can travel a long way in ten months. VisuLogic’s April 21, 2025 paper examined the difficulty of reasoning from pictures.[1] By February 14, 2026, ByteDance was naming it in the Chinese-language announcement of Seed2.0, alongside claims of better visual understanding for production workloads.[5] A research test had become part of a product’s public evidence.

Three sources help interpret that journey: VisuLogic’s experiments, the related information controls in MathVerse, and the software that turns a model’s response into a score. Together, they suggest a useful way to read progress in China’s multimodal AI ecosystem. Ask both whether the answer improves and which visual relationships survive the path to that answer.

Imagine describing a shape puzzle to someone who cannot see it. “Three similar figures, each with a shaded corner” sounds informative. Yet the missing detail might be whether the corner rotates with the figure or stays fixed on the page. A fluent description can preserve the scene while dropping the relationship that decides the answer.

A harder test enters the product story

VisuLogic’s collaboration includes the University of Science and Technology of China, Xi’an Jiaotong University, Shanghai AI Laboratory, SenseTime Research, and Tsinghua University.[3] Its evaluation repository describes 1,000 questions across six broad categories and 23 subcategories.[2] This institutional network supplies both a test and public artifacts through which other teams can examine it.

The project leaderboard, checked on September 21, 2026, lists Seed2.0 at 47.4% and PEREA-1.0 at 52.8%. Those rows sit well above the original paper-era results. The page retains an older abstract saying that most models score below 30%, illustrating how introductory prose and a growing leaderboard can describe different moments.[3]

ByteDance’s Seed2.0 announcement explicitly reports improvement over Seed1.8 on VisuLogic and other visual puzzle tasks. The company connects stronger visual understanding with the demands of documents, charts, and other unstructured material in production workloads.[5] That connection is the field signal: fine visual reasoning is being presented as a foundation for work on messy information, rather than remaining confined to a research demonstration.

The limits matter. The 47.4% is a reported benchmark result; this article does not independently reproduce it or establish matched prompts, inference budgets, and runtimes across the later leaderboard entries. Nor does success on these puzzles directly establish reliable document processing. That transfer needs its own evidence.

Even the human reference has a particular setting. The paper’s 51.4% aggregates answers from 100 science and engineering graduate students, each solving ten randomly sampled questions with two to five minutes allowed per question.[1] A leaderboard entry above it warrants inspection of the evaluation procedures before supporting a broad claim about human visual intelligence.

Two research designs ask where the evidence lives

In VisuLogic’s text-model experiment, GPT-4o produced descriptions conditioned on the question, with instructions to describe fine detail without solving the problem. Language models then worked from those descriptions. With chain-of-thought prompting, Qwen2.5-72B-Instruct scored 28.0% and DeepSeek-R1 26.6%; the authors’ simulated random baseline was 24.9%, close to the theoretical 25% for four choices.[1]

My reading is that this experiment tests an information handoff. A solver receiving an incomplete description cannot inspect an omitted corner. Its failure combines the quality of the caption with its ability to reason from that caption. A stronger claim about the inherent limits of language would require a description checked for every relationship necessary to solve the item.

MathVerse, developed by researchers including teams at CUHK and Shanghai AI Laboratory, approaches the information problem differently. It transforms 2,612 visual mathematics problems into six versions with different allocations of information between text and image. Its project page separates conditions such as text-only and vision-only, making the availability of visual evidence an explicit variable.[4]

The attraction is causal clarity. Changing the channel while preserving the underlying problem helps reveal which evidence a model uses. A high score when the question’s wording already supplies the relevant geometry means something different from success when that geometry must be recovered from the picture.

These projects support a shared evaluation question without establishing that every task requires the same architecture. For a document assistant, the useful engineering inquiry is whether an intermediate description preserves the relationships required downstream. Sometimes a compact text representation may be adequate. Sometimes a reader will need to return to the image. The choice should be tested on the actual task.

The evaluation software completes the story

A further handoff occurs after the model finishes speaking. VisuLogic’s repository describes boxed-answer extraction, predefined extraction methods, and an LLM-based extractor that reduces a response to A, B, C, or D before checking the reference label. It also records integration into VLMEvalKit on April 26, 2025.[2]

Public evaluation code makes this dependency inspectable. In a rerun, I would retain the full response alongside the extracted letter and review disagreements, especially responses that mention several candidates. A model can reason poorly and land on the correct option; an extraction failure can obscure a useful answer. The aggregate score cannot distinguish those cases.

The next persuasive evidence would connect these stages. Give a fixed solver both a model-generated caption and a verified description. Separately, change one decisive visual relationship while preserving the rest of a puzzle: reflect a shape instead of rotating it, or move a mark without changing its count. Inspect whether the answer changes appropriately. These are proposed follow-up tests, not reported VisuLogic results.

For teams building visual assistants, the practical implication is to preserve the input image, exact prompt, intermediate description, full response, and scored answer. The upward-moving leaderboard is encouraging. Its value for product development grows when an evaluator can identify which relationship a system now captures, where it previously disappeared, and whether that improvement survives in the work users actually bring.

Sources

1. Weiye Xu et al., “VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models,” arXiv v1, April 21, 2025 — Table 1, section 4.1, and appendix C.1 specify the caption experiment and human baseline.
2. VisuLogic-Benchmark, “VisuLogic-Eval” — official evaluation repository, dataset scope, release history, model interface, and answer-extraction procedure; accessed September 21, 2026.
3. VisuLogic authors, official project page — institutional affiliations and later leaderboard entries, as displayed September 21, 2026.
4. Renrui Zhang et al., “MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?” — official project page describing six information conditions for visual mathematics evaluation.
5. ByteDance Seed, “Seed2.0 Officially Released,” February 14, 2026 — Chinese-language firsthand announcement discussing VisuLogic and production demands for multimodal understanding.
6. Leiem, “Gate of University of Science and Technology of China,” April 5, 2018 — original photograph and provenance, Wikimedia Commons, CC BY-SA 4.0.