#Japan #Tokyo #Dentsu Research #Tohoku University #Studio Ousia

Triumph in AI: Dentsu Research Institute Shines at NeurIPS 2025

Dentsu Research Institute, headquartered in Tokyo and led by President Hirohisa Iwamoto, has achieved significant recognition in the AI field by winning the Best Static Evaluation award at the prestigious NeurIPS 2025 international conference. This success was achieved in collaboration with Tohoku University's Language AI Research Center, based in Sendai, led by Jun Suzuki, and Studio Ousia, headquartered in Chiyoda, Tokyo, under the leadership of CEO Yasuhiro Watanabe.

NeurIPS (Neural Information Processing Systems) is renowned for being among the top international conferences focusing on AI and machine learning technologies. The competition they participated in, the MMU-RAG (Massive Multi-Modal User-Centric Retrieval-Augmented Generation), aimed to evaluate systems designed for long-form question-answering known as Deep Research. Such systems are expected to process queries accurately and efficiently, a function that has been increasingly essential in today’s data-driven world.

Overview of the MMU-RAG Competition

The MMU-RAG competition was designed to track performance in text-to-text tasks, highlighting the capabilities of systems catering specifically to extensive questions. Deep Research refers to a sophisticated research capability found in generative AI tools, including those in ChatGPT and Gemini, which support autonomous information retrieval and summarization.

However, many existing technologies are proprietary, posing significant barriers to academic access and verification. This is where academia shines, spearheading studies to replicate these sophisticated systems using open-source technologies.

Historically, much of the research has centered around short-form answers, as these lend themselves to straightforward verification and simplified learning assessments. The innovations proposed in this competition tackle the complexities inherent in long-form question-answering, putting forth the need for new methods to assess these extended responses and optimize training processes accordingly. The insights garnered from this competition promise to serve as valuable benchmarks for future academic and corporate endeavors focused on advancing open-source Deep Research capabilities.

The Essence of Deep Research Technology

Deep Research systems integrate various functionalities, known as 'tools', centered around large language models (LLMs) to provide highly sophisticated automated responses that approach the quality of expert analysis. Some potential applications include:

• - Market research and competitive analysis
• - Literature reviews and trend assessments
• - Marketing research initiatives
• - Financial data analysis

The strengths of these systems lie in their fundamental variances from traditional keyword search methods:

1. In-depth Understanding of Search Intent: Rather than simply inputting questions as search queries, the system comprehends the background and purpose behind the inquiries, architecting multi-step research processes automatically.
2. Automated Information Gathering from Diverse Sources: Following the research plan, the systems gather necessary information progressively from LLM knowledge bases, various online resources, and databases comprehensively.
3. Assessment of Information Credibility: The information collected is evaluated against multiple criteria, ensuring reliability and consistency, with clear attributions for the data utilized.
4. Multifaceted Analysis and High-Quality Insights: By validating connections and consistency among pieces of information, the systems derive rich insights that transcend simple summarization.

The Motivation and Goals Behind Participation in MMU-RAG

Dentsu Research recognizes the imperative nature of Deep Research within its AI solutions, especially in improving efficiencies in areas such as digital finance. Historically, the reliance on proprietary Deep Research features posed challenges for customization and cost-effectiveness.

Thus, the intention behind entering this competition lay in making open-source technologies a tangible asset for developing in-house Deep Research capabilities, in partnership with Tohoku University and Studio Ousia. Continuous technical validation and exploration of cutting-edge technologies remain a priority in this collaborative effort.

Undertakings and Outcomes of the MMU-RAG Competition

Under the direction of Yamada from Studio Ousia, the team enhanced the accuracy of long-form answer generation utilizing Qwen3-Next-80B-A3B as a base. A novel framework for automatic evaluation based on Key Point Recall was developed to tackle the intricacies of gauging factual accuracy. The advanced search module integrated with optimal hyperparameter adjustments enabled an efficient processing of long-form Q&A.

Thanks to these efforts, the team not only secured the best static evaluation score in the open-source division but also emerged victorious in the competition.

Implications and Future Applications

The achievement of this collaboration has led to the acquisition of critical knowledge and reusable architectures capable of designing, implementing, and evaluating Deep Research systems exclusively with open-source technologies. This foundational expertise allows various sectors—including academia, corporate knowledge management, and public domains—to develop advanced research functions centered on long-form Q&A.

Dentsu Research aims to systematically integrate this functionality into its range of solutions to boost efficiencies across research, planning, document preparation, and customer interactions. Anticipated applications extend from searching technical documents in manufacturing to researching data for policy-making in the public sector and automating research within consulting projects.

Digital finance stands out as a pivotal domain, emphasizing how the integration of Deep Research technology can facilitate in-depth analyses across regulatory, guideline, and market data sources to enhance the design, assessment, and monitoring of new financial services. Dentsu Research strives to bolster its offerings of digitally native financial functionalities, thereby contributing to a robust decision-making platform across diverse industrial landscapes.

Paper Information

• - Title: An Open and Reproducible Deep Research Agent for Long-Form Question Answering
• - Authors: Ikuya Yamada, Wataru Ikeda, Ko Yoshida, Mengyu Ye, Hinata Sugimoto, Masatoshi Suzuki, Hisanori Ozaki, Jun Suzuki

Dentsu’s ongoing commitment to evolving AI solutions solidifies the synergy between technology and human insight, paving the way for future innovations.