Exploring Agentic AI: Appier’s Research on Enhancing AI Decision-Making and Multilingual Capabilities

In the rapidly evolving field of artificial intelligence, Appier stands as a pioneering company committed to pushing the boundaries of innovation. Notably, they have focused on enhancing the reasoning capabilities of AI systems through a series of significant research papers. These developments are crucial as businesses increasingly integrate AI into core operations, leading to the concept of Agentic AI - Artificial Intelligence systems empowered to make autonomous decisions.

Understanding AI Limitations: Recognizing Information Gaps

One of the foremost challenges in employing AI for enterprise applications is its ability to discern when the information it retrieves is insufficient. This concern was the focal point of one of Appier's recent studies. The team investigated “None of the Above” (NA) scenarios to understand how AI, particularly Large Language Models (LLMs), can better handle situations where valid answers are not available.

Through rigorous testing of 28 leading LLMs, the research uncovered a striking issue: accuracy dropped by 30% to 50% when models were presented with situations where “none of the above” was correct. The AI tended to select suboptimal or incorrect answers instead of recognizing the absence of sufficient data. It highlights the critical need for AI to not only answer questions but also to acknowledge its limitations.

This capability becomes vital in applications such as e-commerce, where providing inaccurate product return policy information could lead to disputes. Similarly, when a gaming company seeks to understand local market nuances, failing to consider linguistic factors can lead to misunderstandings of consumer preferences. Thus, the ability to recognize these gaps is paramount for effective decision-making in business contexts.

To address these challenges, Appier implemented advanced training methods known as Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to enhance the models' recognition of limits. The results were promising, with the DPO method alone improving accuracy by approximately 30 percentage points. This shows that targeted training can indeed strengthen AI's capability to recognize when it cannot provide a valid answer, thus enhancing the reliability of the decision-making process.

The Role of Language in AI Reasoning

While overcoming information gaps is critical, the language in which AI operates is equally significant. Appier's research delved into how multilingual capacities could improve AI reasoning. The findings underscored that reasoning in the right language is essential for cultural understanding and effective decision-making. Often, even when prompted in another language, AI models reverted to high-resource languages like English, which skewed their reasoning processes.

Using a technique called