#Japan #Tokyo #Todoonada #Digidigi #Chappy Log

Innovative Launch of Chappy Log: A Look into LLM Brand Recall

In an exciting move for technology enthusiasts and brands alike, Todoonada Inc. has unveiled a new online platform called the 'Chappy Log'. This site is designed to provide insights into what large language models (LLMs) perceive when asked about various brands within specific categories. The launch took place today, marking a significant step towards transparency in how LLMs generate knowledge and recall associated brands.

Understanding the Chappy Log

The Chappy Log is not just any ordinary online platform. It serves as a public log that documents the 'LLM Recall Set', which indicates what brands come to mind when users pose questions related to a particular category, such as “What comes to mind when you think of beauty?” Users can ask the same question multiple times, and over time, the results will show a stable set of brands that the model consistently recalls. This visualization will cover 84 specific subcategories within the beauty sector, and the data will be updated each time the model undergoes an update.

The platform employs Todoonada’s unique LLM verification tool, known as 'Digidigi', to measure and log this recall in real time.

Features Accessible on the Site

Users can expect to find several exciting features on Chappy Log:

• - Category Pages: These pages will reveal the frequency and ranking of brands that emerge from asking questions about different categories multiple times.
• - Recall Set Stability: The platform will display the reliability of each recall set using four tiers (solid, strong, moderate, unstable).
• - RAG Fire Rate: It will track how often the LLM resorts to a search tool instead of relying on memory during these queries.
• - Model Comparisons: Users can compare how different models, like GPT and Claude, respond to brand-related inquiries.

Additionally, there's a dedicated product page for each recalled item where the LLM provides a description along with related product suggestions. Users can also benefit from an explanation from the character 'Chappy', who summarizes the unique recall characteristics of each category.

Methodology Behind Measurements

The questions asked on the site will all follow a fixed format of “What comes to mind when you think of ______?”, intentionally excluding prompts like “recommended” or “popular” to keep focus solely on pure recall. The recall set of responses is predetermined during the pre-training phase and remains static until the next model update. Each model update will necessitate a re-evaluation to record and save the actual versions provided by the API.

The RAG fire rate is distinct in that it only tracks instances where the LLM attempts to invoke a search tool rather than executing a search. Thus, it’s an indirect measure of the model's reliability in recalling information without seeking outside assistance.

Future Directions for Chappy Log

Looking ahead, Todoonada has plans to expand the categories available on Chappy Log, conduct recurring measurement updates with each model refresh, and implement RAG fire rate assessments using alternative queries about recommendations and popularity.

About Digidigi

'LLM Digidigi' is Todoonada's dedicated tool designed to assess what specific categories LLMs recall and ascertain the presence of specific brands alongside checking for hallucinations. It's compatible with several models like GPT and Claude and is accessible for a subscription fee of 10,000 yen per month. Users of Qlipper can access it for free. For more information, visit Digidigi.

About Qlipper

Qlipper is a PR measurement tool aimed at public relations professionals. Utilizing keywords, it automatically collects articles and measures the impact of PR initiatives through various metrics like virtual page views and ad equivalence values.

Company Overview

Company Name: Todoonada Inc.
Address: 302 Yahiko Building, 7-11-13, Ueno, Taito-ku, Tokyo, 110-0005
Representative: CEO Yasuhiro Matsumoto
Business Activities: Development and provision of PR Technology (Qlipper/Digidigi)
Membership: Japan Public Relations Association

For inquiries regarding this release, please contact:
Todoonada Inc.
Phone: 03-6453-6886
Email: [email protected]