AGEST Employee Wins Top Award at 2024 Software Quality Management Conference

AGEST Achieves Recognition at the 2024 Software Quality Management Conference

AGEST, a leading company in the enhancement of software quality and safety through advanced technology, has made exciting news as one of its employees received the highest award at the 2024 Software Quality Management Conference, organized by the Japan Science and Technology Agency. This significant achievement underscores the company’s commitment to maintaining high standards in software quality assurance and reflects years of specialized knowledge along with innovative ideas, particularly in the field of generative AI.

The paper, presented by the employee, received acclaim for its insights into optimizing testing processes for generative AI, which is increasingly utilized in systems development, particularly in applications like chatbots. The subject of the awarded paper was "Efficiency Improvements in Generative AI Testing Requiring Repetitive Testing – Investigating Cost Reduction Through Similarity Calculation and Paraphrase Judgment." This title encapsulates a crucial aspect of today’s software testing landscape, highlighting the challenges posed by the large volumes of data and inherent complexities of generative AI outputs.

Paper Overview

In recent years, software development has embraced generative AI technologies, which produce answers and texts at scale. However, the quality assessment of these AI-generated contents is not straightforward due to their vast data output and ambiguous nature. In response to these challenges, the research probed the possibility of utilizing similarity measures for quality evaluation in software testing.

The study employed cosine similarity, based on embedding representations, to numerically assess generative AI output. Through a series of experiments, findings indicated that while there remain challenges pertaining to increased scores due to text length, this approach holds tangible potential for supporting intuitive understanding and practical application in the field.

Researcher Profile

Yusuke Kanemaru is a member of AGEST’s QA Business Division and heads the Hokkaido Group. He possesses the JSTQB Advanced Level Test Manager qualification and is involved in system testing and QA tasks from a third-party verification perspective. Since joining AGEST in 2024, Kanemaru has been pivotal in systematically designing tests utilizing his JSTQB certification while also advocating for practical improvements to testing processes. He is actively engaged in research themes related to AI product testing under the Test Solution for AI Committee and has led the Japan Symposium on Software Testing (JaSST) in Hokkaido since 2021.

In reflecting on the recognition, Kanemaru remarked, "The focus of my research was to enhance efficiency in the iterative testing tasks required for generative AI, particularly by exploring cost reduction strategies through similarity calculations and paraphrase determinations. Winning this award inspires me to further contribute meaningful results that will translate into practical applications. I extend my gratitude to my mentors and team members who supported me through this research."

AGEST’s Commitment

As AGEST continues to set benchmarks in software quality assurance, the company reaffirms its pledge to pursue groundbreaking technological advancements. The objective is to foster not only software quality improvement but also a commitment to societal contributions through these innovations.

"Our vision is to bring rich value and experiences to all DX (digital transformation) initiatives through technology," states AGEST’s leadership. The company actively cultivates QA tech leadership in alignment with next-gen solutions, aiming to significantly impact the development of a highly digital society.

For more information, please visit AGEST’s official website.