#USA #Jinan #SentiAvatar #SentiPulse #SUSU

SentiAvatar: The Future of Interactive 3D Digital Humans

In an exciting development for the realms of artificial intelligence and user experience, SentiPulse, a cutting-edge AI company, has unveiled SentiAvatar, a pioneering framework that allows developers to create expressive interactive 3D digital humans. This innovative release is in collaboration with a doctorate team from the Gaoling School of Artificial Intelligence (GSAI) at Renmin University of China (RUC). The framework is now open-source, allowing researchers and developers worldwide to contribute to its evolution.

The Power of SentiAvatar

SentiAvatar is designed to bridge the gap between digital interaction and emotional engagement. Utilizing high-quality avatar SUSU, the framework leverages realistic motion datasets, intricate speech synchronization, and dynamic expressions, making it capable of engaging in real-time conversations. The open-source offering includes the complete framework, the character model of SUSU, and the SuSuInterActs motion dataset, now available on GitHub.

Breaking the Uncanny Valley

A notable focus of SentiAvatar is to tackle the uncanny valley effect, where avatars display movements or expressions that appear unnatural. Traditional digital humans often struggle with expression synchronization; an avatar might gesture, but the body language doesn't align with the conversation's intent. SentiAvatar emphasizes that human communication extends far beyond words, incorporating non-verbal cues like gestures and facial expressions, which are crucial for real human interaction.

Through this innovative framework, SentiPulse has developed an approach to naturally synchronize movement and speech patterns. The limitations of previous attempts stemmed not only from technical challenges but also from inadequate datasets and an unfulfilled need to understand complex human actions.

SuSuInterActs Dataset: The Building Blocks

SentiAvatar employs the SuSuInterActs dataset, meticulously constructed around the character SUSU, who is characterized as a warm and lively 22-year-old. This dataset is a treasure trove of multimodal conversational data, containing over 21,000 clips with 37 hours dedicated to synchronized speech, behavioral annotations, full-body motion, and facial movements. It seeks to fill a significant gap in high-quality Chinese-language datasets for this type of technology.

Advanced Motion Foundation Model

Additionally, SentiPulse's team has pre-trained a proprietary Motion Foundation Model on more than 200,000 diverse motion sequences, totaling approximately 676 hours. This foundation allows the framework to grasp general motion patterns that extend beyond the context of dialogue, thus enabling richer interactions.

Innovative Architecture

The architecture of SentiAvatar features the Plan-Then-Infill dual-channel parallel system. This revolutionary design separates body motion from facial expressions, allowing for meticulous planning of actions followed by precise execution on a frame-by-frame basis. This means that different movements can be choreographed to match spoken content accurately.

Transformative Real-Time Performance

SentiAvatar also sets a new standard for real-time performance. Through state-of-the-art technology, it manages to generate motion sequences of up to six seconds in just 0.3 seconds and supports continuous interaction without creating disjointed responses. This flexibility is essential for maintaining a natural conversation flow, thus providing a solution to the core issues of unnatural expression in digital avatars.

Open Source Collaboration and Future Prospects

In inviting developers and researchers to explore this new frontier, the SentiPulse team emphasizes collaboration. With the framework now open-source, the potential applications are vast—ranging from creating personal 3D companions, enriching gaming experiences, enhancing production in film, or integrating robotics with emotion-infused interaction capabilities.

For those looking to delve deeper, SentiPulse provides access to the framework and technical report on their GitHub page and Arxiv.

About SentiPulse

Founded in 2025, SentiPulse strives to innovate emotional foundation models and improve user experiences where AI and human interaction converge. With a strong team comprised of top-tier researchers from renowned Chinese academic institutions, SentiPulse aims to redefine how we perceive and interact with AI technology, shifting the focus to more meaningful connections rather than mere tools.

SentiAvatar marks a significant leap forward in the creation of lifelike digital humans capable of authentic communication. Through its open-source framework, it sets the stage for a revolution in the landscape of interactive technology.