#USA #San Jose #Data Center #AI Inference #d-Matrix Squadron

d-Matrix Unveils the Squadron for Advanced AI Inference

d-Matrix has officially launched its innovative product, the d-Matrix Squadron, which is designed to revolutionize AI inference in data centers. This robust system comprises multiple racks that provide megawatts of highly efficient inference computing power, catering to the demanding needs of enterprises, AI labs, and government organizations.

Key Features of the Squadron

The Squadron systems leverage d-Matrix's unique memory-centric compute architecture, which enhances the performance of AI applications requiring ultra-low latency, such as personal assistants, cybersecurity measures, and high-frequency trading models. This state-of-the-art solution is already making waves in the tech industry, having been deployed in d-Matrix's Arizona data center for internal R&D and is now shipping to select priority customers.

According to Sid Sheth, the Founder and CEO of d-Matrix, the launch of the Squadron signifies an important milestone as the company transitions from being a niche chip manufacturer to a full-fledged data center compute infrastructure provider. During his keynote address at the OCP Global Summit in San Jose, he expressed excitement over delivering a solution that allows customers to harness the world's most advanced inference infrastructure for AI applications.

Seamless Integration and Versatility

The Squadron is built for scalability and ease of integration into existing data centers. It supports configurations of up to 16 d-Matrix SquadRacks, which are single eight-server racks containing d-Matrix Corsair XPUs. This design ensures that users can create a comprehensive compute cluster with up to 1,024 Corsair XPUs, all interconnected via d-Matrix JetStream NICs, significantly simplifying the deployment process.

One of the standout features of the Squadron is its ability to operate efficiently without requiring liquid cooling, which often complicates data center buildouts. Instead, Squadron's air-cooled architecture allows for direct integration with customers' facilities, minimizing disruption and maximizing operational efficiency.

Advancements in AI Workloads

d-Matrix’s Squadron system is not only adept at running AI models independently on Corsair XPUs but is also optimized for use in heterogeneous computing environments. This approach enables seamless workload distribution between GPUs and XPUs—a technique known as disaggregated compute. During his keynote, Sheth shared significant results that demonstrate the Squadron's capabilities: through a novel Attention-FFN Disaggregation (AFD) method, d-Matrix achieved a processing speed of 845 tokens per second per user, with a total throughput exceeding 20,000 tokens with just 64 concurrent users.

This breakthrough showcases the Squadron's potential to significantly enhance AI processing efficiency and response times, solidifying its position as a leader in infrastructure solutions for AI applications.

Future Plans and Demonstrations

As interest in the Squadron grows, d-Matrix is set to showcase live demonstrations of the AFD technology at their OCP booth during the summit. Attendees will have the opportunity to see firsthand the capabilities of the Squadron systems and learn more about how these advancements contribute to the overall landscape of AI inference technology.

In summary, d-Matrix's Squadron compute clusters represent a leap forward in AI infrastructure, enabling organizations to deploy high-performance inference solutions rapidly. This innovation promises to address the escalating demand for advanced AI processing capabilities across various sectors.

For more information about the d-Matrix Squadron or to explore the Corsair platform further, visit d-Matrix's official website.