#USA #San Jose #AI Inference #d-Matrix #SquadRack

d-Matrix Introduces SquadRack for AI Inference at Scale

In a significant advancement in AI infrastructure, d-Matrix has announced the launch of SquadRack™, the first ever disaggregated, standards-based rack-scale solution designed specifically for ultra-low latency batched inference. This novel system brings forth a new standard in managing generative AI demands, addressing the needs of cloud providers and enterprises alike.

The Power Behind SquadRack

Showcased at the recent Open Compute Project Global Summit, SquadRack comprises highly efficient server nodes equipped with d-Matrix’s Corsair AI accelerators and JetStream I/O accelerators. These components work hand-in-hand to deliver rapid and sustainable AI processing capabilities. By utilizing industry-standard Ethernet, SquadRack can effortlessly scale to manage hundreds of server nodes across multiple racks, enabling the execution of large AI models at impressive speeds.

According to Sid Sheth, CEO and Co-Founder of d-Matrix, "With SquadRack, our goal is to transform the way customers approach AI inference. By emphasizing high efficiency, low latency, and the use of established standards for deployment, we are setting forth a new era in AI infrastructure."

The system has shown to provide an unmatched cost-performance ratio, boasting up to three times better efficiency than conventional solutions, as well as up to ten times faster token generation speeds. This allows users to deploy Gen AI models with parameters that reach up to 100 billion, all while dramatically reducing deployment times due to its integrated architecture.

Collaboration with Industry Leaders

The launch of SquadRack highlights the partnership between d-Matrix and several industry titans, including Arista, Broadcom, and Supermicro. Each player brings essential technologies to the table:

• - Arista Networks provides a cloud networking fabric that can support the demanding requirements of modern AI workloads, ensuring seamless communication between accelerators over standard Ethernet.
• - Broadcom plays a critical role in enhancing performance through its high-performance PCIe and Ethernet connectivity, allowing for advanced scalability within AI infrastructures.
• - Supermicro contributes its cutting-edge AI server platforms that are specifically designed to integrate with d-Matrix's accelerators, offering an all-in-one solution optimized for AI applications.

As explained by Vik Malyala, President and Managing Director at Supermicro, "Our collaboration with d-Matrix combines the best of both worlds: efficient AI acceleration and high-density server design. Together, we provide a viable path for businesses looking to enhance their AI inference capabilities."

Key Technologies and Features

SquadRack integrates several state-of-the-art technologies:

• - d-Matrix Corsair™ Inference Accelerators: These accelerators utilize an innovative compute-memory integration that provides remarkable throughput and ultra-low latency.
• - d-Matrix JetStream™ I/O Accelerators: Designed for efficient, device-initiated communication, these accelerators ensure rapid data transfer across the network, enhancing overall system performance.
• - Supermicro’s X14 AI Server Platform: This platform serves as the backbone of SquadRack, seamlessly integrating d-Matrix's acceleration technology and advanced networking.
• - Broadcom’s PCIe Switches: Essential for scaling operations within a single server node, they contribute to maintaining high performance during intensive AI processing tasks.
• - Arista Leaf Ethernet Switches: These switches facilitate the robust multi-node communication required in sophisticated AI environments.

Availability and Future Prospects

SquadRack is poised to bring a dramatic shift in how organizations leverage AI technology, promoting a more efficient and cost-effective approach to AI inference. With configurations set to be available for purchase through Supermicro in Q1 2026, businesses can soon access this comprehensive solution for their AI workloads.

As the landscape of AI continues to evolve, d-Matrix remains committed to pioneering advancements that enhance the speed and efficiency of AI deployments. By integrating groundbreaking technologies and forming strategic partnerships, d-Matrix is unlocking new levels of performance and sustainability in the realm of AI inference. For further details, explore d-Matrix's official website for upcoming updates and information on their offerings.