#United States #Canton #AMD #Aligned #MEGALODON

Aligned Collaborates with AMD and USC ISI to Advance Next-Gen AI Using MEGALODON Language Model

Aligned Teams Up with AMD and USC ISI for AI Advancement

Aligned, a recognized leader in AI infrastructure and scalable machine learning solutions, has announced a significant partnership with AMD and the University of Southern California's Information Sciences Institute (USC ISI). Together, they will propel the development of MEGALODON, a novel language model designed for next-generation artificial intelligence applications.

A Focused Collaboration

The collaboration aims to harness the computational power of AMD Instinct™ MI300 GPUs, along with Aligned's operational expertise, to efficiently train MEGALODON without relying on NVIDIA architectures. This strategic partnership emphasizes the need to push the envelope in AI performance, reduce computational overhead, and enhance the capabilities of modern language models.

MEGALODON, USC ISI's flagship project, leverages an innovative Moving Average Equipped Gated Attention (MEGA) architecture. This architecture promises improved long-context retention and lowers computational complexity, addressing one of the considerable challenges in training large-scale language models. As dropout rates between tasks increase, optimizing model training is becoming crucial—here, MEGALODON could offer a vital solution.

Bridging Research and Technology

According to Jonathan May, research associate professor at the Viterbi School of Engineering, "As researchers continue to challenge the boundaries of model size and capability, MEGALODON signifies a pivotal advancement in architecture and scalability. This collaboration is instrumental in realizing its full potential."

Aligned plays a vital role in this partnership by optimizing MEGALODON's training cycle on AMD's Instinct MI325X GPUs, enabling USC ISI researchers to exploit the latest AMD GPU compute infrastructure via the ROCm™ platform, a promising alternative to traditional CUDA technologies.

Chris Ensey, CEO of Aligned, expressed enthusiasm about this venture, stating, "We are thrilled to collaborate with AMD and USC ISI to innovate large-scale AI training. Our mission centers on ensuring powerful AI can function across various hardware, and this partnership embodies that commitment."

Innovation Through Collaboration

This initiative not only reflects the technical collaboration among the entities involved but also underscores the broader ecosystem within which ISI operates— encompassing partnerships with organizations like DARPA, NSF, Lockheed Martin, and Chevron. Such collaborations amplify the real-world impact of foundational AI research.

MEGALODON is expected to utilize high-performance computing clusters and advanced storage technologies throughout its development and testing phases, with plans to eventually scale using more advanced AMD GPUs such as the MI355x series.

Key Highlights of the Collaboration

- Innovative LLM Architecture: MEGALODON integrates the advanced MEGA attention mechanism, enhancing scalability and efficiency while managing long-context data.
- Cross-Platform Enablement: This initiative marks one of the first major efforts to train substantial language models on AMD MI325X utilizing the ROCm framework.
- Access to Superior Infrastructure: Partners have shared access to supercomputing resources, high-throughput storage capabilities, and prototype chip labs like MOSIS.
- Institutional Funding: The collaborative work is backed by NSF grants and a commitment from USC to propel open AI research forward.

About Aligned

Aligned specializes in high-performance AI infrastructure, offering advanced tooling, model optimization, and a comprehensive understanding of operations across diverse hardware setups. Their commitment to supporting frontier models and open research holds promise for a more equitable AI future.

About USC ISI

The Information Sciences Institute (ISI) at USC is a globally recognized institute devoted to leading breakthroughs in AI, natural language processing, cybersecurity, and advanced computing. ISI is uniquely positioned within the research and development ecosystem to turn foundational research discoveries into impactful applications.