Elon Musk’s AI start-up, xAI, recently announced that “Colossus,” the world’s most powerful AI (artificial intelligence) training supercomputer, is now online. Based in Memphis, Tennessee, this new AI supercomputer was built in just four months (122 days).
The new xAI training cluster was constructed using 100,000 NVIDIA benchmark Hopper H100 GPUs, a highly in-demand component for training new AI models. This model is designed to train the next generation of xAI’s large language model (LLM), Grok.
The company also plans to double the GPU count to 200,000 in a few months by adding another 50,000 chips of NVIDIA’s new, more advanced H200 series, which offers higher computational memory.
Further, NVIDIA’s new benchmark Hopper H200 GPUs have up to 141GB of faster HBM3E memory, as opposed to the H100, which has up to 80GB of HBM3 memory.
This weekend, the @xAI team brought our Colossus 100k H100 training cluster online. From start to finish, it was done in 122 days.
Colossus is the most powerful AI training system in the world. Moreover, it will double in size to 200k (50k H200s) in a few months.
Excellent…
— Elon Musk (@elonmusk) September 2, 2024
“This weekend, the xAI team brought our Colossus 100K H100 training cluster online. From start to finish, it was done in 122 days. Colossus is the most powerful AI training system in the world. Moreover, it will double in size to 200K (50K H200s) in a few months. Excellent work by the team, NVIDIA and our many partners/suppliers,” Musk tweeted on X (formerly Twitter).
“Exciting to see Colossus, the world’s largest GPU #supercomputer, come online in record time. Colossus is powered by @nvidia’s #acceleratedcomputing platform, delivering breakthrough performance with exceptional gains in #energyefficiency. Congratulations to the entire team!” NVIDIA Data Centre’s X handle wrote on its platform.
The NVIDIA GPU costs around $30,000, suggesting that Musk spent at least $3 billion to build the new supercomputer, a facility that will also require substantial electricity and cooling resources.
Currently, Musk’s Grok 2, xAI’s second generation AI model trained using 15,000 H100 GPUs, is exclusively available to X premium and X premium plus subscribers with image and text generation capabilities. He said it expects to release xAI’s third-generation AI model, Grok-3, by December.
“Hey there! Just so you know, we’re an Amazon affiliate. This means if you buy something through our links, we may earn a small commission. It doesn’t cost you extra and helps keep our lights on. Thanks for supporting us!”
