AMD has unveiled its next-generation accelerators for synthetic intelligence (AI) inference and coaching, amongst different workloads — claiming that the brand new Intuition MI350 Collection delivers as much as 4 instances the uncooked compute efficiency of its last-generation elements and a whopping 35 instances extra inference efficiency — whereas current Intuition customers will be profit from efficiency beneficial properties by upgrading to ROCm 7.0.
“The world of AI isn’t slowing down — and neither are we,” claims AMD’s Vamsi Boppana, senior vice chairman of the corporate’s synthetic intelligence group. “At AMD, we’re not simply retaining tempo, we’re setting the bar. Our clients are demanding actual, deployable options that scale, and that is precisely what we’re delivering with the AMD Intuition MI350 Collection. With cutting-edge efficiency, large reminiscence bandwidth, and versatile, open infrastructure, we’re empowering innovators throughout industries to go sooner, scale smarter, and construct what’s subsequent.”
AMD has unveiled its Intuition MI350 Collection of AI accelerators, power-hungry petaflop-scale elements for AI coaching and inference. (📷: AMD)
AMD has confirmed two fashions at launch, the Intuition MI350X and Intuition MI355X. The previous options 288GB of Excessive Bandwidth Reminiscence 3E (HBM3E) with 8TB/s of bandwidth and delivers 72 tera-floating-point operations per second (TFLOPS) at FP64 precision rising to 18.45 peta-floating-point operations per second (PFLOPS) at FP6 or FP4 precision with structured sparsity; the latter contains the identical reminiscence specs, which AMD says can run massive language fashions (LLMs) and different AI fashions with as much as 520 billion parameters on-device, however will increase the efficiency to 78.6 TFLOPS at FP64 and 20.1 PFLOPS at FP6/FP4 with structured sparsity.
The corporate is not anticipating customers to purchase only a single card, although: with the ever-growing energy calls for of each coaching and operating next-generation fashions, AMD additionally gives the Intuition MI350X Platform and Intuition MI355X Platform — bundles of eight playing cards, providing a mixed whole of two.3TB of HBM3E reminiscence and peak efficiency of 147.6 PFLOPS and 161 PFLOPS respectively at FP6/FP4 with structured sparsity. Anybody seeking to run such a system will want loads of energy and cooling at hand, although: every MI350X has a thermal design profile of an eyebrow-raising 1kW, with the MI355X upping that to 1.4kW.
Each new and current Intuition customers may also be capable to profit from AMD’s upcoming ROCm 7.0 launch, the corporate says, which can ship a claimed tripling of coaching efficiency and greater than 3.5 instances the inference efficiency of ROCm 7.0. “This achievement stems from advances in usability, efficiency, and assist for decrease precision information sorts like FP4 and FP6,” says AMD’s Anush Elangovan of the upcoming launch. “Additional enhancements in communication stacks have [also] optimized GPU utilization and information motion.”
The corporate additionally guarantees spectacular efficiency beneficial properties for current Intuition customers, by means of the brand new ROCm 7.0 and its assist for decrease precision. (📷: AMD)
Lastly, essentially the most affected person and heavily-funded AI experimenters could wish to grasp hearth till the brand new 12 months, with Boppana teasing the next-next-generation Intuition MI400 Collection due for launch in 2026. “The AMD Intuition MI400 Collection will characterize a dramatic generational leap in efficiency enabling full rack degree options for big scale coaching and distributed inference,” he says, revealing fashions providing as much as 432GB of HBM4 reminiscence with 19.6TB/s of bandwidth and efficiency as much as 40 FLOPS at FP4 with structured sparsity. These shall be provided, he says, in an “AI Rack” system dubbed “Helios,” which mixes the playing cards with AMD EPYC “Venice” CPUs and Pesando “Vulcano” AI community playing cards to kind an all-in-one platform for coaching and inference workloads.
Intuition MI350 Collection playing cards, in the meantime, shall be made obtainable on-demand by means of cloud suppliers and for on-premises use from unique tools producers together with Dell, HPE, and Supermicro. Extra info is out there on the AMD web site.
