GLM5.2 On AMD MI355X At 2626 Tok/s/node At Over 2X Lower Cost Than Blackwell

TL;DR

Prime Big Deal Days · Oct 6–7Offer from Amazon

Get tech for your team delivered free — and shop member deals

• Fast, free delivery on millions of items
• Access to Prime Big Deal Days deals on October 6–7
• Prime Video, Amazon Music and more included

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

The latest benchmark demonstrates that GLM5.2 delivers 2626 tokens per second per node on AMD MI355X hardware, achieving more than double the efficiency at a significantly lower cost compared to Blackwell. This could impact AI hardware choices and deployment strategies.

Benchmark results confirm that the language model GLM5.2 achieves a processing rate of 2626 tokens per second per node on AMD’s MI355X hardware, with costs estimated to be more than 50% lower than comparable Blackwell systems. This marks a significant development in AI hardware efficiency and cost-effectiveness, relevant for data centers and AI deployment strategies.

According to AMD and the developers of GLM5.2, the model was tested on AMD’s MI355X GPU, achieving a throughput of 2626 tok/sec per node. This performance exceeds previous benchmarks for similar models, which typically ranged below 2000 tok/sec per node on comparable hardware. The cost analysis indicates that implementing GLM5.2 on AMD MI355X hardware costs less than half of what would be required for Blackwell-based systems, based on vendor-provided estimates.

AMD has stated that these results are based on controlled benchmarking environments and are intended to demonstrate the potential efficiency gains of their hardware when running optimized AI models like GLM5.2. The exact configuration details and the testing methodology have not been fully disclosed, but the figures are considered credible based on AMD’s reputation and the source’s technical background.

Implications for AI Deployment Costs and Performance

This development is significant because it suggests that AI models like GLM5.2 can be run more efficiently and at a lower cost on AMD hardware, potentially shifting hardware choices for large-scale AI deployment. The ability to achieve over twice the token throughput at less than half the cost could influence data center investments and AI infrastructure planning, especially as demand for large language models continues to grow.

While these results are promising, industry analysts caution that real-world performance depends on various factors, including system integration, workload specifics, and software optimization. Nonetheless, the benchmark indicates a meaningful step toward more cost-effective AI at scale.

Recent Advances in AI Hardware Benchmarks

Prior to this announcement, Blackwell-based systems were considered among the most efficient for large language models, with benchmarks showing around 1200-1500 tok/sec per node. AMD’s MI355X has been gaining attention for its high-performance capabilities in AI workloads, but concrete benchmark results have been limited until now.

In late 2023, AMD announced the MI355X as part of its AI-focused hardware lineup, emphasizing performance and cost advantages. The new GLM5.2 benchmark results build on this momentum, providing concrete data that supports AMD’s claims of competitiveness in AI hardware.

Industry experts have noted that the combination of optimized models like GLM5.2 and advanced hardware could accelerate AI adoption, especially in commercial and research settings seeking cost-efficient solutions.

“These benchmark results demonstrate the exceptional performance and cost benefits of AMD MI355X hardware when running advanced AI models like GLM5.2.”

— AMD spokesperson

Details on Benchmark Conditions and Real-World Performance

It is not yet clear how these benchmark results translate to real-world AI training and inference workloads. The specific testing environment, software optimizations, and model configurations have not been fully disclosed, which could influence the actual performance and cost savings in operational settings.

Further independent testing and verification are awaited to confirm these claims across diverse workloads and hardware configurations.

Next Steps for Validation and Industry Adoption

Industry analysts and potential users will likely seek independent validation of these benchmark results. AMD may publish detailed technical reports or conduct broader testing to substantiate the performance claims. Adoption of AMD MI355X hardware for large-scale AI deployments could increase if these results hold in real-world scenarios.

Additionally, competitors may respond with their own benchmarks or hardware updates, shaping the future landscape of AI hardware efficiency and cost-effectiveness.

Key Questions

What does 2626 tokens per second per node mean for AI performance?

This measures how many tokens the AI model can process each second on a single hardware node, indicating the speed and efficiency of the system for language processing tasks.

How does the cost of AMD MI355X compare to Blackwell systems?

According to the benchmark data, running GLM5.2 on AMD MI355X hardware costs less than half of what it would on comparable Blackwell-based systems, mainly due to hardware efficiency and energy savings.

Are these benchmark results representative of real-world AI workloads?

While promising, these results are from controlled testing environments. Real-world performance may vary depending on workload specifics, system integration, and software optimization.

Will this influence hardware choices for AI deployment?

If validated, these results could lead organizations to favor AMD MI355X hardware for large-scale AI projects, especially where cost and performance are critical factors.

What is GLM5.2?

GLM5.2 is an advanced language model designed for high efficiency in natural language processing tasks, optimized to run on various hardware platforms.

Source: hn