In the latest MLPerf Inference v5.0 benchmarks conducted by MLCommons, NVIDIA’s Blackwell B200 GPU has showcased a significant leap in performance, solidifying the company’s dominance in AI hardware. The B200 achieved up to four times the inference performance of its predecessor, the Hopper H100, processing 10,755 tokens per second in server inference tests and 11,264 tokens per second in offline tests. This remarkable enhancement is attributed to the B200’s advanced architecture, notably its second-generation Transformer Engine and support for FP4 precision, which doubles the throughput compared to the FP8 used in the H100.
NVIDIA’s GB200 NVL72 system, integrating 72 Blackwell GPUs, also demonstrated substantial advancements, delivering performance 2.8 to 3.4 times faster than previous models, even when compared on a similar GPU count basis. This underscores NVIDIA’s commitment to enhancing AI processing capabilities.
In contrast, AMD introduced its Instinct MI325X GPU during the Advancing AI event, claiming up to 40% better performance than NVIDIA’s H200 GPU in certain benchmarks. The MI325X features 256 gigabytes of HBM3E memory and is designed to handle large AI models. However, analysts suggest that the MI325X may not effectively compete with NVIDIA’s Blackwell GPUs, which are anticipated to offer superior performance. AMD’s stock experienced a decline following the announcement, reflecting investor skepticism about its competitiveness in the AI chip market.
These benchmark results highlight NVIDIA’s continued leadership in AI hardware performance, with the Blackwell architecture setting new standards in inference capabilities. While AMD’s Instinct MI325X presents notable improvements, it appears to fall short of matching the advancements achieved by NVIDIA’s latest offerings.
