The New AI Accelerator Economic Landscape
-
Russ Fellows
Without question, one of the leading topics of conversation within the world of IT is that of artificial intelligence, with almost no subject having gained traction faster in the past 30 years. To further support this claim, The Futurum Group’s Intelligence division surveyed over 100 CIO’s from Fortune 500 and Global 2000 organizations.
According to these results, nearly 80% of companies are in the process of conducting AI pilot projects. Futurum’s research asked what challenges these IT leaders were facing, and over 50% listed adopting emerging technologies such as AI as their biggest challenge. Additionally, when examining the key issues driving their IT purchasing, issues including modernization, innovation and AI adoption were all in the top five most cited criteria.
This paper seeks to build upon our earlier analysis of the Intel® Gaudi® 2 AI Accelerator and its potential impact on enterprises. In that study we analyzed reported usage of Gaudi 2, including MLCommons results for standard AI training workloads. In this new research, we detail the results of our hands-on testing of Intel® Gaudi® 3 AI Accelerator vs. a leading competitor while running inferencing workloads on two different Llama 3.1 Large Language Models (LLMs).
To provide meaningful, real-world data points for IT and business executives, Signal65 developed an AI testing platform to run and measure AI workload performance. The Signal65 AI test suite was developed in collaboration with Kamiwaza – a commercial AI inferencing platform. The Signal65 AI test-suite leverages the Kamiwaza stack to accurately measuring AI LLM inferencing performance on various hardware and software platforms.
Key takeaways that are detailed in this report:
- Intel Gaudi 3 performance was similar to Nvidia H100 across a set of LLM inferencing tests
- Intel Gaudi 3 ranged from 15% lower to 30% higher performance than the H100
- Intel Gaudi 3 outperforms H100 for small inputs and large output inferencing sessions, while Nvidia outperforms Intel Gaudi 3 with large inputs and small outputs
- When considering pricing, Intel Gaudi 3 produced more work per dollar than Nvidia H100, across the same set of inferencing workloads
- Intel Gaudi 3’s advantage ranged from 10% up to 2.5x
- Intel Gaudi 3 excelled at combinations of small inputs and large outputs, while having less advantage when processing large inputs and small outputs
