Scaling Agentic AI with Arm AGI CPU: 2x More Work per Rack Over Leading x86 Processor
-
Russ Fellows
An independent analysis of the Arm AGI CPU performance running modern enterprise and agentic workloads.
Agentic AI is fundamentally changing the role of the CPU in modern data centers. Beyond model inference, AI agents continuously execute tools, access data, invoke services, and operate within isolated environments. As these workloads scale, CPU performance is increasingly measured not only by how many agents can run simultaneously, but also by how much useful work those agents can complete. With power and space constraining AI infrastructure growth, the rack is clearly a critical unit of performance, where the ability to complete more agentic tasks within a fixed power envelope becomes a key measure of infrastructure efficiency.
Arm is one of the key companies at the forefront of this transition. Leading cloud providers are increasingly building AI infrastructure around Arm-based CPUs, from AWS Graviton, Google Axion, and Microsoft Azure Cobalt, to NVIDIA Vera. Building on this foundation, the AGI CPU is Arm’s first production silicon, purpose-built for agentic AI, combining high-performance Neoverse V3 cores, substantial memory bandwidth per core and high-speed I/O with rack-level density.
To evaluate these capabilities, we worked with Arm to assess AGI CPU performance against AMD’s 5th Gen EPYC 9755 across seven CPU workloads spanning agent execution, tooling, retrieval and model-serving support. The study combines widely used infrastructure components with emerging agentic benchmarks, including Terminal-Bench 2 and SWE-bench, to examine both individual processor performance and the benefits of scaling within a fixed rack power envelope. This analysis provides enterprise technology leaders, architects, and IT decision-makers with evidence to consider when selecting server architectures. The main findings are:
- The Arm AGI CPU delivers 2x the work per rack. Across the seven workloads, the AGI CPU delivers an average of 1.99x the modeled performance per rack of the EPYC 9755 with individual workloads ranging from 1.43x to 3.19x.
- Rack-level density is the key driver. Arm AGI’s focus on efficiency and scalability enables compute density at the rack level. Arm’s rated-power model places 21 to 23 AGI CPU servers in a 24 kW rack against 13 EPYC 9755 servers: 5,712 to 5,888 physical cores against 3,328. The density factor is a ratio of server power, so it holds at any rack power level.
- Higher rack-level performance across agentic workloads. Arm AGI CPU leads across all three agentic performance categories in modeled rack-level performance, averaging 1.7x for tooling, 2.0x for end-to-end agentic workflows, and 2.2x for reasoning/planning, while operating at a lower rated server power than AMD EPYC 9755 (950 to 1,050 W vs. 1,700 W per two-socket server).
- Lower sandbox creation latency. Against a two-socket AMD EPYC 9965, the AGI CPU created sandboxes faster on all three sandbox runtimes reported in Sandbench (runc, Firecracker and gVisor), taking 45% less time for cold starts, 27% less for warm starts and 41% less in the worst case under 4x oversubscription, reducing the overhead of agent environments.
For organizations, that means about half the racks for the same CPU work on web serving, agent replay and model-serving overhead (0.50 to 0.53 racks per EPYC 9755 rack), and 30% to 35% fewer racks for database and vector search. Equivalently, it means about twice the agentic CPU work completed per rack in the same power footprint.
Rack results are modeled from Arm’s rated server power and the measured per-chip results; Appendix A details the model. These results reflect early AGI CPU systems; Signal65 plans follow-on testing on final production hardware across a broader range of workloads, including Signal65 PINNACLE CPU-centric benchmarks, and additional competitive comparisons.
Research commissioned by:
