Deploy Open Models in Minutes: Serving AI Agents at Scale on Microsoft Foundry with AMD Instinct™ MI300X GPUs
-
Russ Fellows
A Signal65 study of deployment speed, concurrent agent capacity and cost efficiency for dedicated, private model endpoints.
Enterprise adoption of generative AI continues to outpace the supply of skilled AI infrastructure engineers. Many organizations want to run open-weight models on dedicated capacity, with predictable performance and a model version they control. Until recently, that meant buying accelerators, waiting through long lead times and hiring scarce engineers to configure and tune GPU serving software.
Microsoft Foundry Managed Compute offers another path. It pairs dedicated GPU and infrastructure, like the AMD Instinct™ MI300X GPUs featured here, with curated deployment templates that Microsoft maintains, so a team can stand up a dedicated, production-ready model endpoint without buying hardware, signing a long-term commitment or tuning GPU kernels. In Signal65’s testing, open models on Microsoft Foundry went from catalog selection to a live MI300X endpoint in 14 to 24 minutes, or under an hour for DeepSeek V4.1 Flash (1017) with its one-million-token context, by choosing the pre-configured deployment templates. The 192 GB of HBM3 on each MI300X GPU lets a model as large as gpt-oss-120b run on a single GPU with room for dozens of concurrent agents, which keeps the GPU count and the hourly bill low.
Key Highlights:
From Foundry catalog to a live MI300X endpoint
On four MI300X GPUs running DeepSeek V4.1 Flash (1017) with a one-million-token context
Of tuned self-hosted per-agent speed at full agent load
gpt-oss-120b agent sessions per $10,000 of MI300X time
Signal65 evaluated the service the way a customer would. We deployed open-weight models from the Foundry catalog, used each template exactly as provided, and drove the endpoints with agentic and conversational workloads. To see what a managed service might give up, we then ran the same models and workloads on MI300X servers in our own lab, tuned by our engineers.
This report covers four widely used open-weight models, OpenAI’s gpt-oss-120b, Google’s Gemma 4 31B, Z.ai’s GLM-5.3 Flash and DeepSeek V4.1 Flash (1017). It is organized around three questions: how easily the models deploy, how many concurrent AI agents each supports for a defined workload, and what a practical budget delivers.
Key Findings
- 14 to 24 minutes (under an hour at one-million-token context) from the Foundry catalog to a live endpoint, with teardown in two to eight minutes and billing only while a deployment was live
- No serving configuration required from the customer, because the templates chose the serving kernels, sized memory and context, and set multi-GPU parallelism
- Up to 32 concurrent enterprise agents on a single MI300X GPU running gpt-oss-120b, at 701 output tokens per second
- 128 concurrent enterprise agents on four MI300X GPUs running DeepSeek V4.1 Flash (1017) with a one-million-token context, at 1,723 output tokens per second
- 96 concurrent enterprise agents on four MI300X GPUs running GLM-5.3 Flash, at 1,316 output tokens per second, plus 3,397 tokens per second on short-prompt chat
- 93% to 100% of the per-agent speed of the same models on our tuned MI300X systems at full agent load
- About $0.09 of GPU time per complete gpt-oss-120b agent session
Research commissioned by:


