AWS Launches SGLang-Neuron on Trainium3 as CEO Says Cloud Adds an AI Chip Every 14 Seconds
Amazon released a public Trainium inference container built with RadixArk and Annapurna Labs as AWS chief Matt Garman warned inference demand has barely started.

By Jin, Founder & Startups
, updated 2 min read
Amazon Web Services on November 12 released SGLang-Neuron, a publicly available AWS Deep Learning Container enabling developers to serve frontier AI models and reinforcement learning rollouts directly on custom AWS Trainium3 chips through standard SGLang APIs.
Built jointly by RadixArk and Amazon's in-house silicon division Annapurna Labs, the plugin runs on native PyTorch via TorchNeuron, compiles custom Neuron Kernel Interface (NKI) kernels with torch.compile, and introduces prefill-decode disaggregation over Elastic Fabric Adapter (EFA), according to the AWS engineering post.
Hardware Installation Pace Hits One Accelerator Every 14 Seconds
The custom-silicon push lands alongside fresh commentary from AWS chief executive Matt Garman, who told Sources this month that AWS is now deploying a new AI chip roughly every 14 seconds, an annualised pace approaching 2.25 million accelerators, yet global capacity constraints are likely to persist for at least two more years.
Inference inflection: Following roughly 39% year-over-year AWS Q3 revenue growth, Garman argued enterprise AI inference has not yet reached 1% of its eventual market scale.
Bedrock AgentCore rollout: To capture stateful agent workloads on top of that compute footprint, AWS highlighted early adopters Itaú Unibanco, Box, Boomi, Epsilon, and Innovaccer testing Amazon Bedrock AgentCore across CrewAI, LangGraph, and LlamaIndex.
Sources
- 1AWS Machine Learning Blog: SGLang-Neuron on Trainiumaws.amazon.com
- 2AWS Machine Learning Blog: Production-Ready Agents at Scaleaws.amazon.com