TechWhoKnows
Log inSign up
Latest
AI
Startups
Venture
Tech
Security
Hardware
Biotech & Climate
Markets & Policy
Companies
TopicsNewsletterAbout
TechWhoKnows

Independent tech news from Singapore, with the sources linked.

Sections

  • AI
  • Startups
  • Venture
  • Tech
  • Security
  • Hardware
  • Biotech & Climate
  • Markets & Policy

TechWhoKnows

  • Companies
  • About
  • Newsletter
  • All topics
  • Search
  • Account

© 2026 TechWhoKnows. All rights reserved.

  • Privacy
  • Terms
Home/Tech/Enterprise & Cloud

AWS Launches SGLang-Neuron on Trainium3 as CEO Says Cloud Adds an AI Chip Every 14 Seconds

Amazon released a public Trainium inference container built with RadixArk and Annapurna Labs as AWS chief Matt Garman warned inference demand has barely started.

AWS Launches SGLang-Neuron on Trainium3 as CEO Says Cloud Adds an AI Chip Every 14 Seconds

By Jin, Founder & Startups

13 Nov 2026, updated 15 Nov2 min read

Amazon Web Services on November 12 released SGLang-Neuron, a publicly available AWS Deep Learning Container enabling developers to serve frontier AI models and reinforcement learning rollouts directly on custom AWS Trainium3 chips through standard SGLang APIs.

Built jointly by RadixArk and Amazon's in-house silicon division Annapurna Labs, the plugin runs on native PyTorch via TorchNeuron, compiles custom Neuron Kernel Interface (NKI) kernels with torch.compile, and introduces prefill-decode disaggregation over Elastic Fabric Adapter (EFA), according to the AWS engineering post.

Hardware Installation Pace Hits One Accelerator Every 14 Seconds

The custom-silicon push lands alongside fresh commentary from AWS chief executive Matt Garman, who told Sources this month that AWS is now deploying a new AI chip roughly every 14 seconds, an annualised pace approaching 2.25 million accelerators, yet global capacity constraints are likely to persist for at least two more years.

  • Inference inflection: Following roughly 39% year-over-year AWS Q3 revenue growth, Garman argued enterprise AI inference has not yet reached 1% of its eventual market scale.

  • Bedrock AgentCore rollout: To capture stateful agent workloads on top of that compute footprint, AWS highlighted early adopters Itaú Unibanco, Box, Boomi, Epsilon, and Innovaccer testing Amazon Bedrock AgentCore across CrewAI, LangGraph, and LlamaIndex.

Sources

  1. 1AWS Machine Learning Blog: SGLang-Neuron on Trainiumaws.amazon.com
  2. 2AWS Machine Learning Blog: Production-Ready Agents at Scaleaws.amazon.com

More on

  • Amazon
  • AWS
  • Trainium
  • SGLang
  • Matt Garman
  • Cloud Infrastructure

Further reading

Agents & Tools

AWS Says AI Inference Now Drives Over Half of AI Revenue as Amazon Pushes Trainium and Nova Ahead of re:Invent

13 Nov 2026

Enterprise & Cloud

AWS Cuts Bedrock AgentCore Cold Starts 15x With Firecracker Snapshots and Ships Native Web Search API

13 Nov 2026

Enterprise & Cloud

AWS Hits $184 Billion Run Rate as Cloud Giants Split Over AI Platform Strategy

13 Nov 2026

Enterprise & Cloud

Terraform vs OpenTofu vs Pulumi in 2026: Licensing, State, and Multi-Cloud Costs

16 Nov 2026

Follow TechWhoKnows for real-time coverage

Stay updated with the latest developments in technology.

XBlueskyThreadsInstagramLinkedInTikTokYouTube

The TWK Newsletter

Pick your topics and we'll email you our stories on them, with links to the sources.

In this story

Related stories

AWS Says AI Inference Now Drives Over Half of AI Revenue as Amazon Pushes Trainium and Nova Ahead of re:Invent

13 Nov 2026

AWS Cuts Bedrock AgentCore Cold Starts 15x With Firecracker Snapshots and Ships Native Web Search API

13 Nov 2026

AWS Hits $184 Billion Run Rate as Cloud Giants Split Over AI Platform Strategy

13 Nov 2026