• 3 min read
Modal says it spun up 1M sandboxes in under a minute
Modal rebuilt its sandbox platform to hit 1 million concurrent sandboxes and tens of thousands of launches per second.
Image: Hacker News
Modal says it has rebuilt its sandbox platform to support millions of concurrent sandboxes and tens of thousands of sandbox creations per second, culminating in a test where it ran 1 million sandboxes concurrently and created all of them in under a minute.
The company says demand has been rising fast, driven by reinforcement learning workloads and increasingly large fleets of agents. Modal already runs millions of sandboxes per day and supports up to 50,000 concurrent sandboxes per customer, but it says its previous architecture was not designed for the next jump in scale.
Modal argues that traditional container orchestration systems struggle at this scale because too many operations grow with the number of containers or nodes. It points to Kubernetes as an example, citing scheduler complexity, heavy dependence on etcd, and node heartbeat traffic that scales with cluster size.
Recommended reading
FBI eyes Nvidia and Google TPU AI systems
Modal says its own original system hit similar limits because sandbox creation and placement depended on global coordination and O(sandboxes) writes to Postgres.
How Modal changed the scheduling path
The new design removes centralized coordination from the critical path. Instead of a single scheduler and a central durable store as the source of truth, workers publish state into a Redis stream, while a horizontally scaled fleet of scheduling servers makes placement decisions from in-memory cached data and contacts workers directly over RPC.
Modal says this leaves sandbox creation with just two network hops and one cheap CPU operation. Sandbox metadata and results are still written to durable storage, but largely asynchronously.
The company says the current Redis-stream approach should remain viable until well over 100,000 workers, and because it does not depend on stream ordering, it could add more streams later if needed.
Building the new platform took months, according to engineer Colin Weld, and required changes across Modal’s backend, worker management stack, and container runtime. One issue: schedulers could push containers so quickly that workers hit rtnl lock contention in the Linux kernel while configuring networking, stretching startup times into tens of seconds until Modal changed its networking setup.
Performance numbers and beta access
In Modal’s benchmark, the company says it could create 1 million sandboxes in under a minute, with the benchmark itself becoming the main bottleneck. It also says per-sandbox responsiveness stayed low even at that scale.
Modal says median sandbox start time on the new system is less than half a second, with scheduling itself now taking only tens of milliseconds. The company says the latency tail is still longer than it wants, and attributes much of that to kernel and network contention when many sandboxes start at once on the same worker.
The new scheduler is already available in Beta, and Modal says it will soon power all sandbox scheduling on the platform.
Enterprise Editor
Marcus follows the money. He covers enterprise software, cloud architecture, and the tectonic shifts in Big Tech strategy. He translates dense earnings calls and complex M&A activity into actionable insights about where the industry is actually heading. If a tech giant makes a silent pivot, Marcus is usually the first to notice.
via Hacker News
