OpenAI Shifts Habitat to Rust, Cutting CPU Use 6x as ChatGPT Storage Hits 70 Million RPS
Updated
Updated · OpenAI · Sep 11
OpenAI Shifts Habitat to Rust, Cutting CPU Use 6x as ChatGPT Storage Hits 70 Million RPS
1 articles · Updated · OpenAI · Sep 11
Summary
Q2 2026 marked OpenAI’s full Habitat rewrite in Rust by two engineers using Codex and GPT-5.5; the new service now handles 95% of production traffic and will replace Python entirely within weeks.
6x better CPU efficiency and 15x better memory efficiency drove the migration, along with lower average and tail latencies, after Python had become too costly at scale despite earlier helping Habitat reach more than 20 million requests per second.
Habitat now serves more than 70 million requests per second, stores over 500 petabytes of data, and supports OpenAI products used by more than 1 billion people weekly across nearly 40 regions.
The platform began in 2024 as a Python client library over Azure Cosmos DB, then became a centralized service in 2025 to avoid brittle multi-service rollouts, tighten security controls, and manage outages and latency more reliably.
OpenAI said the Rust migration is one part of a broader storage overhaul; a second post will detail multi-tenancy, read-performance tuning, and its Azure Cosmos DB scaling strategy.
Why did OpenAI intentionally endure severe Python bottlenecks before finally deploying their highly efficient Rust upgrade?
How did just two engineers and AI rewrite a 500-petabyte storage system handling 70 million requests per second?
Could AI tools like GPT-5.5 completely eliminate the need for massive engineering teams in future infrastructure migrations?
How OpenAI Scaled Habitat to 70 Million RPS: The Inside Story of a Python-to-Rust Rewrite Powered by AI
Overview
OpenAI’s explosive growth, driven by ChatGPT’s massive user base, caused a 10x surge in PostgreSQL database load and exposed the limits of their client-side Habitat implementation. After a major incident triggered by a viral product launch, OpenAI migrated write-heavy workloads to sharded systems and decoupled Habitat into a standalone service. In 2026, two engineers, aided by advanced AI models, rewrote Habitat in Rust, achieving dramatic gains in CPU and memory efficiency. This transition, supported by innovations like connection pooling and caching, enabled OpenAI to deliver faster, more reliable services at scale, setting a new standard for AI-driven infrastructure modernization.