Uber's engineer write-up traces the evolution of its centralized ML platform, Michelangelo, across three phases from 2016 to the present. It details the shift from tree models to deep learning, including the move from Spark to Ray-based trainers, elastic Horovod, and Triton model serving, along with the platform changes that followed. It also covers model quality tiering, Canvas for code-driven pipelines, and the Gen AI Gateway for LLM access.
Uber engineering blog
Loading
Uber describes Shard Manager, an open-source, general-purpose sharding service that separates shard ownership from application logic using explicit, load-aware placement. The post covers its design against alternatives like Apache Helix and Ringpop, the greedy rebalancing algorithm with smoothing, thresholds and cooldowns, and Cadence Matching production results including a 38% CPU reduction on the hottest host. The authors also outline migration coordination challenges and planned extensions...
Uber's Presto team describes replacing the JVM execution engine with Velox, a vectorized C++ engine, covering C++ tooling and observability gaps, Parquet compatibility bugs, and a two-phase routing rollout. The post reports shadow-tested benchmarks showing roughly 2.4x faster queries and about 3x lower CPU use, and lists the remaining gaps.
Uber's Secure AI team describes a defense-in-depth framework for AI agents that spans provisioning-time validation, runtime enforcement, and continuous assurance. The post covers behavioral enforcement beyond traditional authorization, the latency and false-block tradeoffs of runtime guardrails, and the use of small specialized models and per-agent calibration. It also shares lessons on developer adoption and the limits of generic controls.
Uber's data infrastructure team describes improving the efficiency of its large Apache Spark fleet, including Gluten/Velox native vectorized execution that cut runtime and cost on migrated production workloads and the shuffle-strategy trade-offs they measured. The post also benchmarks the Quanton Spark engine against OSS Spark for Apache Hudi ingestion, reporting 4.7x to 5.5x steady-state write speedups with methodology, scaling results, and remaining bottlenecks.
Uber's engineering team describes how it evolved its ads exchange from a product-specific serving stack into a generalized marketplace platform. The post covers a supply, marketplace, and demand separation, the AdSpec abstraction, an extension-based request model, page-level APIs, and fleet isolation using configuration-driven environments. It closes with lessons on shifting from features to capabilities and matching infrastructure boundaries to organizational ones.
Uber Freight describes a declarative fragment composition framework that lets dozens of teams contribute independently to a shared OpenSearch index built from heterogeneous microservice data. The post covers the index and fragment data model, bootstrap via Spark and live ingestion via Flink CDC, and annotation-driven patterns for search contracts, linked entities, and 1:N folds. It also explains how the design supported a migration from SIA to OpenSearch.
Uber announced the company-wide rollout of ChatGPT Work, an agentic assistant built on OpenAI's ChatGPT Enterprise, expanding from a 1,000-employee pilot to more than 30,000 staff. The post covers which tools it connects to and which models power it, but offers no architecture, implementation, or measured results.
Uber's translation service serves millions of translations per second from memory, and this post explains how it cut GC overhead and memory use by replacing sync.Map with a value-based map and moving most data to disk. It covers a 6-byte hashed key scheme with measured collision rates, 256-shard locking, time-based global LRU eviction, and compressed disk offsets. The author reports over 90% lower initial memory and a 99.9% hit ratio with sub-100 microsecond p99 miss latency.
Uber's engineering team describes how it normalizes CPU usage across a heterogeneous stateful server fleet using normalized compute units (NCUs), with per-technology performance factors derived from production telemetry. The post reports results for Kafka, including roughly 8% fewer allocated cores and a narrower utilization spread, and explains why stateful leader-follower workloads need this correction for reliability.
Uber engineers describe the evolution of a legal redlining agent from RAG to feedback-driven retrieval, tone refinement, and agentic clause modification. The post covers its Word add-in architecture, hybrid rules and learning-based decisions, time-weighted feedback sampling, vector search, and measured reductions in contract review time.
Uber describes MCP Gateway, a centralized platform for discovering, governing, and executing more than 800 MCP servers and 5,000 tools. The architecture combines an AutoCrawler control plane, protocol translation across HTTP, gRPC, and TChannel, and built-in authorization, redaction, and observability for scaling agent integrations.
Uber describes a feature-logging framework that makes inference-time features the canonical source for model training, eliminating training-serving skew. The post covers selective logging, compact feature aliases, Flink stream joins, state management, profiling, and Kafka scaling techniques that improved freshness, consistency, and throughput.
Uber Engineering explains how error ownership makes retries context-aware by distinguishing errors caused by a service from downstream failures it merely propagates. The production system limits cascading retry storms across deep service graphs, reducing the maximum storm radius from 25 levels to 3 and preventing up to 9.5 million unnecessary requests during an outage.
Uber engineers explain how they automatically map fail-close and fail-open dependencies across a large service mesh. The approach correlates inbound and outbound failures through middleware, aggregates every request with metrics, and applies probability thresholds to identify reliability-critical call paths.
Uber explains how smart event correlation adds context to observability signals, helping teams reduce alert fatigue and turn noisy alerts into actionable insights. The post connects operational monitoring practices with lessons from ultra-distance racing.
Uber Eats engineers detail how they cut search latency in half by optimizing the full-stack pipeline, including retrieval, hydration, ranking, ad serving, serialization, and presentation. The post explains measured techniques such as request hedging, product-level grouping, parallel rendering, GPU serving, and agent-assisted performance optimization.
Uber explains how it evolved from a single scaling controller to a multi-orchestrator Kubernetes control plane using a dedicated ServiceScale CRD and controller. The post details lessons from stale informer caches, terminal status fields, multi-writer races, automated healing, and large-scale rollout testing.
Uber engineers explain how M3DB’s subclustered placement algorithm reduces shard-migration blast radius, limits cross-node dependencies, and enables safer parallel maintenance. The post details its structural invariants, greedy shard-donation strategy, complexity, and operational trade-offs.
Uber engineers explain how they scaled agentic coding usage 7x while stabilizing AI spend. The post details benchmark-driven model routing, prompt-cache tuning, CLI-based MCP access, code-mode batching, context graphs, and session-level cost analytics.