Uber engineering blog

Loading
From Predictive to Generative – How Michelangelo Accelerates Uber’s AI Journey
Uber's engineer write-up traces the evolution of its centralized ML platform, Michelangelo, across three phases from 2016 to the present. It details the shift from tree models to deep learning, including the move from Spark to Ray-based trainers, elastic Horovod, and Triton model serving, along with the platform changes that followed. It also covers model quality tiering, Canvas for code-driven pipelines, and the Gen AI Gateway for LLM access.
Introducing Shard Manager
Uber describes Shard Manager, an open-source, general-purpose sharding service that separates shard ownership from application logic using explicit, load-aware placement. The post covers its design against alternatives like Apache Helix and Ringpop, the greedy rebalancing algorithm with smoothing, thresholds and cooldowns, and Cadence Matching production results including a 38% CPU reduction on the hottest host. The authors also outline migration coordination challenges and planned extensions...
Velox at Uber: Rebuilding Presto's® Execution Engine for Speed and Scale
Uber's Presto team describes replacing the JVM execution engine with Velox, a vectorized C++ engine, covering C++ tooling and observability gaps, Parquet compatibility bugs, and a two-phase routing rollout. The post reports shadow-tested benchmarks showing roughly 2.4x faster queries and about 3x lower CPU use, and lists the remaining gaps.
Securing AI Agents: Lessons From Real-World Deployment
Uber's Secure AI team describes a defense-in-depth framework for AI agents that spans provisioning-time validation, runtime enforcement, and continuous assurance. The post covers behavioral enforcement beyond traditional authorization, the latency and false-block tradeoffs of runtime guardrails, and the use of small specialized models and per-agent calibration. It also shares lessons on developer adoption and the limits of generic controls.
Making Every Core Count: In Pursuit Of Faster Apache Spark Jobs at Uber
Uber's data infrastructure team describes improving the efficiency of its large Apache Spark fleet, including Gluten/Velox native vectorized execution that cut runtime and cost on migrated production workloads and the shuffle-strategy trade-offs they measured. The post also benchmarks the Quanton Spark engine against OSS Spark for Apache Hudi ingestion, reporting 4.7x to 5.5x steady-state write speedups with methodology, scaling results, and remaining bottlenecks.
Designing a Generalized Ads Marketplace Platform
Uber's engineering team describes how it evolved its ads exchange from a product-specific serving stack into a generalized marketplace platform. The post covers a supply, marketplace, and demand separation, the AdSpec abstraction, an extension-based request model, page-level APIs, and fleet isolation using configuration-driven environments. It closes with lessons on shifting from features to capabilities and matching infrastructure boundaries to organizational ones.
Freight Search: Composing a Shared Search Index from Independently Owned Fragments
Uber Freight describes a declarative fragment composition framework that lets dozens of teams contribute independently to a shared OpenSearch index built from heterogeneous microservice data. The post covers the index and fragment data model, bootstrap via Spark and live ingestion via Flink CDC, and annotation-driven patterns for search contracts, linked entities, and 1:N folds. It also explains how the design supported a migration from SIA to OpenSearch.
Uber brings ChatGPT Work to all employees with OpenAI
Uber announced the company-wide rollout of ChatGPT Work, an agentic assistant built on OpenAI's ChatGPT Enterprise, expanding from a 1,000-employee pilot to more than 30,000 staff. The post covers which tools it connects to and which models power it, but offers no architecture, implementation, or measured results.
Translations at Scale: Memory Optimization Techniques That Kept Uber's P99 Under 1ms
Uber's translation service serves millions of translations per second from memory, and this post explains how it cut GC overhead and memory use by replacing sync.Map with a value-based map and moving most data to disk. It covers a 6-byte hashed key scheme with measured collision rates, 256-shard locking, time-based global LRU eviction, and compressed disk offsets. The author reports over 90% lower initial memory and a 99.9% hit ratio with sub-100 microsecond p99 miss latency.
Ensuring Similar Computational Resources to Stateful Workloads Running on a Diverse Server Fleet
Uber's engineering team describes how it normalizes CPU usage across a heterogeneous stateful server fleet using normalized compute units (NCUs), with per-technology performance factors derived from production telemetry. The post reports results for Kafka, including roughly 8% fewer allocated cores and a narrower utilization spread, and explains why stateful leader-follower workloads need this correction for reliability.
Scaling AI in Legal: Building Uber’s Redlining Agent
Uber engineers describe the evolution of a legal redlining agent from RAG to feedback-driven retrieval, tone refinement, and agentic clause modification. The post covers its Word add-in architecture, hybrid rules and learning-based decisions, time-weighted feedback sampling, vector search, and measured reductions in contract review time.
Designing MCP Gateway Uber's MCP Management Platform
Uber describes MCP Gateway, a centralized platform for discovering, governing, and executing more than 800 MCP servers and 5,000 tools. The architecture combines an AutoCrawler control plane, protocol translation across HTTP, gRPC, and TChannel, and built-in authorization, redaction, and observability for scaling agent integrations.
Taming the ML Firehose: Scaling Feature Consistency
Uber describes a feature-logging framework that makes inference-time features the canonical source for model training, eliminating training-serving skew. The post covers selective logging, compact feature aliases, Flink stream joins, state management, profiling, and Kafka scaling techniques that improved freshness, consistency, and throughput.
How Uber Protects Against Retry Storms
Uber Engineering explains how error ownership makes retries context-aware by distinguishing errors caused by a service from downstream failures it merely propagates. The production system limits cascading retry storms across deep service graphs, reducing the maximum storm radius from 25 levels to 3 and preventing up to 9.5 million unnecessary requests during an outage.
Large-Scale Automated Dependency Analysis Across Uber's Service Mesh
Uber engineers explain how they automatically map fail-close and fail-open dependencies across a large service mesh. The approach correlates inbound and outbound failures through middleware, aggregates every request with metrics, and applies probability thresholds to identify reliability-critical call paths.
From Signals to Context: Lessons from Lavaredo Ultra Trail
Uber explains how smart event correlation adds context to observability signals, helping teams reduce alert fatigue and turn noisy alerts into actionable insights. The post connects operational monitoring practices with lessons from ultra-distance racing.
Halving the Time: How Uber Eats Rebuilt Its Search Pipeline
Uber Eats engineers detail how they cut search latency in half by optimizing the full-stack pipeline, including retrieval, hydration, ranking, ad serving, serialization, and presentation. The post explains measured techniques such as request hedging, product-level grouping, parallel rendering, GPU serving, and agent-assisted performance optimization.
Running a Software Factory Efficiently at Uber Scale
Uber engineers explain how they scaled agentic coding usage 7x while stabilizing AI spend. The post details benchmark-driven model routing, prompt-cache tuning, CLI-based MCP access, code-mode batching, context graphs, and session-level cost analytics.