Databricks' Compute Platform team describes how fixed request-rate limits failed to protect long-running cluster-creation workflows, and how adaptive concurrency limits with slot-based tracking, bounded queues, and fair scheduling let them raise default cluster-creation rate limits by 7.5×. The post covers retry storms, distinguishing system degradation from tenant overload, and reported outcomes including over 99.99% fewer rate-limit errors.
Backend & Distributed Systems
System design at the scale where the design starts to matter — microservices and the monoliths they replaced, event-driven architecture, Kafka and message queues, gRPC and API design. This is where the migration write-ups live: the ones with latency graphs, a rewrite in Go or Rust, and an honest account of what the old system could no longer do.
Loading
Datadog describes how its metrics aggregation system evolved across three generations, from static partition ownership with local RocksDB state, to dynamically assigned consumers with state snapshots in blob storage, to a best-effort streaming layer where durable reconciliation of partial aggregates is delegated to the timeseries storage engine. The post explains the recovery, scaling, and cost trade-offs behind each design and why the final architecture lets the streaming layer resume at the...
GitHub describes the Agent Host Protocol (AHP), an open standard that keeps Copilot agent sessions continuous across VS Code, the GitHub Copilot app, mobile apps, and cloud sandboxes. The post explains why per-client integrations failed, how a URI-channel, reducer-based state design handles reconnection and concurrent mutations, and what was learned from building two independent hosts in TypeScript and Rust. It also covers how the protocol stays agnostic to agent behavior and where to connect...
Lyft describes how Smooth Cruiser, its driver safety scoring feature, measures driving signals relative to local road conditions, regional peer groups, and a rolling window of recent rides. The post also covers the data platform behind the score, including ML-based harsh event detection, an Apache Airflow ETL pipeline, and DynamoDB for low-latency score retrieval, along with transparency features that show drivers the rides behind their scores.
A DuckDB User Group workshop built a pipeline over 2.1 million Amazon appliance reviews to find products with falling ratings. DuckDB SQL selects the largest rating drops and the most helpful recent complaints, and only those 40 reviews go to an LLM for theme summarization. The post also covers running the job on Hugging Face and deploying the workflow as a Gradio app with API endpoints.
Vercel announces eve 1.0, an open-source framework for production AI agents, and details the engineering behind it. Median hosted turn latency fell from 3.04s to 414ms through a single-orchestrator workflow design and WebSocket event writes, and estimated workflow cost for a 50-turn session dropped about 76% by storing only changed conversation data. The post also covers OpenTelemetry-based tracing, multi-tenant agent configuration, and dynamic tool loading.
Vivek Nair describes how a team hardened the Claude Docs backend before its September 2026 launch by running Claude in a long-lived loop against a chaos-testing staging cluster. Key lessons include using copy-on-write to fix per-keystroke memory growth, which cut projected pods from over 500 to about 50, and choosing Rust with minimal dependencies so telemetry traces back to owned code. The launch saw 60 server errors across 2 billion requests with no rollbacks.
Netflix describes the intelligence layer behind its AI Discover TV feature, an agent harness that combines LLMs with recommendation systems. The post covers a plan-and-execute architecture with controller and executor components, validated plans built from composable capabilities, model-agnostic design, asynchronous page generation, and safety rules enforced in code rather than prompts. It is an engineering write-up of the trade-offs and decisions made along the way.
Salesforce engineers describe building AI-Assisted Test Failure Buddy, a Python and FastMCP tool exposing eight MCP tools to Claude Code and Cursor, to triage over 600 test failures caused by retiring the SOAP API login mechanism. The post explains how deterministic code handles API calls, batching, signature matching, and reassignment while AI interprets unmatched stack traces and ambiguous ownership, reducing estimated two to three days of triage to about 90 minutes.
In a podcast recorded at WeAreDevelopers 2026, John Romero, creator of Doom and the WAD file format, discusses the self-modifying code and blitting techniques that let Wolfenstein run quickly on early PCs. He also covers the mod-friendly architecture behind Doom's success and the creative problem-solving behind Quake, arguing that human ideas outperform AI coding agents.
Explains how Databricks Lakebase Postgres uses a tiered architecture so that object storage almost never sits on the query read path. Production stats show 99.46% of page reads hit compute-node caches, with pageservers serving most of the rest from local NVMe, and object storage handling only about 1 in 28 million accesses. The post also covers how image and delta layers keep pageserver replay chains short.
Pinterest Engineering describes rebuilding Guardian, its real-time abuse and risk platform, as Guardian V2 on modern data infrastructure. The post covers the move from a monolithic V1 to separated analytics, signal evaluation, and replay paths using StarRocks, FlinkSQL, and Iceberg, plus stream splitting that cut indexed column scale from O(20K) to O(K).
LinkedIn rebuilt its data lineage platform to capture lineage from executed query plans across more than 15 compute and data-movement systems. The design routes events through Kafka into an append-only log, stores the graph in MySQL with a breadth-first traversal engine, and attaches type, confidence, and granularity to every edge. The post reports a production graph of roughly 6 million datasets with P99 traversal latency near 200 ms.
Presents split-and-scale, a method that increases latent capacity of pretrained 3D tokenizers at inference by applying the same tokenizer to smaller sub-regions of a scene, since 3D scenes are represented as continuous functions of coordinates. The authors ablate compute, memory, latent capacity, and reconstruction quality, and report competitive results against larger models exceeding single-GPU memory, plus an autoregressive 3D generator built on the approach.
Dan Winship of Red Hat explains why the Kubernetes kube-proxy ipvs backend is being deprecated and scheduled for removal in Kubernetes 1.40, covering its original data-plane and control-plane performance motivations and its later compatibility, batch-update, and scheduler limitations. The post recommends migrating to the nftables mode, or to the substantially improved iptables mode on older clusters, and gives concrete configuration guidance for kubeadm clusters.
The post introduces a Firecrawl kit for Docker Sandboxes that gives isolated coding agents live web search, scraping, and crawling through one allowlisted host, api.firecrawl.dev. It explains how the kit's mixin descriptor declares install, credential, phased network-policy, and agent-context capabilities, with the API key injected by a host-side proxy so it never enters the sandbox. It also covers verification steps and known limitations such as proxy 403s and organization policy overrides.
Tailscale introduces netmap caching, a client feature that saves a copy of the tailnet network map to disk so nodes start faster and can still reach peers when the coordination server is unavailable. The post explains the netmap contents, how only changed chunks are written, and how DISCO keys are exchanged through encrypted DERP-relayed WireGuard messages to re-establish direct connections without the coordination server. It also notes the staleness tradeoff over time.
The post explains how LiveRamp's identity resolution runs as a Databricks packaged clean room, where the provider ships code and the customer runs it against its own governed data. It covers the architecture, including read-only OpenSharing access, serverless compute in a central clean room, the separate customer-only and shared output schemas, and how the design avoids exporting PII. It also contrasts packaged and approval-based clean rooms.
Fly.io describes replacing its 6PN private network's fleet-wide phonebook, a BPF map of every migrated Machine kept synchronized via Corrosion, with a lazy design. VXLAN unknown-address events trigger on-demand route lookups by s6pnforwarderd, leaving a table of 100 to 200 routes per host. The post covers fwmark-gated fail-closed filtering, the tradeoff of a roughly one second first-connection latency, and a dual-running rollout across the fleet.
Cloudflare introduces Spend Management in beta, adding opt-in spending caps for Workers and Durable Objects. The post explains the two-stage metering pipeline, the account-partitioned Usage Engine, and the Rating Engine that turns usage into spend. It also describes how caps reach the global network asynchronously through Quicksilver, keeping rating out of the request path.