Cloudflare's analytics traffic nearly doubles every year. How it took 70 GB/s off the pipeline

Cloudflare's data pipeline carried about two billion events a second in May 2026, and the volume behind it was growing about 90% a year (doubling every…

Loading

cloudflare·2026-10-15·PT2M7.4S

video loads only when you press play

Cloudflare's data pipeline carried about two billion events a second in May 2026, and the volume behind it was growing about 90% a year (doubling every…

// the short version

Cloudflare's data pipeline carried about two billion events a second in May 2026, and the volume behind it was growing about 90% a year (doubling every 12.9 months by Cloudflare's fit) while hardware grows linearly. In mid-May the Data team halved one internal dataset, Ready-Analytics, from about 26 GB/s to 13 GB/s; two months later organic growth had it back at about 20.

// what to take away

Cloudflare's analytics pipeline grows about 90% a year while hardware grows linearly; when one dataset was halved in May, growth had taken back half the savings by July.
Every 2026 change follows one rule: shrink data as early as possible and copy it as late as possible. Logfwdr now sends each event once, and the second region gets its copy from Buftee after sampling, saving about 38 GB/s.
Logfwdr samples the square root of each key, which thins busy keys and keeps rare ones (about 25 GB/s), and rollups are added up at every hop instead of only before ClickHouse (about 7 GB/s): 70 GB/s in all.

What this pipeline is, and why its growth is a problem

Cloudflare's Data team runs the pipeline that carries events from Cloudflare's global network, tens of thousands of servers, to a few central data centers. Those events feed Cloudflare Logs, the SQL API, observability, internal analytics and customer dashboards. In May 2026 it carried about two billion events per second, from customer traffic and from Cloudflare's own internal teams.

The volume grows exponentially and hardware does not. Cloudflare's forecast chart fits the May 2026 trend at 90.3% growth a year, which is a doubling every 12.9 months. New servers arrive at a roughly steady rate, so the gap between what the pipeline has to carry and what the fleet was sized for widens every month unless the data itself is made smaller. Cloudflare says that over five years its improvements have been worth about two and a half years of deferred hardware expansion.

The stops a byte crosses

The pipeline has four stages, and every byte that is not dropped pays at each one:

Logfwdr runs on every server. It is an internal Go service that accepts events from services on that machine (Workers and FL, Cloudflare's front-line proxy, among them) and forwards them in batches.
Receivers in the central data centers take those batches over Cloudflare's backbone, the long-distance network between data centers. They sample high-volume streams and push the result into buffers.
Buftee is Cloudflare's in-house distributed buffering system, organised as named buffers that downstream consumers read from.
Inserters read from Buftee and write into ClickHouse, the column-oriented analytics database Cloudflare relies on for queries.

The backbone costs bandwidth, and every central stop costs CPU and memory. That is why the order of operations matters: a step that removes data saves something at every later stop, and a step that duplicates data costs something at every later stop.

Sending once, and copying late (about 38 GB/s)

To survive losing a whole region, analytics data is stored in two central regions, and ClickHouse in each needs a full copy so queries can fail over instantly. (Data under Customer Metadata Boundaries is the exception: it stays in its designated region.) The simplest way to get two copies was to double-write at the source: Logfwdr sent two full copies of every non-regional analytics event, one to each region, and each region processed its copy end to end.

That was manageable while only a few streams used a second region. Once disaster resilience covered almost all analytics streams, Logfwdr's outbound traffic doubled.

Cloudflare had already solved a version of this for Logpush, which treats the central data centers as one shared pool of receivers and fails over automatically. Analytics now works the same way:

Shared ingest pool. Logfwdr sends each event exactly once, to whichever central receiver is available.
Late replication. The second copy is made as late as possible: data lands in Buftee, and remote inserters stream a copy across to the other region's ClickHouse.
Failure handling. If a region goes down, Logfwdr sends everything to the surviving one. If the link between data centers blips, each cluster keeps ingesting its local share, and the remote inserters catch up from Buftee's retention once the link is back.

On the first datasets moved, this removed about 43 GB/s of compressed transit and added only about 7 GB/s of cross-data-center replication. Cloudflare gives two reasons for the gap. The copy is made after the receivers have sampled the stream, so it carries less. And the central batches are larger and more uniform than the small batches leaving thousands of individual servers, so they compress much better. HTTP analytics fell by about 20 GB/s (more than half), and Ready-Analytics by 43%, from about 28 GB/s to 16 GB/s. Across the pipeline from spring to fall, Cloudflare counts about 38 GB/s saved.

Square-root sampling, in Logfwdr as well as the receivers (about 25 GB/s)

Sampling already ran in the receivers. The change was to also run it in Logfwdr, on every server, so less data crosses the backbone at all.

The algorithm, as Cloudflare describes it:

Group by key. Each dataset has a key, chosen by the team that produces the data, used to index and sample it.
Shuffle. The records for each key are shuffled, so the sample is not biased by arrival order.
Keep the square root. From each key's group, the sample keeps a number of records equal to the square root of the group's count.
Scale back up. Estimates are scaled by the sample rate. Keeping 10 of 100 records is a rate of 10/100, so each kept record counts for 10 and the estimated total is 10 × 10 = 100.

records   kept   counts as   estimate
    100     10          10        100
 10,000    100         100     10,000

The square root is what makes this safe to apply twice. A busy key is thinned hard and a rare one barely at all, so rare events keep their detail while the most frequent keys stop dominating storage. Cloudflare checked it by comparing rows written to ClickHouse against the estimated totals across keys from the least to the most seen, and found the same estimates whether data was sampled once or twice.

Fitting power laws to that test, rows stored grew with volume at an exponent of about 0.485 when sampled once and about 0.319 when sampled twice. In practical terms, a key with 100 times more data than another used to store about 10 times the rows and now stores about 4 times. In the test, 11.6 million events were represented by 18,000 rows instead of 79,000. (Two independent square-root passes would suggest an exponent of 0.25; Cloudflare notes the real figure is higher because Logfwdr samples per machine and per batch, and the receivers sample after combining some of those batches.)

Applied to three of Cloudflare's top datasets, every one at least halved the compressed data sent to the receivers: about 25 GB/s together, over 2 PB a day.

Adding up at every hop (about 7 GB/s so far)

Some datasets are rollups: aggregates computed at ingest time for a fixed set of questions. Examples from the HTTP dataset are the number of unique IP addresses seen in a minute, and the count of GET, POST, PUT and other requests per domain per minute. Cloudflare describes this as the opposite tradeoff to sampling. Sampling answers any question approximately; a rollup answers a fixed set of questions exactly, as long as the dimensions and metrics are chosen in advance.

These aggregates used to be computed at the end of the pipeline, in the inserters. Nothing about a sum or a count requires waiting that long, because partial results can be combined anywhere. So aggregation now happens at every step, grouped by a key in the same way sampling is:

on each Logfwdr instance,
again at the receivers, which push the re-aggregated events into dedicated buffers partitioned by key,
again in the inserters, because those buffers bring together records from many receivers,
and finally inside ClickHouse, using its aggregating table engines.

The first dataset moved, HTTP traffic, went from more than 9 GB/s to under 2 GB/s of compressed traffic, roughly an 80% cut, even though it is Cloudflare's highest-cardinality dataset. Cloudflare has since built Ready-Aggregates, a one-click aggregating product for internal teams, alongside Ready-Analytics, its one-click analytics product.

What the 70 GB/s and the 2031 projection count

The three changes together removed about 70 GB/s of compressed traffic from Cloudflare's backbone between spring and fall 2026 (about 38 from load balancing, 25 from sampling on the top datasets and 7 or more from aggregation). They also cut CPU and memory in the central data centers. For the Ready-Analytics receivers alone, the first two changes cut usage by about half, freeing about 750 CPUs and 3.5 TiB of memory.

The Ready-Analytics story is the clearest picture of the exponential. Load balancing dropped it from about 26 GB/s to 13 GB/s in mid-May. Two months later organic growth had it back to about 20 GB/s. Sampling then brought it to about 8 GB/s. Cloudflare's reconstruction says that without both changes it would have passed 60 GB/s by September.

Cloudflare's two forward-looking figures are projections, not measurements. Throughput is now about eight months behind the trajectory projected in May, and refitting the trend with the 2026 savings (66.4% a year, doubling every 16.3 months) halves the data volume Cloudflare projects for 2031. Both depend on the growth curve continuing the way the fit assumes.

What came before 2026

The video covers only this year's changes. Cloudflare's post also describes two earlier ones. In 2022 the team split a single monolithic message schema, which forced every internal consumer to ingest every product's fields, into smaller product-specific schemas; even with some fields duplicated, backbone bandwidth fell by 35 to 40%. In 2024 it switched compression from LZ4 to zstd, rolled out first to a few servers to check CPU cost, and cut compressed throughput by nearly 40%, so the same pipe carries more than 1.5 times the uncompressed traffic.

Terms

Logfwdr: Cloudflare's log forwarder, an internal Go service on every server that accepts events from local services and forwards them in batches to central receivers.
Buftee: Cloudflare's in-house distributed buffering system. Receivers write into its named buffers and consumers such as the ClickHouse inserters read from them.
ClickHouse: an open-source, column-oriented database built for analytics queries over very large tables.
Backbone: the private long-distance network between Cloudflare's data centers, the expensive hop in this pipeline.
Late replication: making a second copy of data as late in the pipeline as possible, after it has been filtered and compressed, rather than duplicating it at the source.
Square-root sampling: keeping the square root of each key's record count, so busy keys are thinned hard and rare keys keep most of their detail.
Sample rate: kept records divided by original records; dividing by it turns a sampled count back into an estimate of the original.
Rollup: an aggregate computed at ingest time, such as requests per domain per minute, that answers a fixed question exactly.
Cardinality: how many distinct values a dataset's keys take; high cardinality makes aggregation collapse less.
Ready-Analytics: Cloudflare's internal one-click analytics product, a shared ClickHouse schema many internal teams write into.

Questions

How much traffic does Cloudflare's data pipeline carry?

About two billion events per second in May 2026, according to Cloudflare, growing about 90% a year by its own fit.

Why not just buy more servers?

Hardware is added at a roughly steady rate while the data nearly doubles every year, so capacity bought once is overtaken quickly. When Cloudflare halved one dataset in May, growth recovered half of that within two months. Cuts that are a share of the traffic keep paying as it grows.

Does sampling make Cloudflare's analytics less accurate?

Sampled data gives estimates, scaled by the sample rate, rather than exact counts. Cloudflare's test found the same estimates whether data was sampled once or twice, with rare keys keeping their per-event detail. Datasets that need exact sums and counts use rollups instead of sampling.

Why is replicating late cheaper than writing twice at the source?

The late copy is made after the receivers have sampled the data and after it has been batched into larger, more uniform blocks that compress better. On the first datasets moved, the second copy cost about 7 GB/s instead of the 43 GB/s that double-writing had used.

What is the general lesson for other data pipelines?

Steps that shrink data, such as filtering, sampling and pre-aggregation, belong as close to the source as they can go, and steps that duplicate data belong as late as possible. That framing is ours; the post describes each change in its own terms.

// further reading

Rewinding the exponential ↗: Cloudflare's post, with the charts behind every number here.
Cloudflare incident on November 14, 2024 ↗: how Logfwdr and Buftee fit together, from the incident that stressed them.
ClickHouse AggregatingMergeTree ↗: the kind of aggregating table engine the last stage uses.

// transcript

Cloudflare's analytics data nearly doubles every year, and its hardware doesn't. In May, its data team halved the traffic from one internal dataset, and by July, growth had taken back half of what they'd saved. So this year they took 70 GB/s off the network, with three changes built on one idea. The data starts on every Cloudflare server, where a log forwarder collects events, like each request served, and ships them over long links to receivers in a few central data centers. The receivers sample the busiest traffic and buffer it, and from there it's written into ClickHouse, an analytics database. Every byte costs something at each hop it crosses, so the idea is to shrink data as early as you can, and copy it as late as you can. Copying comes first, because analytics data is kept in two regions in case one goes down, so the forwarder used to send each event twice over those long links. Now it sends one copy, and the second is made at the very end, after sampling has thinned it out, which took about 38 GB/s off the network. Sampling itself moved up next, from the receivers onto every forwarder. Records are grouped by a key, and the forwarder keeps the square root of each group, so a hundred records become ten and ten thousand become a hundred, each one standing in for the ones dropped, and the estimated totals don't move, which saved another 25 GB/s on three of the biggest datasets. Some datasets need exact totals instead, like how many GET requests a domain served each minute, and those used to be added up right before ClickHouse. Now every hop adds up what it holds, so three events that each say one travel on as one event that says three, cutting another 7 GB/s so far. Together that's 70 GB/s, which puts the traffic about eight months behind where it was heading, and since each cut grows with the traffic, Cloudflare now projects half as much data for 2031. The traffic will keep doubling either way, so the job is making sure there's less of it to double.

// source

This explainer is based on Rewinding the exponential: How we keep hardware usage down by Cloudflare ↗. The original reporting and technical work belong to its publisher.