Federated Content Routing: A Three-Tier Architecture for Resilient Content Routing in IPFS

Paper · IEEE ICC 2026

On April 15, 2025, a cascading failure in IPFS's main content index made billions of files effectively impossible to find, even though the data itself was still stored all over the network. My IEEE ICC 2026 paper argues the root cause was architectural, and proposes a design that keeps lookups as fast as a central index while surviving the failures that take one down.

The trade-off nobody wanted

Production IPFS stacks drifted to the central indexer for one reason: latency. In the paper's measurements, a vanilla Kademlia lookup has a P50 of 450 ms and a P99 of 3.8 seconds; the central indexer answers in 45 ms and 300 ms. No product team ships a 4-second tail. But choosing the indexer means accepting that when it goes down, content routing goes down with it.

DHTs also struggle in cloud deployments for reasons that have little to do with the algorithm:

• Churn. Autoscaling and rolling updates keep changing who is in the network, and lookups fail while routing tables reconverge.
• Cold start. A fresh node needs minutes of DHT traffic before its routing state is useful, which defeats elastic scaling.
• Economics. A DHT assumes altruistic peers. Production infrastructure has to be provisioned, paid for and kept reliable.

The paper's argument is that the two systems conflate two jobs with very different requirements. Serving queries needs to be fast and never wait on coordination. Changing routing state (who provides what, who is in the network) needs to be correct, even when some participants misbehave. Pull those apart and you don't have to pick one property.

Three tiers

• Tier 1, edge functions. Stateless workers at CDN points of presence answer the popular lookups. Popularity is Zipfian (a small set of CIDs gets most of the traffic), so aggressive caching makes most queries fast, with hits served in under 50 ms.
• Tier 2, regional BFT clusters. Each region runs 3f+1 replicas of a Byzantine fault-tolerant control plane (HotStuff-like), so it stays correct with up to f faulty or malicious replicas. Consensus is used only for changes: membership and advertisement updates. It never sits on the query path.
• Tier 3, the global DHT. Plain Kademlia, intentionally slow, as a universal fallback and for bootstrapping. Regions publish summaries of their routing state into it, so even a region that has never seen a CID can find out which region does.

The key move: query snapshots, not consensus

Running consensus on every lookup would make queries as slow as commits. Instead, each region periodically cuts a snapshot of its routing table, every 10 seconds or every 10,000 updates. The snapshot (a root hash, an epoch number and a timestamp) is signed by a quorum of replicas with a threshold signature and pushed out to the edge through CDN object storage.

That gives the edge a property it otherwise couldn't have: it is untrusted but verifiable. An edge cache can serve stale data, but it can't forge a quorum signature, so it only caches entries that carry one, and clients check the attestation. A lookup looks like this:

lookup(cid):
  (snapshot, signature) = edge_cache.get(cid)
  if present and verify_threshold(snapshot, signature, quorum):
      return snapshot.providers
  # miss, or a cache entry that fails verification
  snapshot' = tier2.get_snapshot(cid)          # regional API
  verify and cache snapshot' for TTL (30 s)
  return snapshot'.providers                    # Tier 3 on error

The design rests on four invariants. Every answer is backed by a quorum-signed regional snapshot. Each region's advertisement history only moves forward, with one writer per epoch. During a reconfiguration, reads see either the old state or the new one, never a mix. And provider revocations become visible within the 30-second freshness budget.

Changing the system without breaking it

A content-routing network changes shape constantly: regions are added, clusters resized, software upgraded. The paper formalizes an SLO-gated reconfiguration protocol that holds three budgets the whole time: P99 latency ≤ 300 ms, error rate ≤ 0.1%, and freshness ≤ 30 s.

1. Dual-write. Updates go to both the old and the new configuration while queries keep reading from the old one.
2. Progressive traffic shift. Traffic moves to the new path in steps of 5, 10, 25, 50, 75 and 100%. If any SLO is violated for more than a minute, it rolls back one step automatically.
3. Drain. The old path is decommissioned only after at least 10 minutes of stable SLOs.

Cryptographic fence tokens make sure edge caches only serve entries from epochs at or after the cutover, which is how the "never a mix" invariant holds in practice.

The megaprovider problem

IPFS has matured into a network where a few large services announce billions of CIDs from relatively few peers. DHT provider records expire after 24 hours, so a provider has to re-announce everything it stores at least daily. When one provider has more records to announce than the DHT servers can absorb, the pigeonhole principle guarantees saturation, and once re-announcing everything takes longer than 24 hours, content starts dropping out of reach.

In the federated design, announcements flow into the regional clusters as batched advertisement chains with monotonic head pointers, are committed once through consensus, and then fan out: signed snapshots of hot CIDs to the edge, periodic summaries to the global DHT. Announcement volume is absorbed by Tier 2, query latency is handled by Tier 1, and Tier 3 guarantees that nothing is ever unreachable just because an upper tier is down.

What it runs on

The implementation is deliberately cloud-native: Kubernetes operators for lifecycle, Envoy for traffic shifting, PebbleDB and DynamoDB for state, Kafka between tiers, and Prometheus and Grafana for observability. Tier 1 runs on Workers and Lambda@Edge. Tier 2 runs as StatefulSets with 7 replicas per region (tolerating f = 2), serving lookups directly from its database instead of traversing a DHT. Tier 3 is Kademlia with FullRT acceleration. Content itself is served over Bitswap from stateless nodes behind load balancers, so the whole pool looks like one provider and scales without state migration.

Results

Evaluated across three regions (four for failover tests) against the central indexer, vanilla Kademlia and Kademlia with FullRT, with workloads replaying Zipf-distributed query traces and advertisement streams:

Normal-operation latency essentially matches the central indexer: the edge cache removes the multi-hop overhead that makes DHTs slow. The difference shows up when things break. Lose the edge and the central indexer has nothing to fall back on; the federated design reroutes to the regional clusters and stays under a second at P99. Under cascading failures, where each failure makes the next more likely, isolation between tiers and SLO-gated changes keep availability at 99.2%. Ingestion scales linearly to 500 million advertisements per day, and per-query cost comes out at roughly half that of the centralized alternative.

The paper is explicit about its limits: the availability model depends on assumptions about how correlated failures are across tiers, cache hit rates depend on how skewed popularity is, cloud pricing changes, and economic attacks are out of scope.

The takeaway

Decentralized and fast aren't opposites. They just can't be the same component. Keep consensus off the read path, make the fast path verifiable instead of trusted, and keep a slow path that is always there. That combination gives CDN-like latency without a single point of control or failure.

→ Read the full paper on IEEE Xplore