Storage and Distribution

Once data is content-addressed and peers are connected, the central question becomes: how do you replicate, cache, and incentivize storage across a swarm of self-interested peers?

Swarming

Swarming is the technique of downloading different pieces of a file from different peers simultaneously, turning consumers into distributors. BitTorrent pioneered practical swarming with two key mechanisms:

Rarest-first piece selection: download the least-available pieces first to maximize swarm-wide availability and prevent the "last piece" problem.

Tit-for-tat reciprocity: preferentially upload to peers who upload to you, discouraging free-riding. Combined with optimistic unchoking (randomly uploading to new peers), this creates a cooperative equilibrium.

IPFS uses Bitswap, a block exchange protocol that tracks per-peer ledgers of bytes sent/received and makes decisions about whom to serve based on historical reciprocity. Unlike BitTorrent's piece-oriented swarming, Bitswap operates on individual content-addressed blocks (CIDs), making it more general but less optimized for single-file sequential downloads.

GraphSync is an alternative to Bitswap for transferring large DAG subsets: the requester specifies a selector (a DAG traversal query), and the responder streams matching blocks in a single round trip. Useful for large directories and Filecoin retrieval.

Caches and Pins

Ephemeral caching: nodes cache recently accessed blocks but may evict them under storage pressure. This provides natural "popularity-proportional" distribution—hot content gets cached widely.

Pinning: explicit durable retention; pinned content survives garbage collection. Pinning services (Pinata, web3.storage, etc.) provide persistent availability for IPFS content.

Provider records: nodes advertise what they store by publishing provider records to the DHT. These records have TTLs and must be refreshed; stale records lead to failed retrievals.

Garbage collection: nodes periodically remove unpinned, unreferenced blocks to reclaim space. GC policies balance storage efficiency against availability.

Large Data Strategies

Erasure coding (e.g., Reed-Solomon): encode k data blocks into n > k coded blocks; any k suffice for reconstruction. More storage-efficient than full replication for cold data—you can achieve equivalent durability with 1.5x overhead instead of 3x. Used by Filecoin and Tahoe-LAFS.

Full replication: simpler, faster retrieval, higher storage cost. Preferred for hot or small data where read latency matters.

Locality-aware fetching: prefer nearby peers (measured by RTT or AS distance) for bandwidth efficiency. The DHT may return many providers; the client should rank by proximity.

Sparse replication: download only the parts of a dataset you actually need. Hypercore supports this natively—you can access entry #1000 without downloading entries #1-999.

Disk layout: sequential writes, batch operations, and compaction for performance on both spinning disks and SSDs. Flatfs (IPFS), Badger, and LevelDB are common block stores.

Incentive Mechanisms

Peers need reasons to store and serve data for others. Without incentives, P2P networks suffer from free-riding (consuming without contributing):

Bilateral tit-for-tat: direct reciprocity as in BitTorrent. Simple, effective, but only works when both peers want something from each other simultaneously.

Storage markets: Filecoin creates a marketplace where storage providers commit capacity with cryptographic proofs (Proof-of-Replication, Proof-of-Spacetime) and earn tokens. This decouples storage from simultaneous need.

Bandwidth markets and vouchers: micropayments or credits for serving data. Swarm (Ethereum) uses bandwidth accounting with chequebook contracts.

Altruism and social norms: many networks rely on community goodwill, especially for SSB pub servers, Syncthing relays, and Tor relay operators.

Reputation: PubSub scoring in GossipSub acts as a soft incentive—well-behaved peers get better service. Reputation is a form of identity-based authorization.

How Storage Connects to Everything Else

Storage depends on Content Addressing for verifiable blocks and Overlay Networks for discovery. Security Threats like content poisoning directly target storage integrity. State Sync and CRDTs extends storage with conflict-free mutation. Testing must model churn (nodes going offline) and its effect on data availability. Compliance addresses data retention obligations and the right to erasure in append-only systems. The design patterns of content-addressed networks and rendezvous/trackers directly shape how storage is discovered and accessed.