Cloudflare's new Local Uploads feature for R2 significantly reduces cross-region write latency by decoupling the initial data ingest from background replication. This architecture allows clients to write data to the nearest Cloudflare PoP, making objects immediately accessible while asynchronous tasks handle replication to the bucket's home region, improving performance for globally distributed users. The system leverages Cloudflare Durable Objects for consistent metadata management across its distributed infrastructure.
Read original on InfoQ ArchitectureCloudflare R2 Local Uploads addresses the critical challenge of high latency in cross-region object storage writes, a common problem in globally distributed applications. By introducing a new architectural pattern, R2 optimizes write performance without requiring users to change their bucket's primary region. This is particularly beneficial for applications with users spread across various geographies, where fast and reliable uploads are paramount.
The core of Local Uploads' design is the separation of the initial data ingest (upload) from the eventual data replication to the target bucket's region. When a client enables Local Uploads and performs a write, the data is first written to Cloudflare's storage infrastructure in the client's nearest region (PoP). Simultaneously, the object's metadata is published to the bucket's primary region. Cloudflare then asynchronously handles the background replication of the physical object data from the local PoP to the bucket's home region using queued tasks and automatic retries.
Key Architectural Benefit: Immediate Consistency
A crucial aspect of this design is that the object becomes immediately accessible and consistent after the initial local write completes, even before the background replication finishes. This is achieved by decoupling physical storage from logical metadata and utilizing a global metadata service built on Cloudflare Durable Objects.
Cloudflare Durable Objects play a pivotal role in maintaining consistency and reliability across this distributed system. Durable Objects provide a single, consistent place to store and manage data, even as applications run globally. In the context of R2 Local Uploads, they power the global metadata service, ensuring that once an object is locally written and its metadata published, it's immediately visible and accessible worldwide, regardless of the physical data's replication status.
It's valuable to compare R2 Local Uploads with similar offerings from other cloud providers. While AWS S3 Transfer Acceleration also aims to reduce upload latency, it employs a different mechanism. S3 Transfer Acceleration uses CloudFront edge locations for optimized routing over AWS's network to the S3 bucket, but it does not perform local writes with asynchronous replication in the same manner as R2 Local Uploads. This highlights a fundamental difference in architectural approaches to solve the 'distance problem' for object uploads.
This architectural innovation demonstrates a practical approach to building highly performant and globally distributed storage systems, emphasizing trade-offs between immediate consistency, eventual consistency, and network latency.