Introduction to the cdnjs Architecture Migration
Operating a global open-source content delivery network at scale requires robust infrastructure. As of June 23, 2026, cdnjs runs exclusively on Cloudflare’s Developer Platform. Serving an average of 108,000 requests per second—totaling roughly 9 billion requests daily across more than 330 Cloudflare data centers with a 98.6% cache hit rate—cdnjs is a foundational component of the modern web. It provides immutable, hash-verified JavaScript and CSS libraries directly via <script> tags without requiring API keys, user accounts, or rate limits.
While the serving tier has lived on Cloudflare Workers and KV since 2020, the publishing pipeline relied on an external stack: Google Cloud Platform (GCP) Functions, a git-sync virtual machine, and a massive GitHub repository acting as the source of truth. Migrating this ecosystem end-to-end onto Cloudflare’s native primitives—Workers, Workflows, D1, Queues, Workers Cache, R2, KV, and Containers—eliminated architectural debt, streamlined observability, and pushed critical platform limits higher for all developers.
The Legacy Architecture and Its Pain Points
The previous publishing pipeline was built incrementally over a decade, leaving behind significant architectural friction. A single package update traversed multiple disparate systems, resulting in five distinct pain points:
- No Shared Trace: Updates passed through Google Cloud Functions, Cloud Storage object events, Pub/Sub topics, a git-sync VM, and Workers KV without a unified correlation ID. Google Cloud Logging and Cloudflare Logpush could not be easily joined.
- Split-Brain Storage: Files were written to Workers KV at the edge and a GitHub repository simultaneously. When these dual stores drifted out of sync, no automated reconciliation mechanism existed.
- Object-Event Gluing: The ingestion pipeline used storage buckets as makeshift message queues. A function dropped an archive into a bucket, triggering an event for the next function, lacking dead-letter queues, backlog visibility, and reliable replay capabilities.
- 26 Sharded Functions: Checking npm for updates required 26 separate Google Cloud Functions—one for every letter of the alphabet—each with independent deployments and log streams.
- The GitHub Repository Bottleneck: The git-sync mirror grew past 1.1TB of packed storage. GitHub’s archive generation failed for packages of this scale, forcing maintainers to manage a 274-entry
.gitignorefile to block problematic releases.
Rebuilding the Ingestion Pipeline with Cloudflare Workflows
The new publishing pipeline replaces fragmented cloud functions with Cloudflare Workflows, providing durable execution where state is persisted across every step. Every ten minutes, a cron trigger initiates PackageUpdatesWorkflow to inspect npm and GitHub for new package versions.
For each discovered version, the system spawns a DownloadPackageWorkflow to ingest the tarball directly into R2. Subsequently, a ProcessingWorkflow handles individual file extraction, minification, and compression. Finally, a PublishingWorkflow writes the metadata to KV, asset files to R2, and updates the Algolia search index.
Because Workflows provide built-in durability, any network timeout or transient error causes execution to resume automatically from the last successful step rather than failing outright or corrupting state.
Handling CPU-Intensive Tasks with Containers and Durable Objects
While Workers handle fast, lightweight execution, pre-compressing text-based assets into Brotli and gzip demands intensive CPU resources. To accomplish this without exhausting Worker CPU limits, the pipeline decouples execution using Cloudflare Containers and Queues:
- Each
ProcessingWorkflowwrites uncompressed files to an R2 staging bucket. - The workflow dispatches a job to a Cloudflare Queue and hibernates.
- A Rust compression service running inside Cloudflare Containers pulls the job, compresses the assets, and writes the output back to R2.
- An R2 event notification wakes up the waiting workflow to resume execution.
To coordinate parallel execution across thousands of files within a single package release, the architecture utilizes a lightweight Durable Object as an atomic counter. The parent workflow increments the counter when spawning child tasks, and children decrement it upon completion. Once the counter reaches zero, the Durable Object awakens the parent workflow to proceed with publication.
Modern Storage Architecture: R2, KV, and Workers Cache
The revised storage topology separates operational concerns across specialized data layers:
- R2 Object Storage: Acts as the single source of truth for all file content. Without practical size limits, R2 comfortably houses large bundles, source maps, and font packs that previously strained KV limits. Furthermore, its S3-compatible API allows external mirrors to access the full cdnjs catalog using read-only credentials.
- Workers KV: Retains only lightweight metadata, including package manifests, version lists, and Subresource Integrity (SRI) hashes, optimized for high read throughput and infrequent writes.
- Workers Cache: Replaces the previous custom internal caching layer between the edge and the Worker, integrating directly into Cloudflare’s native Developer Platform.
- DigitalOcean Spaces: Functions as an automated disaster-recovery mirror and live fallback. The serving worker queries DigitalOcean Spaces if R2 experiences downtime, ensuring high availability.
- Subrequest Limits: Packages containing thousands of files rapidly exhausted the standard limit of 1,000 subrequests per Worker invocation.
- Workflow Step Limits: Long-running pipeline operations approached the 1,024 steps per Workflow threshold.
Overcoming Migration Challenges and Raising Platform Limits
Migrating millions of existing files without altering asset hashes presented a major hurdle. Regenerating files via minifiers introduced non-deterministic output changes, which would break SRI hashes pinned by developers across the web. Consequently, the team elected to copy existing assets verbatim from KV to R2 rather than rebuilding them.
This massive data migration exposed hard platform boundaries:
Rather than engineering complex workarounds, the cdnjs team collaborated with Cloudflare’s platform engineers to raise these thresholds globally. Subrequest limits were increased up to 10 million on paid plans, and default Workflow steps were expanded to 10,000 (configurable up to 25,000). These enhancements directly benefit all developers building on Cloudflare’s platform.
Conclusion and Future Outlook
Migrating cdnjs entirely onto Cloudflare’s Developer Platform eliminated reliance on external infrastructure, simplified security posture by removing service accounts and virtual machines, and dramatically improved observability. With a unified architecture running on Workers, Workflows, R2, KV, Queues, Containers, and Durable Objects, the infrastructure is positioned to explore advanced features—such as native ES module transformations on publish—while continuing to serve billions of requests reliably every single day.
Frequently asked questions
What is cdnjs?
cdnjs is a free, open-source content delivery network for JavaScript and CSS libraries used on roughly 12% of all websites. It allows developers to drop a single tag into web pages without signups, API keys, or rate limits.
Why did cdnjs migrate to Cloudflare's Developer Platform?
While serving was already handled by Cloudflare Workers, the publishing pipeline relied on an aging Google Cloud Platform setup, a git-sync VM, and a massive GitHub repository. Moving to Cloudflare's Developer Platform unified the architecture, improved observability, and resolved split-brain storage issues.
How does the ingestion pipeline handle CPU-heavy compression?
The pipeline uses Cloudflare Workflows for orchestration, writing uncompressed files to R2, sending jobs to Queues, and offloading heavy compression tasks to a Rust service running inside Cloudflare Containers.
What platform limits were changed during the migration?
To accommodate the massive scale of cdnjs, Cloudflare raised the Worker subrequest limit from 1,000 up to 10 million on paid plans and increased default Workflow steps from 1,024 to 10,000 (configurable up to 25,000).
Primary reference: Review the original announcement for exact release details. This article is an independent explanation and does not reproduce the source text.
