How to Bypass Cloudflare CDN Caching

In some certain situations, Google AdSense or Ad Exchange, can not serve properly because Google Crawler can not crawl the ads.txt file from a CDN cache, therefore, publishers are required to serve their ads.txt from their original server. This instruction will give you a few easy steps to solve Cloudflare caching problem.

What is a CDN?

A Content Delivery Network (CDN) is a network of geographically distributed servers that work together to deliver internet content quickly, efficiently, and securely. By storing cached versions of website content, like images, videos, JavaScript, and HTML, on multiple servers worldwide, CDNs ensure users receive content from a server closest to their location. This minimizes latency and improves performance.

What is Caching?

Caching is the process of storing copies of files, data, or content in a temporary storage location (cache) to reduce the time and resources required to retrieve it. By keeping frequently accessed data closer to the user or system requesting it, caching improves performance, reduces latency, and optimizes resource utilization.

How to Bypass Cloudflare Caching?

1. Log in to the Cloudflare dashboard ↗ and select your account and domain.
2. Go to Caching > Cache Rules.
3. To create a new empty rule, select Create rule.
4. Enter a descriptive name for the rule in Rule name.
   For example, “ads.txt file“.
5. Set the Field = URl, Operator = equals, value=/ads.txt (must include the slash)
   
6. Choose Bypass cache for cache eligibility
7. Deploy
8. Then check the CF-Cache-Status. The CF-Cache-Status header output indicates whether the ads.txt is cached or not. To investigate cache responses returned by this header, use services like Redbot ↗, webpagetest.org ↗, or a visual tool like Chrome’s Dr. Flare plugin ↗.

Then browse to your ads.txt URL, for example www.domain.com/ads.txt

If the CF-Cache-Status is DYNAMIC, which means the ads.txt is no longer cached by Cloudflare and problem is solved.

By bypassing Cloudflare’s cache for the ads.txt file, you ensure it is always served directly from the origin server and remains compliant with Google that rely on real-time data from the file.