Traffic Advice, .well-known/traffic-advice, and how to stop the 404 errors in a Next.js app

In Avocaty, we recently added Better Stack Logs to ingest all logs from Vercel, and I found repeating 404 errors caused by the request https://carta.avocaty.io/.well-known/traffic-advice

What was this /.well-known/traffic-advice request?

It turns out it’s the Chrome Privacy Preserving Prefetch Proxy, a feature of Chrome that prefetches cross-origin content without exposing user information, so that if a user ended up navigating to a prefetched URL, it would load faster.

When you’re responsible for the stability of a web application like me, repeating 404s become an annoying distraction. They make it harder to spot signals of bugs or even hints of malicious usage.

But I couldn’t just blocklist this log line! That’d be too easy. 😄 Being nit-picky as I am, I decided to investigate it, handle it and write this post.

Are you receiving requests at .well-known/traffic-advice?

The traffic-advice file is simply a configuration file to tell proxies if it’s ok to prefetch your site or not. It seems only Chrome’s prefetch proxy is doing it, even though they wrote a draft spec that could be used by other vendors.

There are certain challenges with prefetch proxies and some situations in which you wouldn’t want the page prefetched. This file exists to indicate those proxies what do you want them to do about your site.

Examples in which you wouldn’t want your pages to be prefetched by a proxy:

• The page content depends on the user’s location.

• The website has strong security requirements, and you don’t want to risk exposing the site to the situations mentioned above with compromised TLS certificates.

• The page content has PII, for example, if it’s a user profile, and that data shouldn’t end up cached in proxies.

Otherwise, you can enable it and maybe some users will benefit.

Adding .well-known/traffic-advice to Next.js

Let’s go ahead and explicitly enable Chrome’s proxy to prefetch our site in our Next.js app.

Step 1: add the .well-known/traffic-advice file

For enabling the prefetching of 100% of the cases on all URLs, you can simply add the following content into /public:

[{
  "user_agent": "prefetch-proxy",
  "google_prefetch_proxy_eap": {
    "fraction": 1.0
  }
}]

The file would be in /public/.well-known/traffic-advice. No extension!

Step 2: add the MIME type application/trafficadvice+json

The Chrome team decided not to include an extension on the file for consistency and extra security, so we must update our web server to send back a specific MIME type with the file.

To do it, simply in next.config.js, add a configuration for headers:

// Your Next.js config in next.config.js

module.exports = {
  // ...
	async headers() {
    return [
      {
        // config for Chrome Privacy Preserving Prefetch Proxy
        source: '/.well-known/traffic-advice',
        headers: [{ key: 'Content-Type', value: 'application/trafficadvice+json' }],
      },
    ];
  },
};

Final step: confirm it works!

Once deployed in a preview or production environment, you can verify the setup using this little traffic-advice checkup tool or with curl -i:

curl -i https://carta.avocaty.io/.well-known/traffic-advice

HTTP/2 200
...
content-type: application/trafficadvice+json
...

[{
  "user_agent": "prefetch-proxy",
  "google_prefetch_proxy_eap": {
    "fraction": 1.0
  }
}]

What if you don’t want to be prefetched?

Maybe the content of your website is location-sensitive, has PII, or for some other reason, you just want to opt out.

Then you can ask the proxy not to prefetch URLs from your domain with the following .well-known/traffic-advice file:

[
    {"user_agent": "prefetch-proxy", "disallow": true}
]

References

Thanks very much to jasom for his blog post. He links to all the other resources and explains how to set it up with Apache. You have his full post here: https://www.jasom.net/private-prefetch-proxy-well-known-traffic-advice-apache-htaccess-nginx/

Also learn more here:

• More info about the Traffic Advice file.

• Chrome blog post explaining its private prefetch proxy.

That’s all, folks.

P.S: I find it amazing that we can speed up the entire internet by introducing caching mechanisms outside the infrastructure of the website owners, directly on the browsers and search engines. Like the Google AMP cache was doing as well. How far we’ve come with the World Wide Web.