-
Fighting AI bot traffic is basically a full time job now
The past six months have been an absolute grind fighting and blocking malicious AI bots from ruining your traffic reports. What started as "some reports of excessive bot traffic" last September has turned into a near-constant escalation, with new bot generations, geographic surges, and increasingly sophisticated evasion tactics. We've been posting about it quite a lot on Twitter as things develop, but it's worth summarizing at this point.
Early September brought an uptick in bot traffic that stood out from the usual background noise. It looked like a new wave of AI crawlers, not the traditional scrapers our existing filters already handle. This prompted us to finally enable Cloudflare's "bad bot" blocking across all our tracking domains, something that we had been resisting for a while:
-
AI, UTMs, and referrers
This weekend, we released some major changes to referrer and UTM parameter processing.
UTM parameters (utm_campaign, utm_source, etc) are over 20 years old now, and were originally designed to track online marketing. Over time though, their usage has become much more casual, as referrers have been slowly reduced to a rotting corpse thanks to Google, who started this whole thing with their "secure search" trial back in 2010.
-
Proxy tracking updates: Cookies, Caching, and X-Forwarded-For
We released proxy tracking back in May, and it's been a hit. But there were a few issues brought to our attention over the months since then, which we spent the last week addressing during Portland's latest Icepocalypse.
You will need to reinstall a fresh copy of your website's backend proxy code to get these updates. These updates are optional, but recommended. This is just an unfortunate reality with this kind of tracking. We don't anticipate these kinds of changes very often, and we always strive for backwards compatibility with any kind of update to code that our users install on their websites, as is the case here.
-
Log ad-block visitors with reverse proxy tracking!
Slow, annoying ads are the top reason I've used ad-blockers for literal decades now. Many users care about the tracking/privacy aspect as well. I personally don't mind being tracked online, but I'm slightly biased.
Analytics used to slow down websites too, so ad-blockers block those services as well. Today, ads are still slow and annoying, but thanks to modern browser tech, analytics is almost completely unnoticeable. Many analytics services these days are also privacy-friendly by default — including Clicky, when we joined the bandwagon in March 2022.
-
IPv6 update
IPv6 support is now live. Yes, this is extremely overdue. 17 years of technical debt, where it was assumed that every IP address was an IPv4 address, combined with a gigantic codebase, made this much more challenging than you would ever expect.
For reasons including privacy, storage size, and avoiding a massive database schema update, we're only saving the last 32-bits of IPv6 addresses – even if you have privacy disabled in your settings. There's a lot more info here about how we handle IPv6, if you care about the technical details.