HTTP/2, sticky table headers, tracking code fixes

No posts for a few months so I figured I'd group a couple of things into a single post.


Last summer we migrated to Nginx for load balancing and it's been fantastic. A few months later, Nginx released support for HTTP/2, which I've been salivating over ever since. We recently acquired some new servers that are much higher end than the ones we were using before. We migrated load balancing to these and took the opporunity to update Nginx to the latest and greatest while we were at it. This means we're now live on HTTP/2 across the board - our own load balancers, plus our CDN (Cloudflare).

From some tests we ran, the only other analytics service (that we like to compare ourselves to) that supports HTTP/2 is Google Analytics.

Sticky table headers

When you scroll down a report, our navigation tabs have "stuck" to the top of the screen for quite a while now. But the table headers in most of the reports we have would disappear. Not a big deal before, but since we added all that yummy segments data to the reports last April, there's a lot of colums to look at.

Today, we pushed up an update so the table headers will now stick to the bottom of the tabs when you scroll down, as you can see in the screenshot below.

This was tricky, because we use tables for these particular reports. (Don't give us heck about using tables here, this is exactly what they were designed for and is pretty much the only place they're used on our site). You can't just add position:fixed to a table row element, so instead we worked out a solution where we clone the table and remove everything but the first row, and stick that to the top, right below the tabs. It works great!

Tracking code queue encoding issues fixed

When we released heatmaps in Oct 2012, we made a change to our tracking code to queue up certain types of beacons to make them more accurate (javascript events and goals) and more efficient (heatmaps).

The queue is stored in a cookie in JSON format, which is URL encoded, then decoded when the cookie is read. The bug was that there was an extra URL decode in there, which badly broke JSON-parsing when a URL or title had "quote" characters in them. What made it difficult to find and fix was that other areas of the tracking code, I had inadvertently worked around this bug, thinking it was just a feature of browsers that they always decoded cookies when reading them. Alas no, it was just my own damn fault.

This was a very uncommon bug to come across, but for the few people who emailed us about it over the years, it was really nasty. A fatal JSON parse would basically kill javascript entirely which could severely break a site with our code on it. Thankfully this is all fixed now.

White label iframe auto-sizing

This is only of interest to our white label customers since, for security reasons, we don't allow Clicky itself to be included within an iframe - unless it's a white label.

Why do we let white labels use iframes? Because it's simply the best way to integrate the analytics reports into a company's existing site, making it completely seamless. The problem is that you have to manually declare an iframe's width and height, and as someone clicks around pages, the actual height of each page is going to change. Since some of our reports can get pretty tall, the best solution was to just set a ridiculously large height on the iframe so that it would never have to scroll internally.

Good news! Now we have added messaging support to pass a message to the parent document with the height of the iframe document as each new page loads. Full details are here.


There have been a ton of minor tweaks and bug fixes pushed up in the last 3 months as well. Sometimes there's a longer period of time between blog posts than there used to be. Don't worry, we're not slacking. We generally only post when there's a major new feature to talk about. There's lot of things going on behind the scenes at any given time that aren't necessarily of interest to our customers, but these things take away precious time, time that we wish we could spend 100% of writing code.
0 comments |   Jan 15 2016 11:56am

Filter out traffic based on country, organization, referrer, and more!

Some of the most requested features we get are to filter out all traffic from specific organizations, and filter out all traffic from specific countries (or only log traffic from specific countries, for "hyper local" sites). For the countries thing, we used to link people to an old blog post that showed how to do this, however it stopped working a year or two ago when Maxmind killed their javascript API.

Well, no fret! These types of filters, and more, are now officially supported directly from the new "Visitor tags and filters" site preference page.

These are all of the filters you can create from this new filter hub. And like the IP/UID filters of yore, every filter can either be site-specific, or apply globally to all sites in your account.

  • IP/UID - This was what you used to do from this page (and nothing else). Filter out traffic based on IP address or UID (tracking cookie).
  • Pages - Filter out page views that match specific patterns. This used to be part of the main site preferences page (hidden under "advanced"), but we wanted to centralize all filters, so it's here now, and with much better wildcard support than there used to be.
  • Query parameters - Enter in query parameters you want stripped out from URLs (the main purpose of this filter type, to help normalize your content report), with the option to also ignore those page views completely if they contain these query parameters.
  • Countries - Ignore all traffic from specific countries, or only log traffic from specific countries.
  • Referring domains - Full wildcard support with this filter type. Two options here. Either ignore all traffic that comes from specific domains, or, log it and attach the referrer to the visitor but DON'T log the referrer to the top referrer (and related) reports. This second option is kind of weird but everything exists for a reason.
  • Organizations - Filter out all traffic that we resolve to specific organizations/hostnames. Full wildcard support.

Clicky is the best service in the biz when it comes to filtering out robot activity and spam referrers. We are able to be so good at this because you guys let us know about new ones that pop up in your logs. So while this new filter system can be used to filter out bot/spam activity immediately for your own account, we still want you to let us know if any traffic in your logs is suspect, so we can update our global block list to benefit everyone. So don't let this make you lazy!
4 comments |   Oct 21 2015 12:58pm

Nginx and A+ SSL

Back in June, we migrated to Nginx for load balancing, which has been fantastic. I've been wanting this for years and I was so happy when it finally happened.

We're using some older hardware we had lying around, the parts combined for each server cost barely $1,500 when originally purchased. Compare this to the $7,000 we paid for each of our old Kemp load balancers, not to mention the $1000+ support contracts per load balancer we needed for software upgrades and warranty exchanges (which we've had to do 3 times over the years. Of a total of 6 that we've owned, 3 of them have died and had to be replaced. 50% failure rate. Compared to the ~60 servers we've built over 9 years and only had 1 die, ever.) These sub-$2,000 machines can handle more than 3x the load of our $7,000 proprietary load balancers. 3x the performance for 1/4 the cost = 12x more for our money, plus no more support contracts.

Beyond expense and reliability issues, we also wanted to have really great SSL. The kind with SPDY HTTP/2 and an A+ rating. With Kemp, we had none of this.

The final step for an A+ was enabling HTTP Strict Transport Security, which took some consideration because there's really no going back. There were a number of things to double and triple check and some code to update, etc. But as of today, it's live. This feature tells a browser to always connect to a domain via HTTPS, even if a page/link tells it to do otherwise, ensuring all of your interactions with a web site are always secure.

Out of curiosity, I checked all of our competitors' SSL ratings. Only one of them has an A+, and surprisingly it's not Google. Otherwise it's a mix of A, B, and C. When we were on Kemp, we had a C, and now we're A+. From worst to first, oh yeah!

2 comments |   Oct 04 2015 6:28pm

Webhooks and push notifications for alerts

We just pushed some new updates for our alerts system.

First, webhooks. Basically this means we'll ping a URL that you give us everytime that alert is triggered, and you can do what you want with the data we pass through to it. Full details on this are on the alert set up page.

A while back we added push notifications for Safari on OS X. More recently, Google Chrome added push notification support, including on Android, which is killer. We don't have an app so the thought of finally having mobile push notifications was exciting.

We wanted to add support for Chrome notifications pronto, but the security behind push notifications is a flippin' nightmare to say the least.

Then we discovered Roost, which handles all that for us. So we signed up and implemented that into our site, which was much easier than doing Safari manually was. Roost supports Safari OS X, Chrome, and Chrome on Android, and Firefox is "coming soon". Two-thirds of you already access Clicky on Chrome or Safari, and when Firefox support comes out, we'll be looking at almost 90% coverage, so that's pretty great.

The only thing we don't like is that we weren't able to add audio support to these push notifications. Oh well!

So how do you set these up? On the alert set up page is a new checkbox for push notifications, along with a link to manage your push notification devices. Click that link to add your supported browser to your list of devices (if it's not supported, it will tell you). Then go back to the alert set up page and you can select which device(s) to push to for that alert.

Here's what the alerts look like in action. When you click on an alert, you will be taken to the visitor session on Clicky. Enjoy!

A few other things.

1) No iOS push notifications yet. Hopefully either Apple or Google will add them to the iOS versions of Safari or Chrome soon!

2) We killed the old Safari push notification system. That means you'll need to opt-in again on Safari to get alerts.

3) Push notifications work in the background, so even if your favorite browser isn't supported, you can still opt in using a supported browser to get push notifications on your device. When you click them though, it will open in the browser you opted in with (e.g. Chrome).
1 comment |   Jun 15 2015 8:18pm

Clicky Supersize XL Extreme edition

I've been wanting to make Clicky take advantage of larger screens for a long time but there were a lot of challenges involved beyond just changing the width of the site container. (I know that sounds pretty lame but just trust me).

But after last week's release of our new Segments feature, it was clear this needed to be the next major project. Segments are great, but they take up a lot of horizontal space which really crippled a few reports. For example, the Links (referrers) report:

[Note: Click any of these screenshots to see a full sized version]

That just killed me, having every referrer be basically unreadable beyond the domain name. Well, here's what that report looks like now:

Much better, eh?

Here's a summary of what's changed:

  • Instead of a fixed width of 960px, the width is now dynamic with min/max boundaries of 960px and 1200px. We wanted to make sure everything still worked fine with the old width (e.g. this is important for some of our white label customers who have Clicky integrated via iframe), but let people with larger screens take advantage of them.

    I'm not really a fan of sites that go the full width of your browser window, as I feel the readability suffers on really large monitors. I tested various max widths and anything over 1200 just felt wrong. BUT... my opinion on that may change. Feel free to try and convince me.

  • We now have the browser do string-shortening for us, instead of doing it on the backend. This always ensures that the text takes up the full width available, which was extremely important in order to support a dynamic width. Doing it on the backend has also always been a major issue for non-English character strings, as our old shortening code was not "multibyte friendly", so I'm glad that's behind us now!

    This was the major challenge of this release. The basics of it are of course dead simple, but it's the small details that matter. The voodoo magic I had perform to get this to work in all the variety of scenarios I needed it felt like I was jamming a square peg through a round hole over and over (and over) until it finally fit.

  • Spy looks gorgeous. We got a much higher-res map, combined with the wider screen, it's killer. (Screenshots below). We've also moved the controls (e.g. map zoom) to the top left corner so they're in a fixed position instead of moving around when the map size changes.

  • There's not really any "small" text anywhere anymore. Now that we have so much more room, the font sizes (almost) everywhere now are more of a "normal" size, which makes things a lot more readable.

  • The main site dashboard and the user homepage both need some work to take better advantage of all the extra space, but I didn't feel delaying this release was warranted just for that. I wanted to take some time to think up and test different ideas. For example one thing I'm considering is having an option for a third column on the dashboard. So hopefully we can get some updates out to those in the next few weeks.


Here's a few screenshots comparing the old Spy to the new one. Click any of them to see full size!



This was heavily tested with all 5 of the major browsers, and everything is cool, including all the way back to IE7. However there are bound to be a few bugs so please let us know if you experience any layout issues. Screenshots and details of your OS/browser version are of course appreciated.
17 comments |   May 07 2015 3:08pm

Next Page »

Copyright © 2019, Roxr Software Ltd     Blog home   |   Clicky home   |   RSS