Filter out traffic based on country, organization, referrer, and more!

Some of the most requested features we get are to filter out all traffic from specific organizations, and filter out all traffic from specific countries (or only log traffic from specific countries, for "hyper local" sites). For the countries thing, we used to link people to an old blog post that showed how to do this, however it stopped working a year or two ago when Maxmind killed their javascript API.

Well, no fret! These types of filters, and more, are now officially supported directly from the new "Visitor tags and filters" site preference page.

These are all of the filters you can create from this new filter hub. And like the IP/UID filters of yore, every filter can either be site-specific, or apply globally to all sites in your account.

  • IP/UID - This was what you used to do from this page (and nothing else). Filter out traffic based on IP address or UID (tracking cookie).
  • Pages - Filter out page views that match specific patterns. This used to be part of the main site preferences page (hidden under "advanced"), but we wanted to centralize all filters, so it's here now, and with much better wildcard support than there used to be.
  • Query parameters - Enter in query parameters you want stripped out from URLs (the main purpose of this filter type, to help normalize your content report), with the option to also ignore those page views completely if they contain these query parameters.
  • Countries - Ignore all traffic from specific countries, or only log traffic from specific countries.
  • Referring domains - Full wildcard support with this filter type. Two options here. Either ignore all traffic that comes from specific domains, or, log it and attach the referrer to the visitor but DON'T log the referrer to the top referrer (and related) reports. This second option is kind of weird but everything exists for a reason.
  • Organizations - Filter out all traffic that we resolve to specific organizations/hostnames. Full wildcard support.

Clicky is the best service in the biz when it comes to filtering out robot activity and spam referrers. We are able to be so good at this because you guys let us know about new ones that pop up in your logs. So while this new filter system can be used to filter out bot/spam activity immediately for your own account, we still want you to let us know if any traffic in your logs is suspect, so we can update our global block list to benefit everyone. So don't let this make you lazy!
4 comments |   Oct 21 2015 12:58pm

Nginx and A+ SSL

Back in June, we migrated to Nginx for load balancing, which has been fantastic. I've been wanting this for years and I was so happy when it finally happened.

We're using some older hardware we had lying around, the parts combined for each server cost barely $1,500 when originally purchased. Compare this to the $7,000 we paid for each of our old Kemp load balancers, not to mention the $1000+ support contracts per load balancer we needed for software upgrades and warranty exchanges (which we've had to do 3 times over the years. Of a total of 6 that we've owned, 3 of them have died and had to be replaced. 50% failure rate. Compared to the ~60 servers we've built over 9 years and only had 1 die, ever.) These sub-$2,000 machines can handle more than 3x the load of our $7,000 proprietary load balancers. 3x the performance for 1/4 the cost = 12x more for our money, plus no more support contracts.

Beyond expense and reliability issues, we also wanted to have really great SSL. The kind with SPDY and an A+ rating. With Kemp, we had none of this.

The final step for an A+ was enabling HTTP Strict Transport Security, which took some consideration because there's really no going back. There were a number of things to double and triple check and some code to update, etc. But as of today, it's live. This feature tells a browser to always connect to a domain via HTTPS, even if a page/link tells it to do otherwise, ensuring all of your interactions with a web site are always secure.

Out of curiosity, I checked all of our competitors' SSL ratings. Only one of them has an A+, and surprisingly it's not Google. Otherwise it's a mix of A, B, and C. When we were on Kemp, we had a C, and now we're A+. From worst to first, oh yeah!

2 comments |   Oct 04 2015 6:28pm

Webhooks and push notifications for alerts

We just pushed some new updates for our alerts system.

First, webhooks. Basically this means we'll ping a URL that you give us everytime that alert is triggered, and you can do what you want with the data we pass through to it. Full details on this are on the alert set up page.

A while back we added push notifications for Safari on OS X. More recently, Google Chrome added push notification support, including on Android, which is killer. We don't have an app so the thought of finally having mobile push notifications was exciting.

We wanted to add support for Chrome notifications pronto, but the security behind push notifications is a flippin' nightmare to say the least.

Then we discovered Roost, which handles all that for us. So we signed up and implemented that into our site, which was much easier than doing Safari manually was. Roost supports Safari OS X, Chrome, and Chrome on Android, and Firefox is "coming soon". Two-thirds of you already access Clicky on Chrome or Safari, and when Firefox support comes out, we'll be looking at almost 90% coverage, so that's pretty great.

The only thing we don't like is that we weren't able to add audio support to these push notifications. Oh well!

So how do you set these up? On the alert set up page is a new checkbox for push notifications, along with a link to manage your push notification devices. Click that link to add your supported browser to your list of devices (if it's not supported, it will tell you). Then go back to the alert set up page and you can select which device(s) to push to for that alert.

Here's what the alerts look like in action. When you click on an alert, you will be taken to the visitor session on Clicky. Enjoy!

A few other things.

1) No iOS push notifications yet. Hopefully either Apple or Google will add them to the iOS versions of Safari or Chrome soon!

2) We killed the old Safari push notification system. That means you'll need to opt-in again on Safari to get alerts.

3) Push notifications work in the background, so even if your favorite browser isn't supported, you can still opt in using a supported browser to get push notifications on your device. When you click them though, it will open in the browser you opted in with (e.g. Chrome).
1 comment |   Jun 15 2015 8:18pm

Clicky Supersize XL Extreme edition

I've been wanting to make Clicky take advantage of larger screens for a long time but there were a lot of challenges involved beyond just changing the width of the site container. (I know that sounds pretty lame but just trust me).

But after last week's release of our new Segments feature, it was clear this needed to be the next major project. Segments are great, but they take up a lot of horizontal space which really crippled a few reports. For example, the Links (referrers) report:

[Note: Click any of these screenshots to see a full sized version]

That just killed me, having every referrer be basically unreadable beyond the domain name. Well, here's what that report looks like now:

Much better, eh?

Here's a summary of what's changed:

  • Instead of a fixed width of 960px, the width is now dynamic with min/max boundaries of 960px and 1200px. We wanted to make sure everything still worked fine with the old width (e.g. this is important for some of our white label customers who have Clicky integrated via iframe), but let people with larger screens take advantage of them.

    I'm not really a fan of sites that go the full width of your browser window, as I feel the readability suffers on really large monitors. I tested various max widths and anything over 1200 just felt wrong. BUT... my opinion on that may change. Feel free to try and convince me.

  • We now have the browser do string-shortening for us, instead of doing it on the backend. This always ensures that the text takes up the full width available, which was extremely important in order to support a dynamic width. Doing it on the backend has also always been a major issue for non-English character strings, as our old shortening code was not "multibyte friendly", so I'm glad that's behind us now!

    This was the major challenge of this release. The basics of it are of course dead simple, but it's the small details that matter. The voodoo magic I had perform to get this to work in all the variety of scenarios I needed it felt like I was jamming a square peg through a round hole over and over (and over) until it finally fit.

  • Spy looks gorgeous. We got a much higher-res map, combined with the wider screen, it's killer. (Screenshots below). We've also moved the controls (e.g. map zoom) to the top left corner so they're in a fixed position instead of moving around when the map size changes.

  • There's not really any "small" text anywhere anymore. Now that we have so much more room, the font sizes (almost) everywhere now are more of a "normal" size, which makes things a lot more readable.

  • The main site dashboard and the user homepage both need some work to take better advantage of all the extra space, but I didn't feel delaying this release was warranted just for that. I wanted to take some time to think up and test different ideas. For example one thing I'm considering is having an option for a third column on the dashboard. So hopefully we can get some updates out to those in the next few weeks.


Here's a few screenshots comparing the old Spy to the new one. Click any of them to see full size!



This was heavily tested with all 5 of the major browsers, and everything is cool, including all the way back to IE7. However there are bound to be a few bugs so please let us know if you experience any layout issues. Screenshots and details of your OS/browser version are of course appreciated.
17 comments |   May 07 2015 3:08pm

New feature: Segments!

It's been a while but the wait was worth it! (Yes, we're still here!)

Segments has been in demand FOR. EVER. and we could not be more excited to finally offer this to you. Think of it as the one-off segmentation we already offer, but integrated into nearly every report automatically and with almost no impact on speed. You can also sort the reports by any of these new columns. It's pretty nice!

(A very important note, that this data for your main Content report is vastly different than any of the others, but for good reason. This is fully explained here.)

One reason we avoided doing this for so long is that we didn't think we could make it happen without a drastic hit on performance. But a lot of creativity went into this on the backend and the results are great. One of the aspects of the great performance is data sampling for higher traffic sites. Don't worry, we only do it for high traffic sites and/or large date ranges. It's explained in full detail on the new Segments knowledgebase article.

If your site has more than 10,000 page views in the date/range being viewed, sampling takes effect on all data past the last 10,000 page views. By default it's 25% but Pro Plus and higher members (Upgrade) can choose a few other options, including 100% if you really want it. You can change this setting on your user preferences page, or using the new sampling menu, next to the trend menu, so you can change it quickly as needed as you hop between sites.

We plan to make this new feature available via the API as well, but first we want to give it a day or two and see the impact this on our resource usage once thousands of people are using it at the same time. Just playing it safe.

There are various other tweaks and small changes with this release, but probably only one that many will notice.

Our old "family style" reports, e.g. browsers grouped by family (Firefox, etc) had to go the way of the dinosaur. They were a really nasty hack and anytime we made global updates to reports like this, it was always a huge challenge to get those changes into those family style reports. We've never had an update to all reports as big as this one though, so I just had to declare bankruptcy on that code and move on.

But I felt good about it, because the backend API changes I had to make for this new data structure allowed a lot more flexibility so this worked out well and it's fast.

You can still view individual families. You'll just have to use the new menu system I built into these reports, as you can see in the screenshot below. This affects Browsers, OS', Hardware, Campaigns, Custom data -- I killed the search engines one since there were only a few categories anyways, and if you use split tests, those are also not grouped by family anymore because even people who do use that feature tend not to have more than one or two total.

That's all for now!
8 comments |   Apr 26 2015 9:21pm

Next Page »

Copyright © 2015, Roxr Software Ltd     Blog home   |   Clicky home   |   RSS