'Media searches' and other new stuff

One of our most requested features has been to segment out image searches from "normal" searches. We have just added this, so you should start seeing a new entry in your traffic sources report:

This feature has been requested countless times, as far back as June 2008. We're calling it "media searches" instead of "image searches", since we want to be as general as possible and may add support for video searches or other types of media in the future. Image searches from Google, Yahoo, and Bing are supported at the moment. We've also renamed "Direct / Bookmark" to just "Direct", and "RSS readers" to "Syndication".

Other new features and bug fixes from today:
  • Windows phone 7 detection (will show up as just "Windows phone" though)
  • The new "Google Web Preview" bot is now blocked by default as requested
  • AOL recently moved to "q" as the default search parameter in their search engine, so their searches were just showing up as links in Clicky. We've updated our code to support this new parameter. It still supports the old ones too (query and encquery) in case those are still used anywhere.
  • In October, we added support for google.com/aclk referrers, which is what we see when someone clicks on a paid search on Google. However, there's been a bug where these were referrers were showing up in your main "links" report, which normally only shows non-search referrers. We've fixed this bug, which was probably quite annoying for those of you running paid searches on Google.

That's all for today! You should start seeing us go back to our regular feature release schedule, where we release new stuff almost every week. The last few months have been insanely hectic from the massive redesign and tons of traveling and a big partnership integration we'll be announcing soon. We are excited to get back into the groove of writing lots of code because that's what we live for.
12 comments |   Dec 06 2010 5:29pm

ClickyTouch for iPad available now

Steve has updated ClickyTouch to version 1.2, which is now a universal bundle (single application) that takes full advantage of the iPad's larger screen real estate when installed on an iPad. New features include a sidebar menu that's always visible (on the iPad), an inline browser for viewing referrers, the ability to choose a date other than "today", and a few new reports.

If you already have an old version of ClickyTouch installed, simply update it in the app store to get the new features. If you don't have it yet, you can download it here.

Please note this is a third party application, not commissioned by Clicky, so Steve is asking a very reasonable $2.99 USD to purchase the app. However, Steve is offering a free copy to 5 people who leave a comment on this post. Simply leave a comment and include your Clicky username or site ID, and we will pick our 5 favorite comments over the next few days and send you your promo code. Note that promo codes only work in the US app store right now, which is most unfortunate, but this is a limitation on Apple's end, not ours!

See ClickyTouch.com and our previous post for more details on the app.

25 comments |   Dec 01 2010 10:51pm

Changes to tracking

Monday was a doozy. But good things have come out of it.

When we added pinging to our tracking code back in April, a few of you raised concerns that the new way we calculate time-on-site values may be misleading, since lots of people use tabbed browsing and open pages in the background, not to view them until minutes or hours or sometimes days later. This has its consequences on our end too: a user who opens a page in the background and isn't actually viewing the page, they are still sending us "pings" for 10 minutes (by default). This basically means for someone not actually using your site, they could still send ~15 overall "hits" to our tracking servers during that time. Big waste of resources on our end.

What happened on Monday was unrelated to pinging, but the pinging does add a lot of network activity to our servers. It accounts for just over 50% of our overall network requests. So when our load balancers crashed on Monday due to overwhelming network activity (theory is that it was related to Cyber Monday craziness), disabling pinging in our tracking code helped us calm the storm quickly. Pinging has been disabled since approximately 12pm PST on Monday, which is why your bounce rate and your time-on-site values since then have been off kilter.

We really want to alleviate the load these unnecessary pings put on us, and we want to make your analytics more meaningful too. So we took some suggestions you gave us back in April and integrated them into our new tracking code, which was just deployed about 20 minutes ago.

What changed

Now when a visitor opens a page in a background tab or window, the page view is not logged and the pinging does not start until that page is actually being viewed by the person. If they open it in the foreground, tracking will work as it always has. This change only affects people opening background tabs. We think this will provide much more meaningful data to you, because now we'll only be logging the visit once they start using your web site.

The only downside to this is that if you are comparing Clicky's visitor log to that of another service you may also using on your site, some of the arrival times for your visitors won't match up. For example, IP address may show up in Clicky as arriving at your site at 1:30pm, whereas your other service may show them as arriving at 1:25 instead. When you see something like this, the reason will be because that visitor initially opened your page in a background tab at 1:25pm, but didn't start viewing/using the page until 1:30pm.

Pretend that when this person started using your site at 1:30pm, they looked around at a few pages and left at 1:35. Would you rather we report that they were there for 10 minutes from 1:25 to 1:35 (the old way), when they were really only there and using your site for the last 5 minutes of that visit (the new way)? The new way more accurately reflects this visitor's actual "session" on your site, and hence is a better way to look at things in our opinion.

This is what "analytics" is all about - "analyzing" your data to make it more meaningful. Like how our bounce rate is different than other services, but it's better because it's different. This is another one of those things. We celebrate our differences as the purple cow of analytics, and hope that you do too.

Update: No action is required on your part. The tracking code that your site links to, hosted on our servers, has been updated, so everyone gets it automatically.

Also, after this being live for most of today, the difference on our end is quite noticeable. The load on our load balancers has dropped by almost 15%, and the load on the tracking servers behind them has dropped more than 30%. Yay!

Update 2: Although it worked fine in testing, there is some strange bug with Safari that was causing this browser not to be tracked 100% of the time. Based on our numbers it looks like about 1/3 of Safari visitors weren't getting tracked. We have updated the code so that this new method does not apply to Safari anymore. Since Opera doesn't support document.hasFocus() and Chrome's implementation is wonky (it always returns "true"), this means that basically only Firefox 3+ and IE7+ support the new method. This is more than 2/3 of internet visitors though.

Update 3 (Dec. 8): We have reverted to the old version of our tracking code. While most users/sites experienced no problems, a few experienced drastic differences in visitors logged.
30 comments |   Nov 30 2010 11:41pm

Downtime today and tracking changes

Around 12pm PST, Clicky was 100% offline for about 30 minutes. This means there is a bit of missing data from everyone's stats today.

So what happened? Network activity was much higher than normal, enough to overload our load balancers and cause them to crash. Before that happened, we did notice the site seeming to get slower and slower over a few hours leading up to the crash, and load on our tracking servers (behind the load balancers) was almost twice what it normally is. We initially thought we may have been under attack because the network traffic was so much higher than normal, but this does not appear to be the case.

Someone on Twitter suggested it may be related to increased activity due to so called "Cyber Monday", a shopping holiday invented by the retail industry in 2005 to get you to spend more money. This seems plausible, however I would expect most of the extra browsing on the internet in general today to be on major site's like Amazon - sites that are way too big for us to track so we probably wouldn't be affected by it, in other words.

A similar thing happened back in April when we first added "pinging" support to our tracking code. Pinging is what lets us determine much more accurately than most services your bounce rate and time-on-site values, because our tracking code continually talks to our servers while a visitor sits on one page. Back then, we rolled out this feature over the weekend, but then when Monday rolled around, the extra spike in traffic from these pings was so high that the same thing happened - load balancers went boom. We were pinging too agressively, it seemed. Some quick detective work showed that the pings were accounting for 80% of the traffic we were now logging - a 400% increase, in other words, basically overnight. This was quite a bit higher than we had anticipated, so we made some changes to the code to basically cut in half how many pings our code sends - they now only account for about about 50% of our total incoming "hits".

The point is, making this changed saved our skin then, and today it has done the same thing. After we crashed, we quickly updated the tracking code to disable pinging and threw it on the CDN. Within minutes, the load balancers were happily chugging along again. This does mean for the time being, your bounce rate and time-on-site values will be back to non-awesome mode, but if it's a choice between that and Clicky offline, we choose the former.

This change is only temporary though. We're evaluating how to make it more efficient. One complaint people had when we first released the new pinging is that lots of people open tabs in the background and leave them there for a long time before actually viewing the page. So not only does this make your time-on-site values perhaps higher than they should be, it also leads to increased load on our end to track all these excess pings.

So right now, for our own stats, we're testing a new version of the tracking code that only starts tracking a page view and hence starts pinging when one of the following events occurs:
  • Mouse movement
  • Key press
  • Page scroll
  • Page coming into focus (when a tab is loaded in the background then displayed later, this will fire)

We think monitoring for these events before logging a page view and starting the pinging will lead to much more accurate traffic data, and quite a bit less load on our end. There may be some odd cases where none of the aforementioned events occur on a "valid" page view, but we'd guess they account for less than 1% of your traffic. Of course, if anyone has feedback or ideas on other events we should be listening for, we're all ears.

Ideally we'd always log a page view immediately when it is loaded "in focus" but Javascript does not provide a way to check the current focus status - only to detect events when the focus changes. This is an extremely frustrating limitation but we must live with it.


It turns out there is a new method with HTML5, document.hasFocus(), which is a way to determine on demand whether the current document is in focus. It is not yet supported in Opera, and Chrome always returns "true" even when it shouldn't (bug filed), but Firefox 3+, IE7+, and Safari 5 (at least, haven't tested in Safari 4) all support it properly. The test tracking code we are running on our blog and on getclicky.com has been updated to use this new method instead of relying on other events (mouse movement etc).

Relying on events led to about a 5% drop in visitors being tracked, so we knew this wouldn't work. Somehow, we discovered the hasFocus() method, which has VERY little discussion online. Most people must not know about it yet. But anyways, this is what our new test code does:

  • If the browser supports document.hasFocus()...
    • If the current document has focus at the time the tracking code is executed, a page view is logged immediately and pinging starts. (Since Chrome has a bug where this method always returns true even when it shouldn't, Chrome users will always be logged immediately. All other modern browsers, except Opera, will work properly).
    • If the current document does NOT have focus, we setup an event listener to wait for the "onfocus" event to fire, at which point the page view is logged and pinging starts. This will apply to anyone who opens a page on your site in a background tab or window - we won't log their visit until they actually start viewing your page. This will help alleviate the extra load of "pings" on our end, and will result in more accurate usage data of your web site since we won't log a visitor until they are actually viewing your web site.

  • If the browser does NOT support document.hasFocus()... (currently only Opera, and old, outdated browsers)
    • Page view is logged immediately
    • Pinging starts immediately
    • (This is the way it used to work by default anyways)

We think this way of doing things will work great, and if all goes well from our testing, we will deploy it to our CDN and it will become the new default tracking code.

By the way, pinging is still disabled in our public tracking code. Once this new code is tested and deployed, pinging still start again, and your bounce / time-on-site values will return to their normal state.
29 comments |   Nov 29 2010 6:10pm

ClickyTouch for iPhone

Steve Reynolds has spent the last few weeks developing a native iPhone application called ClickyTouch. It looks pretty dang slick and you can download it right here. Steve's description of the app:

Keep up to date throughout every view of your stats, with self refreshing data such as active users on your site, your bounce rate, average session time and overall comparison percentage versus this time yesterday. In as many cases as possible, data refreshes whilst you look at it! No refresh buttons here!

View the following data in a simple, beautiful summary screen:

Live Active Visitors, Bounce Rates, Time Average, Overall Visitors, Actions, Searches, Bookmarks, Links, Social Media, Goals and Campaigns

We use Android phones at Clicky so we haven't gotten a chance to play with it ourselves, but the beta testers we've seen talking about it on Twitter have all been quite enthusiastic. Let us know what you think.

Update: This is a third party app, not commissioned by Clicky. Hence, Steve is asking a reasonable fee of $2.99 USD for the app. However, all users are able to use it, whether they have a premium account with Clicky or not. Also, the app does ask for your username and password to automatically download and keep up-to-date the sites in your account. One user mentioned they don't want to share that information with a third party app, however we assure you that Steve is completely legit and is not the type of person who would do something evil with that information. He has been a very long time user of Clicky and just wants to spread the love!

49 comments |   Nov 05 2010 9:13am

Next Page »

Copyright © 2019, Roxr Software Ltd     Blog home   |   Clicky home   |   RSS