Path analysis (beta release)

Almost 3 months in the oven, we are finally ready to release a BETA of our path analysis feature. We're releasing as beta because there are still a few quirks and we want your bugs reports and feedback on the interface and what you think it could do better. This is a Pro or higher feature, of course.

Part of what took so long was making it fast, because we know our visitor filtering isn't exactly the fastest thing in the world, especially for high traffic sites over large date ranges. But we are limiting this to a maximum range of 31 days at once for the time being, to make sure resource usage doesn't get too ridiculous, until we know how you are using it.

There are two different features in this release.

Simple path analysis

I like this one because it's quick and dirty and requires zero forethought. When viewing your Content or Events reports (planning to add downloads and outbounds later), you'll see an icon next to each item:

Click this icon (not the star, the other one!) to see an inline popup of the top 30 next and previous pages, for whatever date range you are viewing. If you're interested in seeing top next/previous pages for a page listed in there, click it and it will keep on going. Here's a good example for our upgrade page:

The top previous page is /user/. Makes sense as that's the starting point for most registered visitors, and there are links to upgrade right from that page.

Second previous page is /user/sites - this is where people register new sites on Clicky. If they're at or over their limits for how many sites they can track or how much traffic they're allowed, we prompt them to upgrade first.

Third previous page is /compare/, which is our competitor comparison matrix. From our upgrade page, we have a link to this comparison matrix, so people can compare our pricing and feature set, which we think will help win over more customers since most of our competitors are 400+% more expensive than us (ouch!). So how would this be a previous page? Well, after people look at it, they go back to the upgrade page! Well, some of them anyways :|

The next two aren't horribly relevant but the sixth one, /user/users is. That's where you add additional user accounts. If you're at your user limit or your account doesn't have access to that feature, then we say, hey, you gotta upgrade!

I could go on all day but the point is simply that seeing what pages are actually pushing people towards paying us for our service is extremely interesting. I of course had a hunch about these most popular paths since I designed the entire flow of our web site, but I didn't actually know. Now I do and it's great!

Real path analysis

This other aspect is much more powerful but takes longer to setup. That's to be expected of course, and if you take the time to do so it is pretty fantastic. In the future, we plan to group this together with goals for multi-step funnels. That's not making the initial cut though.

To reach this report, go to Content, then the Path Analysis sub-tab. It gives a brief overview of how to use it. Click the link at the top to start analyzing. Once you enter in some paths and click "run analysis" on the bottom right, it will analyze your input and return a report like the one below. This example is fairly simplistic but I don't want to showcase too many private details of our business here.

Until we integrate this into goals, you can use this as a kind of faux multi-funnel goal system, because you can save paths to quickly access them later.

Important note: In the screenshot above, see how the green arrow on the right has a little "break" in it? This represents "infinite" steps in between each path. In other words, it means the visitor didn't necessary go directly from the first page to the next, but after viewing the first they eventually viewed the second one. This is always the default when adding new paths in this interface. But if you only want to analyze direct path 1 -> path 2 flow, just click that arrow and it will become closed, representing a direct step:

After you make that change, you will have to click the Run Analysis button again. In fact, any change you make in this interface you will have to do that. The reason for this is if we were auto refreshing it with every change you made, it would take a lot longer to create exactly the report you want. So instead, we let you do all the steps you want first, and only process it when you tell us too. This is an important note because it works differently than all other reports on Clicky, which all react instantly to anything you click.

So, play around with it and let us know of any bugs you find or additional features you'd like. Barring any serious bugs, we're going to take at least a few weeks break from this because we could really use it. So we'd expect that the "official" release with some of your suggestions and hopefully all bug fixes would be out in about 1 month.
9 comments |   Mar 06 2012 4:49pm

Database maintenance this weekend

Database servers 1-3, 6-10, and 14-33 will halt traffic processing for up to 10 hours this weekend while we perform some necessary maintenance. The servers will remain online so you will still have access to all of your data, there just won't be any new data processed for up to 10 hours. Most servers will be done much sooner than that, 10 hours is just the absolute max that any of them should take.

This process will start at about 10pm on Friday (USA PST), which is the same time that our normal database backups occur.

When each server finishes, it will begin processing its backlog of traffic that hasn't been processed yet. This means there will be an additional couple of hours for each server before it is caught up with real time again.

The purpose of this is in preparation for transitioning to our new infrastructure. We have invested heavily in new hardware over the last 6 months and we are finally (almost) ready to make it live. This is one of the last steps before we start.

To determine which database server a site is hosted on, take a look at the preferences page for that site.
13 comments |   Mar 01 2012 1:33pm

Custom outbound link pattern matching, and iframe tracking

A lot of people seem to run plugins or custom code on their site that automatically convert links to internal redirects, so that you can count them internally. For example, instead of linking to "" (affiliate link), you might link to "" instead. This page then redirects the person on to the real link.

The problem with this has been that our tracking code has no idea what's actually going to happen once someone clicks that link. Without actually following the link, it simply appears to be an internal link, so we ignore it by default. There have always been several ways to track these links (CSS tagging and manual data logging), however these methods required way too much pain on your end. We don't like pain.

So, we added a new customizable tracking variable today, called outbound_pattern. This allows you tell our tracking code, with just a few lines of code in one place, what your redirect URLs look like so that we can track clicks on them automatically. You can define it as a string, if there's only one pattern we should be looking for (should apply to most of you), or if you have multiple patterns, you can define more than one in an array.

The most common redirect URLs that I've seen are one of these three:

What you'll want to use as your custom outbound pattern is the the base part of these URLs, e.g. '/go/' or '/outbound/' or '/aff/'. You'll want to make sure whatever it is you use though is unique to your redirects, otherwise other links may start getting logged as well.

Single pattern example:
var clicky_custom = {};
clicky_custom.outbound_pattern = '/go/';

Multiple pattern example:
var clicky_custom = {};
clicky_custom.outbound_pattern = ['/go/', '/aff/'];

This code would go directly above your existing tracking code. And wrapped in HTML script tags of course.

If you're running WordPress, our plugin has been updated to support this new functionality as well.

iframe tracking

A while back, we tried to make tracking iframes better, in an automated way. This ended up causing some problems though so we killed it fast. Now we're bringing it back, but it's disabled by default so it will only process if that's what you want it to do.

Here's the problem. If the tracking code is installed in an iframe, we track the iframe, but in most cases your goal is to track the parent document instead. Otherwise the page URLs and page titles you see in our reports will be for the iframe itself, and we won't be able to log referrers.

So, if your goal is to track the parent document but can only put the code in an iframe or "widget" (e.g. with Apple's iWeb), set clicky_custom.iframe=1, so that our code will grab the parent's URL, title, and referrer.

99% of you that have the code in an iframe, your goal is to actually track the parent document. If this applies to you, set this variable, and you should be good to go.
6 comments |   Feb 15 2012 12:03pm

Long term metrics, and other goodies

Our newest update is all about the long term, and it's one of the best things we've ever done. I'd recommend grabbing a fresh pair of pants before reading on because you're going to need them before you're done reading this.

It all started out innocently enough: A fairly simple update to goals that before I knew it had morphed into something else entirely. 80% of the stuff in this update was never planned, all of these ideas just starting coming to me as I kept working on it. "Hmmm this would be sweet, oh this too, oh and this..." Within days my bullet point todo list for this update was ridiculously long and ended up taking almost 3 weeks to complete. But that's the way it usually goes when I feel I'm on to something good.

New goal report

The main goal report still looks the same - until you click on a goal, that is. This new "single goal" report is awesome. Here's an example of one goal we track for new users registering with Clicky:

Talking points:
  • You can now graph all goal metrics (conversions, conversion rate, revenue [not shown], visits to goal, and time to goal), simply by clicking on any of those large-sized metrics at the top of the report. Previously you could only graph conversions. We've had a million requests to graph this other data - now you finally can. We considered graphing all of them at once but they all have vastly different scales and it simply did not work well.

  • "This visit" and "first visit" - Like most non-enterprise level services, Clicky used to only give you "this visit" data, e.g. data from the current session where the visitor converted. While interesting, most visitors do not convert on their first visit to your web site, so this data doesn't show the long term effects of your marketing efforts - example, which campaign they originally came in from when they bought something two weeks later, or what site they originally were referred by, if any.

    And that's exactly what "first visit" is all about. When a visitor converts, we now look up both their current session and their very first session on your site (that we know about it) immediately and summarize it in a table that lets us generate this report really fast, even over large date ranges. I can pull up this new report for a 30 day date range for any of the goals we track in less than 1 second. Not bad!

    Going down these two new columns are five categories (only two are shown here to keep the size reasonable). These categories are campaigns, referring domains, searches, split tests, and goals, which we feel are the most relevant metrics to track in relation to tracking conversions. Previously you could see this data if you filtered your visitors by a goal then chose the appropriate segment, but we wanted to bring these important metrics front and center and make it load super fast.

    Trends are also shown for all data types in this/first visit columns, and you can of course click the red/green percentage next to any of them to graph historical data just like you can with any other report. Of note however is that we've only been tracking this data for 3 days, so there isn't much history yet.

New visitor details page

The old visitor details page did the job, but I used to refer to it as "Old Betty" if that tells you anything about my feelings towards it. I wanted to make it look nice, and add way more details to it. So... I did!

  • Jam packed with details - Click here to see what this report used to look like, for the same visitor. There used to be a lot of wasted white space, with things like one line for browser, one for OS, and one for screen resolution. These are all related, so why not group together? Same with hostname/org, IP + ARIN/RIPE lookups, etc.

  • All time goals - Everytime a unique visitor completes a goal for the very first time, we've logged it to a table for over a year now I think. This was originally so we could calculate the "time to goal" and "visits to goal" metrics, making sure we were only counting each person once per goal. While making this new update, I realized how interesting this data could be. I can't believe we've had it for as long as we have and never shown it to you like this. So, that's what this is. Every unique goal that this unique and beautiful snowflake has ever completed is shown here, in chronological order, with the date shown next to each one. Clicking on that date will take you to the session where they completed this goal!

  • This visit / first visit - Just like the new goal report, we also show this kind of information in the visitor details pain now. So now whenever you're viewing the details for any of your visitors, you can immediately see interesting details from their first visit as well - including how long ago it was (in this example, 288 days and 15 hours ago). Want to see that first visit? Click on the date. Bam.

Oh, you thought we were done?

New visitors report

  • Reorganization - If you use goals or campaigns, icons for those were shown near the left for visitors who had either one attached to them. The problem with this is that most visitors don't have goals or campaigns attached, so this added a lot of empty white space in the middle of the report which always bothered me. So, we moved these icons to the very right hand side of the report. (This example is a bit cluttered because I filtered by visitors who had completed any goal, just to make sure there was lots of things to see).

  • Direct Spy access - In the screenshot above, you'll see a third icon as well next to some visitors. What is this? Oh, it's only the best thing ever! If you're old school, you'll recognize this used to be the icon we had for the Spy report, before we removed the icons because we ran out of space. If you see this eye, that means that the visitor is online now, and you can click the icon to go straight to Spy with this visitor already pre-filtered! (If you don't have a premium account, you don't have Spy, so instead we just show a green dot icon indicating their online status).

    (My only problem here is that the eye icon is available as a choice to you when setting up goals and campaigns. So you might want to change that if you're using it for anything, to prevent confusion. But there is no better icon to represent what this new function does, so I had to use it. Had to.)

  • Quick access to visitor details - So those visitor details I was just telling you about before? Well, there's no need to actually go to the visitor details page anymore, because they're now available directly on the main visitors report. Next to the "actions" link (e.g. "3 actions"), is a grey arrow pointing top right. Click this icon and you'll immediately see the full visitor/session details page in a lightbox without having to go to a new page.


The code that generates graphs when you click on a trend (those red/green percentages) was a mess and had been rewritten multiple times in several places for specific functionality. This is called horrible design. I took the time to rewrite it from scratch and it's way smarter and centralized in one place now, as it should be. Previously this code was unable to graph anything that had a parent/child relationship - for example, the this/first visit columns in the new goal report, that data is stored as "children" of the "parent" goal. I wouldn't have been able to let you graph this without rewriting this awful code. I really had no choice here.

So what's the bonus? Well, if you use our Twitter keyword monitoring, of course we keep history for all of that data - but we never showed you the trend percentage because we didn't want you to try to click it and have nothing happen (this data is stored with same parent/child relationship in the database). But since that mess is fixed, I have enabled the trends on the Twitter report, including the ability to graph any of them by clicking it. I use our Twitter monitoring a bit obsessively so, considering we released this feature almost 3 years ago, it's about damn time is all I have to say.

One more thing

From a UX perspective, the fact that we graph both hourly and daily data using the same colors is confusing, because we only log some data as hourly, and we show hourly by default if we have it. The only solution I can think of is to have the two types be different colors, but I can't find anything that works with our overall color scheme. We have a heavy emphasis on blue and orange. So I tried changing one of them to orange, and it looked horrible (screenshot). I'm just wondering if anyone has any ideas in this department, because I'm at a loss.

Ok, one more

In development for 2 months now is our path analysis feature. We're still working out the wiggles but expect it to be out within a few weeks and it will make an excellent partner to goals (although of course you will be able to use it without goals).

20 comments |   Feb 01 2012 9:46pm

Count your chickens while they're still hatching

About 3 months ago, Google announced that all users logged in to their Google account, their search terms would be hidden from the referrer string when they clicked through. Net result is no trackers can see these search terms (surprisingly, this includes Google Analytics). Google says this is for privacy reasons. That's great, but I've said a thousand times, search analytics is one of the best reasons to run analytics in the first place so it sucks for site owners.

Anyways, the way our code processed blank search terms, they would show up like this:

This has led to a lot of confusion and many people thinking Clicky is broken, because they weren't aware of this change on Google's end (understandable).

Google updated their Analytics product to show the search as "(Not provided)" if it was blank. We've decided to do the same thing, although we are labeling it as "[unknown]" instead:

(Update: Per request, we changed this to [secure search] instead)

The main difference though is that this will now show up in your main searches report too. Previously blank searches just weren't logged in there. Now you will see an item for "[unknown]", and unfortunately it's probably pretty high on the list. For our own stats on, it's the #3 "search term", representing about 1 out of every 6 searches.

I wouldn't be surprised if Google made this the default for all searches on Google sites within the next year or two. Hence I say to you, count your chickens etc, because that will be a horrible day for site owners the world round.

At the very least however, I am happy to know that Google is not providing a back door to their own analytics product that allows them to log the searches but not anyone else. That would just be evil, not to mention anti-competitive.
22 comments |   Jan 04 2012 10:40pm

Next Page »

Copyright © 2018, Roxr Software Ltd     Blog home   |   Clicky home   |   RSS