Split testing, new features for goals, and better visitor history

Alrighty, yesterday I went over a few new features after they were released, but that was just so you didn't think they were broken :) If you haven't read that post yet, please read it here, so you understand a few limitations of the new features until some historical post processing has been completed.

Split testing

We now have built in support for split testing. We're planning on using this extensively on getclicky.com itself, but we wanted to get it out as soon as possible so we haven't created any tests yet. But I wanted to have a screenshot for you, so I created a fake split test on our user registration page. There was no actual difference between what the visitors saw, we were just telling Clicky tracking that there was. Here's what it looks like:



As should be evident, to make your split tests valid, you need a good sample size. There was absolutely no difference between what these visitors saw, yet the conversion rates are 49% vs 37%. I'll let this run indefinitely, at least until we get two different registration pages to actually test, to see what the differences end up being. They should converge with time.

More details on setting up split tests are included on the main goals setup page. When you create a split test, you can choose what specific goals you want to track conversions for (as we did here, we're just tracking our "new user" goal instead of all goals), or you can let it default to tracking conversions for all goals. Both options have their ups and downs so it depends on what you're doing.

If you want to use a GUI interface to design your split tests, we have partnered with Visual Website Optimizer to provide that for you. More details on that are also on the goal setup page.


Better visitor history

Previously we were tracking the time of someone's first visit to your site, but now how many times they visited. Nor did we keep track of their first session ID. That's all fixed now! When viewing any visitor's details, you'll see how many total times they have visited your site, and the time of their first visit is now a link directly to that session, so you can see what they did on your site and how they got there (search etc) the first time they showed up.





Dynamic goals

Goals no longer need to be pre-defined. They still can be of course, but you now have the option to define them on the fly within Javascript. Dynamic goals have no limits, so that's the a huge benefit. Downside is you can't customize them (e.g. choose an icon) or use them with funnels.


New goal report

We've updated the main goal report to include a few new pieces of information. Tracking conversions is great, but wouldn't it also be nice to know how long it takes the average visitor to convert? That's exactly what we've added! Here's a snippet of our goal report, showing two goals that are very important to us - how many new people are registering, and how many people are upgrading. We offer a 21 day trial when new people register, but on average people are upgrading after only 14 days. That's fantastic! And for new users, it looks like the average person registers on their second visit after a 4 day lull from their first visit. Interesting! Of course, this is only about 12 hours of data, so it doesn't tell the whole story. A bigger sample size will give us a more accurate picture of how you use Clicky.





Better goal integration in all reports

When viewing any report, previously we had an extra column for conversions and conversion rate for each segment. However, the way we were calculating this wasn't taking into account which specific goals that segment was completing. It was just keeping an overall tally, and when you clicked the conversion number, it would filter your visitors down to those who had completed "any goal" for whatever segment you were viewing.

Now we'll show way more up front. As you can see int he screenshot below, we've added inline revenue to the reports, and when you click on the conversion number, it pops up a menu showing you all the individual goals that segment has completed, ordered most to least. You can click on any of those goals to immediately filter your visitors down to just that segment+goal, or click the "all visitors" link at the bottom to view visitors from that segment who have completed ANY goal.




Cookie domain

You can now specify the domain for the tracking cookie we set, instead of just ".yoursite.com" by default. It's a bit complicated and you should really only do this if you know what you're doing, but we've had requests for it so here it is. See docs here


That's all the major stuff. There's a few other little goodies here and there. e.g. all split testing data is available via the API, and has been added to the iPhone web app too. Hope you enjoy!
11 comments |   Feb 11 2011 2:35pm

New features. Some of them might seem broken, and here's why!

Our new release is out. It's been in the works for almost 30 days. There is metric happy ton of new stuff in here, especially if you use goals. But... it's been a LONG 48 hours preparing for some big database changes behind the scenes, so the last thing I want to do right now is write the nice, detailed blog post this release deserves. I'll do that after some sleep! Plus, that will give the database servers a chance to process some of the new data we're tracking, so I can show you screenshots of the new features with real data from Clicky's traffic.

If you have no idea what's new, please see this post from a few weeks ago, where we announced what was coming soon. There's a lot more to this release than in that post, but it covers most of the major points.

There are a few features I want to discuss right now, because if I don't, you're going to think they're broken. That's because they rely on a core feature that we just added for unique visitors, where we now track how many times each unique person has visited your site, and the session ID of their first visit.

Previous to today, we were not storing this information in an optimized way. Part of this update changes that, and hence we can use this information in a couple of interesting ways (with more to come in the future). So, now we have the ability to store it, but that doesn't mean it's magically there already. There are scripts running in the background that are parsing through data all the way back to April 16 2010, which is the date we started tracking "true" unique visitors. Based on the speed these scripts are running, we estimate it will take most database servers between 1 and 2 weeks to complete. So until that at least 2 weeks have passed, if you see some strange numbers - this is why.

Here are the ways we are using this data right now:

Visitor details: Link to first visit, and total visits




When viewing visitor details, previously we only showed the date and time of their first visit. But now, we'll also give you a link directly to that first session. This allows you to quickly see what this visitor did the first time they visited your site. Quite interesting, if I do say so myself! But, until we have gone through the last 10 months of data and "discovered" their true first visit, the link probably won't be right. It will just be a link to their oldest session as we have thus far determined. You may notice we're also displaying the total numbers of time this person has visited. This also won't be accurate until this 10 month chunk of data has been fully processed.


Goals: Time to goal, and visits to goal



When a visitor completes a goal, we now check out how many total visits they've had and how long ago they first visited as of that point in time, and store that calculation in the database. When viewing your goal report, we'll show you the average of these two values for all visitors who have completed that goal for whatever date range you are viewing. This gives you a great way to gauge how quickly your visitors convert on average. (Also, we'll only use a visitor's data in this calculation the first time they complete a goal, so the numbers don't get skewed by people who complete the same goal multiple times. However, this aspect of your historical data is not being post-processed. So if you have goals that can be completed more than once, until most of your repeat customers have completed the goal at least one more time, these numbers may be wacky).

The screenshot above is one of the goals we track, new user registration, as of about 30 minutes past midnight on this wonderful Friday morning. We have 13 new users, with the average user taking 2 visits and 1 day, 12 hours from the time of their first visit to when they finally registered. (I think there's a huge outlier here, when I viewed it about 5 minutes before snapping this screenshot, the average time was 3m 30s. I'm also noticing a bug now - the "visits" number is supposed to be rounded to the nearest tenth of a number, not to a whole number - I'll fix that tomorrow!)

Since we were already tracking the time of their first visit, the "time to goal" value should be mostly correct (excluding your repeat customers for now), but the "visits to goal" will not be, until we have gone through this 10 month period of data for all of your visitors.

-----

This post was much longer than I expected, and I only covered 2 out of maybe 10 major new items. I will write a nice post tomorrow detailing everything that's new. I just wanted to clarify a few things tonight so you don't think these particular new features are broken - they just won't be accurate until a few weeks have passed.
10 comments |   Feb 11 2011 12:50am

Quick downtime Saturday ~6pm PST

Our network host is requiring all customers to move their servers into a new server room they built last year. We've been putting this off for as long as possible, but the moment of truth has arrived.

We will try our hardest so there's no time when our network is actually 100% offline, because that affects your web sites. The plan is to move one of the paired load balancers and one of the tracking servers into the new room, then take the other stuff offline and let the already-moved LB and tracking server take over tracking temporarily. When the other servers are moved and come back online, everything will magically pair back together. This way there won't be more than a 5-10 second blip, and no tracking data will be lost.

The site will be in maintenance mode during the move, which we expect to take about an hour. When the database servers come back online, they will be behind real time by about an hour, so they will take a little bit to catch back up.

We've had our servers in the same set of racks in the same physical room for more than 4 years, so we don't expect to have to do this ever again.
6 comments |   Feb 04 2011 7:26pm

New features coming soon

It's been pretty quiet around these parts recently, so in case anyone was wondering if we were asleep at the wheel, the answer is NO! We took a few weeks off for the holidays but have been hard at work on lots of new goodies for the last two weeks. I just wanted to outlay what you can look forward to in our new release, which should be out by the end of this month.

  • Split testing - split tests will be defined 100% dynamically using our clicky_custom javascript object, which means there will be no limits on how many tests or versions of each test you can have, and no annoying forms to fill out. You'll also be able to define the specific goal you want to track for this split test, so that we only calculate the "best" version of your test based on one goal which is likely desired the majority of the time. Otherwise, we'll be tracking all goals that occur after this person visits one of your split pages.

  • Dynamic goals - For campaigns, we support both static and dynamic campaigns, so you don't have to define them first if you don't want to. We'll be adding the same thing to goals. One reason is to allow you to have more goals - there won't be any limits with dynamic goals. Secondly, we think it's easier to not have to predefine everything. This makes sense especially when combined with our new split testing feature. You will be able to define both your split tests and the goal you want to track for that test on the fly (if you want).

  • Total number of visits per unique visitor - this seems obvious but we're not currently tracking it directly. That's going to change so we can quickly query this value for any of your visitors. This will be particularly useful for goals, so you can see how many visits it takes the average person to complete any of your web site's goals (we'll be adding this as a new column in the main goal report). We'll also be reporting the average time too, which will be based on the time of each unique visitor's first visit to your site.

  • Customizable cookie domain - If you don't know what this means without further explanation, it probably won't be something you want or need. But, we have had a number of users ask to be able to set this to something other than the default ".yoursite.com", so you'll have that option now.


As we develop new features, other ideas or tweaks for those features usually come to us constantly. So, chances are there will probably be some extra stuff other than what's outlined above. We'll dive into much greater detail about each new feature in a new blog post when it's released.
9 comments |   Jan 19 2011 3:47pm

'Media searches' and other new stuff

One of our most requested features has been to segment out image searches from "normal" searches. We have just added this, so you should start seeing a new entry in your traffic sources report:




This feature has been requested countless times, as far back as June 2008. We're calling it "media searches" instead of "image searches", since we want to be as general as possible and may add support for video searches or other types of media in the future. Image searches from Google, Yahoo, and Bing are supported at the moment. We've also renamed "Direct / Bookmark" to just "Direct", and "RSS readers" to "Syndication".

Other new features and bug fixes from today:
  • Windows phone 7 detection (will show up as just "Windows phone" though)
  • The new "Google Web Preview" bot is now blocked by default as requested
  • AOL recently moved to "q" as the default search parameter in their search engine, so their searches were just showing up as links in Clicky. We've updated our code to support this new parameter. It still supports the old ones too (query and encquery) in case those are still used anywhere.
  • In October, we added support for google.com/aclk referrers, which is what we see when someone clicks on a paid search on Google. However, there's been a bug where these were referrers were showing up in your main "links" report, which normally only shows non-search referrers. We've fixed this bug, which was probably quite annoying for those of you running paid searches on Google.


That's all for today! You should start seeing us go back to our regular feature release schedule, where we release new stuff almost every week. The last few months have been insanely hectic from the massive redesign and tons of traveling and a big partnership integration we'll be announcing soon. We are excited to get back into the groove of writing lots of code because that's what we live for.
12 comments |   Dec 06 2010 5:29pm

Next Page »




Copyright © 2018, Roxr Software Ltd     Blog home   |   Clicky home   |   RSS