bit.ly integration

One of the best parts of owning a small company is being able to act fast. There's no red tape or clueless management to deal with - you want something done, BAM, it's done! A perfect example is when Google Chrome was released. Within several hours of the announcement, we added support for it to our web browser reporting. If I remember correctly, it took Google Analytics several weeks to get Chrome detection integrated, which we found amusing (ya know, since they're made by the same company and all).

Anyways, I was recently reading a story on TechCrunch about bit.ly, and it briefly mentioned at the bottom about how WebTrends (a fellow Portland analytics firm) announced a partnership with bit.ly to integrate bit.ly's analytics into their reports later this summer. I didn't know bit.ly had a stats API (because I use clicky.me, of course!), so I jumped on the opportunity. Two days later, it's part of Clicky.

bit.ly's existing API only provides very basic data unfortunately - the only thing you can access is the total number of clicks that your last 15 shortened URLs have had. I imagine this partnership with WebTrends will provide them with access to all of the analytics that bit.ly gives you for a link, when viewed on their web site (geographic data, referrers, etc - here's an example), which we don't think is fair really. Their API should be open to everyone, so we're a bit disappointed there.

But no matter. We think you'll still like this integration, and we've made it extremely easy to access the full bit.ly report for any of your links simply by clicking on the link. Try it out in our demo account.

How to set it up

This feature does require a Pro or higher account (upgrade here if necessary). For any site you want to add this to, just go to your site preferences main page, click "preferences" again (near the top), and then click the "advanced settings" link and enter your bit.ly username where requested.

Available reports

This is available under the "Links" report, by clicking the new bit.ly sub-tab. It has also been added to the "Short URLs" dashboard module, and to our iPhone web analytics. We wanted to add it the API too but there's a bug with accessing third party data via the API at the moment that we still need to fix.

Enjoy!



12 comments |   Jun 08 2010 3:59pm

Say goodbye to search analytics

Google just announced their new secure search beta, and DuckDuckGo announced similar measures. DDG's are a bit more thorough, but the end result is the same - the search term is not passed through the referrer, and hence no analytics tool (not even a good old log analyzer) will have any idea of what a visitor searched for to reach your site.

They both claim to do this in the name of privacy. Google's is a beta so it's off by default, and it doesn't explicitly say "privacy from webmasters", but that is part of the end result. DDG, on the other hand, explicitly mentions "not leaking your search terms", and this feature is enabled by default for all users - even over non-HTTPS connections. (Technical note: when you click a link on an HTTPS page, your browser does not send a referrer, which is why HTTPS search engines will result in "secret" searches that we can't see).

Being able to search over HTTPS is probably a good thing for people in countries with vast censorship, such as China. I can understand offering this as an option that people can use. But I really hope Google never considers making this the default, because that would be very irritating for web masters - we would have no idea what people were searching for to get to our site, which is arguably the #1 reason to run analytics in the first place.

My big problem is that DDG is hiding the search term from sites by default. That's not good for the internet at all, and I'm scared that Google may end up doing something similar (via HTTPS or not), and perhaps other search engines in the future.

Part of the problem is that this gives users a false sense of security. Yes, someone "snooping" your connection won't be able to tell what you're searching for, but the sites you click through to will probably have a good idea, based on your landing page - not to mention they can also see their IP address and every page they have ever viewed on my site, ever. And yet somehow, not knowing this visitor's specific search term is protecting their privacy? Please. The only thing it does is make the life of a web master a much bigger pain in the ass than it was before.

I'm taking time out of my amazing Hawaiian vacation to write this blog post because that's how much I care about this issue, and believe that it is not good for anyone. Of course, I wouldn't be surprised to learn that Google Analytics will somehow magically be able to track these "secure" searches. Wouldn't that be convenient for the likes of every other analytics service on the planet?

This all seems very familiar, doesn't it? Other search engines may start doing this too, but really, with 90% marketshare, Google is the only one that matters. So what can you do? Blog about it. Tweet. Complain. Let Google you know that you do not like this, and let DuckDuckGo and any other engine who does this in the future know the same.
47 comments |   May 22 2010 10:25am

New engagement reports

We have a couple of new reports for Pro and higher members. These break down your visitors' engagement levels to quite a bit of detail, both in terms of time on site as well as number of actions performed. The reports are quite self explanatory, as you can see from the screenshots below, so there's really not a whole lot to say here. We think these will be a big hit in particular with those of you who don't like our new bounce rate, because part of this report basically shows you the "old school" bounce rate: the number of visitors who had only 1 action.

This is available in a new dashboard module, as well as full reports with a bit more detail under the Visitors tab. We also added it to the iPhone web app. You can also get the data from the API. The docs have been updated with the new type values you will need to request.




7 comments |   May 11 2010 9:32pm

Why Clicky's new bounce rate is the best in the biz

The standard definition of a bounce is a visitor who only viewed one page on your web site. Your web site's bounce "rate" is the percentage of visitors who "bounced". This metric is supposed to give you a very generic estimate of your visitors' engagement level. If you have a high bounce rate, then your site is apparently very unengaging.

But are single-pageview visits really the best way to define engagement? In the age of blogs and social media, we don't think so.

Our new release added pinging support to our tracking code. This means that as a visitor to your site stays on a single page, they are still talking to our tracking servers, so we know they are still on your web site. This lets us determine how long a visitor is actually on your web site, even if they only have one pageview.

There is a significant difference between a visitor with one pageview who spends less than 30 seconds on your site, and a visitor with one pageview who spends 5 minutes on your site. But no other analytics service will tell you about this difference - no matter how long any single-pageview visitors are on your site, they are all considered bounces. We think this is horribly broken and misleading, so we decided to redefine what a bounce is.

How Clicky determines a bounce

A visitor who has only one pageview, and who is on your site for less than 30 seconds is what we now consider a bounce. So any visitor who has more than one pageview, or any visitor who has only one pageview but is on your site for at least 30 seconds, is now what we consider "engaged" / not a bounce.

Say you have a blog post that you have shared on a few social media networks like Facebook and Reddit, and you get 1000 visitors to it. Chances are that 95% of these visitors will only view the article that is being linked to - one pageview. Maybe half will read the whole article, half will read part of it, and a few will click through to your front page to see more. Any other analytics service would report a 95% bounce rate for these visitors. But a bounce is negative, so it makes it sound like only 5% of these visitors were engaged. But that's not true at all - half of them read the entire stinking article!

Since Clicky now sends pings, we can measure how long each visitor is actually online. For the example above, Clicky would report a bounce rate closer to 50% for these visitors. This is a much more accurate representation of how many of these visitors actually "bounced" from your site, versus how many actually stayed online long enough to read what you had to say. Because of this, we think we now have the best bounce rate metric in the industry.

Arguments against this change

We've had some feedback that this new measurement is no good either, because there are lots of people who used tabbed browsing and open a bunch of tabs in the background, then go and read them later. This is true, but this is also more of a "power user" tactic. As long as less than 50% of users are doing this (and we think it's probably more like 10%), then our bounce rate is a much more accurate representation of engagement.

Some people also suggested we make this a different metric altogether, such as "engagement rate". We considered this, but ultimately we decided that the term "bounce rate" is just defined incorrectly in the first place. This shouldn't be a new metric - this is the way bounces should be defined, period. This is why it's called "analytics" - your traffic data is "analyzed" to help you make sense of it. Reporting that 90% of your visitors only had one pageview doesn't really tell you anything, but reporting that 50% of your visitors had one pageview and were online less than 30 seconds does tell you a lot about your web site's traffic.

This doesn't mean we don't want to make it better though. We will probably add something that detects whether or not the page being viewed is "in focus" or not, and only do the pinging if it is actually being read. This will require a lot of testing, however. For example, a visitor opens a link to your site in the background, but doesn't actually read it until an hour later. If we don't ping until the page comes to the foreground, their session will have already expired so the pings won't register when they are processed on our end. Should we delay the page view log until the page comes to the foreground? That might work, but what if they move the window/tab to the background again and then come back to it another hour later? As you can see, it gets a bit complicated, so we're not going to change it until we have taken the time to determine the absolute best way to do it, and test the heck out of it.

We'd still like to hear your feedback on this change of course, and what you think would make it better. So have at it in the comments.
28 comments |   Apr 26 2010 5:43pm

What happened

At approximately 6:30AM PST, our load balancers went offline. We have server monitoring tools that check our public IPs from a remote location every minute, so we were aware of the issue immediately. But it took a bit to figure out what was wrong and get them back online. Total downtime was about 90 minutes.

At first we thought it was a network problem, as happens from time to time, but quickly noticed that a few public IPs we have that are not behind load balancers were fine. Ok, so it's the load balancers then? We have a double load balancer setup with automatic failover, so if one dies, the other takes over. We've tested this and it works great. But the chances of both dying at the same time? Impossible. But we couldn't access their public IPs. That was mildly depressing.

But then we remembered they also have internal "management" IPs for admins to login to. We tunneled into the network and we were able to login to both load balancers, but after looking around a bit, there was nothing obviously wrong. They were online, they could see the internet, they just weren't passing traffic through to the servers behind them. After a bit of banging head upon keyboard, we checked the load balancer logs and saw some messages we'd never seen before, repeating over and over every minute:

Apr 19 07:09:11 : Machine rebooting too often - Passifying

Passifying? Ok, well at least we see that the load balancers were rebooting themselves for some reason. So we did a manual reboot on one to see what would happen, because they weren't rebooting anymore - this message was simply being output every minute so that the admin looking at the log files could not miss it.

Upon reboot, they would pass traffic through for about 20-30 seconds, then go boom. We managed to check the load stats before they went offline, and saw they were at 100%. Well that's not good.

So the problem was that our load balancers were extremely overloaded. We just added pinging to our tracking code, which means visitors to your web sites talk to our tracking servers a lot more often than they used to. We obviously knew this would happen, so what we did was we actually activated our new tracking code last Tuesday for all sites, well before the actual release (Friday night), so we could see the effects that it had on our tracking servers. The load went up significantly, on both the load balancers and the tracking servers, but not enough to be concerned about. It was still a very manageable level.

But at that point, our tracking servers were not actually logging the pings. They were receiving them, but they were discarded. That makes a difference. We made our full release on Friday night so it's been running over the weekend and were watching it very closely. Things seemed great. Of course, the weekend's traffic is quite a bit lower than during the week, particularly Monday. Monday is the biggest traffic day of the week for most sites, so it's the biggest day of the week for our service too.

So Monday morning rolls around, and once most of the US is awake, the spike went up quite a bit higher than it did over the weekend. What happened was the load balancers were getting timeouts when talking to the tracking servers, which means they were keeping connections open for a lot longer than they normally would. This type of thing quickly spirals out of control, hence, the problem this morning.

When our database servers process traffic every minute, they keep a log of what they just did - how many actions were processed, how many pings, how long the process took, etc. This is extremely useful data. So we took a peak at this and saw that pings were accounting for over 80% of traffic being processed. This means that the pinging functionality increased the hits to our tracking servers by 400%, overnight. That's a little more than we were expecting. We were expecting maybe a 200% increase at the most.

In an effort to get things back online as quickly as possible, we changed the tracking code so that the total pings it will send are half as many, and the initial 15 second quick-check ping doesn't happen. After uploading that to the CDN, we rebooted the load balancers again and they came online and have stayed online since, with a load of about 65%. When viewing the database logs now, the pings are more inline with our initial expectations - about a 200% increase over normal traffic.

We want to ping more often though, so we'll be tweaking all sorts of things over the course of the day and probably throughout the week, to get the most pings possible with the least amount of load on our servers. We'll post an update later to let you know how it's going.

This is another one of those unfortunate cases that aren't really testable until they're live. We knew there would be a big spike on our end, and we thought we had it under control, but we just didn't know quite how big it was going to be until the system was fully live and all of our sites were sending us Monday traffic. But for now, things are stable, and that's what's important.
21 comments |   Apr 19 2010 9:27am

Next Page »




Copyright © 2018, Roxr Software Ltd     Blog home   |   Clicky home   |   RSS