Authentication improvements to on-site analytics

There's been an authentication bug with the on-site analytics widget (OSA) for about a year. It only affected a small number of customers, so it was hard to track down. This was not a security bug, but rather, OSA would just simply not display for a small number of people.

While mucking around with heatmap updates earlier this week, I was also updating some of the OSA code and a lightbulb went off in my dang head as to the cause.

Here's how OSA works. When you login to Clicky, a cookie is set for "in.getclicky.com", which is our tracking domain. This cookie is used for authentication purposes. When you visit your own web site, the tracking code tracks your visit and sends that info to in.getclicky.com. If that cookie is set, then we check if it matches the owner of that site ID on our end. If so, then we call some code to display the OSA widget (and at the same time automatically ignore your own visit).

OSA was released when our domain was still getclicky.com. So at that time, browsers considered the cookie to be first party cookie because it was a sub-domain of the site you were already on. Life was good.

Two months after OSA was released, we acquired clicky.com. But we left *.getclicky.com as our tracking domains, so people wouldn't have to change their tracking code and everything could just be transparent.

This change meant that setting a cookie for "in.getclicky.com" when logged on to clicky.com was now a third party cookie as far as browsers were concerned. So when we started getting emails about OSA not working, we assumed that the cause was third party cookies were disabled in their browser. And in lots of cases, this was the cause. But there were a few people who, no matter what they did, could not get the widget to work.

So that lightbulb I was talking about earlier... when in.getclicky.com tells our tracking code to execute the OSA widget, the widget is loaded from clicky.com, not in.getclicky.com. We hadn't made any major updates to heatmaps or the widget since the domain change, until earlier this week when I was messing with that code, so I hadn't really dived into that block of code for a while. Sure, I had looked at the code since it was written, but you can't really get into the zen with existing code until you start playing with it.

The problem was that the third party cookie would authenticate you on the tracking domain, but unless you had checked the "remember me" box when logging in to clicky.com, you had no authentication to load the OSA widget from clicky.com, so it would just exit out. Most people use the "remember me" option so that's why this only affected a small number of people.

What we've done is now when your beacon (with proper cookie) is sent to in.getclicky.com, before calling the OSA widget, we tell the tracking code what your sitekey is. The sitekey is used to authenticate requests with the API and other things. So now when OSA tries to load by sending a request to clicky.com for the data, it's authenticated with the sitekey.

I'm always 100% of the time logged in to clicky.com with the "remember me" option set. Once I discovered this issue, I logged out and sure enough, OSA would not load for me anymore when I was on clicky.com, even though my cookie for in.getclicky.com was still set. Now after making this change, whaddaya know... It works! And it should work for you too if this was affecting you!
0 comments |   Jan 11 2014 2:41pm

Alerts for organizations

We had a customer request for being alerted by visits from certain organizations, whether generic in nature (the example given here was "school"), or something more specific.

Yeah we should have already had this. Took less than 30 minutes to add it. This will be extremely useful for those of you who use Clicky as a lead generation tool.

Another example would be, pretend you work at a newspaper and you have a feeling that a major competing newspaper is stealing your stories. Setup an alert to see when they visit, as shown in the screenshot below. (Not saying NYT does this of course, it's just an example).

This will match both organization names and hostnames.




UPDATE JAN 10: We just made another change to alerts, so the item that triggered the alert is now included in the alert notification. For example if you use wildcards in organization alerts, more than one organization can trigger that alert, but you won't know what the actual organization is until you click through and arrive at our web site. We've had this request before and now especially with the new organization alerts, we figured it was time we added this. This change affects all alert types of course, not just organizations.

While making this change we also found a bug with alerts for dynamic campaigns (ones that use "UTM" variables). Namely, alerts didn't work for these at all. This has been fixed!
5 comments |   Jan 09 2014 5:48pm

Heatmap updates: Longer storage, better scaling, and new filters

We just pushed some updates to heatmaps today.

First, we've extended the storage time to 6 months, up from 3. When we released these about 15 months ago, we were worried that they would take up too much space if we kept them around for too long, so we were limiting to 3 months of data. It took a while before "lots" of people were using them, so we couldn't analyze the impact on storage space fairly reliably until recently. Good news, they take up less space than we had anticipated because we made some good decisions on the design side of things. Of course, the data older than 3 months from right now has already been purged, so you don't have 6 months now. But 3 months from now, you will.

Second, we added a few new filter options when using heatmaps via the on site analytics widget. The new filters are under the "More..." menu, as shown below. These new filters allow you to view heatmaps for a page for just new visitors, returning visitors, visitors online now, or "registered" visitors. Registered means they have a "username" custom data field attached to them.

We'll probably add a few other filters as we think of them. What would you like to see here? One thing we considered was some kind of engagement filter, e.g. only visitors who bounced or only visitors who did NOT bounce. However, if you think about it, almost everyone who bounces would not have clicked anything on your site, so it wouldn't really tell you much.




Last, we changed the scaling of the heatmaps. The problem was that for pages with lots of clicks, there would generally be a few extremely "hot" areas, e.g. in the navigation bar, and they would be so much hotter than the rest of the page, the other clicks would not even be visible. We've changed it so the scale of hotness now only goes from 1 to 10, rather than 1 to whatever it would happen to be as stored in the database.

Here is an example of a heatmap for our homepage using the old method, for "all clicks":



Most visitors coming to our site are already customers, so the two login links are going to be the most clicked items, by about a thousand miles. Of course, there are plenty of other clicks on this page (e.g. new people signing up are going to click the big ass "register" button), but we can't see them unless we apply some filters first, such as only showing visitors who completed our "new user" goal. It would be real nice to be able to see ALL the clicks on this page though, wouldn't it?

So here's what the new scaling method does to the exact same pool of data in the last screenshot, most of which was completely invisible:



It's certainly noisier but the hot areas still stand out, and yet you can see every single click on the page, because the range from min to max is so much smaller.

Let us know what you think!
3 comments |   Jan 09 2014 3:21pm

Twitter direct messages

Four and a half years ago we released several Twitter features, one of those being to receive alerts via direct message on Twitter. This has been an awesome feature and we have used it ourselves since day one, to be notified of certain goals happening on our site, such as a new paying customer. Earlier this year we released our uptime monitoring feature, and we added Twitter direct message alerts there also.

So we are sad to say that today, these features have been removed, and it's unlikely they will ever come back.

Sometime in mid-October, Twitter added some pretty strict throttling. We never saw any "news" about this, but within a few days we started getting tweets and emails about Twitter alerts no longer working. A bit of research led us to this page on Twitter's web site, which says that all accounts are now limited to sending 250 direct messages per day, whether via the API or not.

We send many many thousands of direct messages a day, hence this feature is now severely broken. We blow through 250 direct messages by 1 AM most days. So unless Twitter revises this limit to something much higher, this feature is permanently dead.

Many other services have probably been affected by this throttle. It just really bothers me how much Twitter keeps slapping third party developers in the face, the very people who helped build Twitter into what it is today. One thing I know, we will never add another Twitter feature to Clicky, ever. Our Twitter analytics feature still works, thankfully. For now, anyways...
11 comments |   Nov 11 2013 3:06pm

Sticky data: Custom data, referrers, and campaigns saved in cookies

[Do not panic. You do not need to update your tracking code.]

We've just pushed a major update to our tracking code so it works a bit more like one aspect of Google Analytics now, that being that some additional data is saved in first party cookies (set with Javascript) for visitors to your site. This data being referrers, dynamic (UTM) campaign variables, and custom data set with clicky_custom.visitor (renamed from clicky_custom.session - don't worry, the old name will still work indefinitely).

We're calling this "sticky data" and the point of it is two fold. First, for referrers and dynamic campaigns, this will better attribute how your visitors originally arrived at your site, as they visit in the future. In other words, if they find your site via a link or search, all future visits to your site where they go directly to your site instead of through a link, this original referrer (and/or campaign) will be attached to these new sessions. This will be particularly useful for those of you who have setup goal funnels using referrers or campaigns. These cookies are set for 90 days. Google does 180, which we think is a bit too long, so we're doing 90 instead.

(Note: This does not work for "static" (pre-defined) campaigns that you create in Clicky's interface. It only works with the dynamic ones created with e.g. "utm_campaign" etc variables).

Second, if you set custom visitor data, we've thought for a while now how great it would be if that data stuck across visits. For example if someone logs in and you attach their username to their session, that's great - everytime they login that is. But what about when they visit your site in the future but don't login? Well, now that we save this data in a cookie, their username will still get attached to their session so you'll still know who they are. utm_custom data will also be saved! These cookies are set "indefinitely", more or less.

A lot of you use custom visitor data to attach things that are very session specific though, such as shopping card IDs, that kind of thing. With this in mind, there are only 3 specific keys we'll save by default for custom visitor data. Those keys are "username", "name", and "email". Of course, if you have others you want to save in cookies, you can customize it with the new visitor_keys_cookie option. Click that link to learn more.

We think the vast majority of you will like this new sticky data. However, if for some reason you don't, we created another new clicky_custom option as well, sticky_data_disable. Setting this option will disable this data being saved to or read from cookies, without having to fully disable cookies. And of course, if you have fully disabled cookies, this data will never get saved in the first place.

Originally we wanted to add support for parsing GA's "__utmz" cookies, which is what GA uses to store campaign and referrer data for 6 months. The cookie format is fairly straight forward but upon investigating our own GA cookies we saw a lot of inconsistency across all the sites that it had been set for. So we're going to hold off on that for now.

Our privacy section on cookies has been updated to reflect these updates.

Enjoy!
5 comments |   Sep 27 2013 1:27pm

Next Page »




Copyright © 2018, Roxr Software Ltd     Blog home   |   Clicky home   |   RSS