New option to pass alerts to the API

When you create alerts for something like a goal and configure them to be sent via email or twitter direct messages (RIP), what you get is a short URL that will take you right to the visitor session details on Clicky. A handy, but manual, process.

We had a request today to allow the alert IDs to be passed straight to the analytics API so that automated processes could get more data about these visitors. We thought this was a great idea so hopped right on it, and this is now available.

Full details and some example code are in the docs, but briefly, if you extract the alert ID from the end of the URL then pass that to the API as alert_id=XXX, then we'll look up the session ID (first verifying the session belongs to the site_id and sitekey you also pass in) and convert the request to be as if you had originally sent in a type=visitors-list&session_id=YYY request.

You need to know what you're doing to implement automatic parsing of emails, but for the people who have the ability to do this, we think you will like this new feature quite a lot.

Here's an example API request for an alert we have setup for our blog:

http://api.clicky.com/api/stats/4?site_id=32020&sitekey=71bf7c1681e22468&alert_id=18227240
0 comments |   Feb 11 2014 3:43pm

Bing's secure search will be worse than Google's, for most sites

I never realized how much I would come to love Bing in the last year. I don't use it, but now that Google is pretty much 100% secure search across the, almost all of the search phrases that we end up logging are from Bing. Yay for being able to tell what people searched for when coming to our site!

Earlier today I saw this announcement that Bing is now testing secure (HTTPS) searches.

It's different than Google's though, at least right now. For a small number of sites, it changes nothing. But for most sites, it's much worse. Why?

On Google, when do a search over HTTPS, when you click the result it actually sends you through an intermediate, non-HTTPS page. For example if you go to https://google.com and search for "Clicky" and click the first result, the URL you actually go to is this:

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC4QFjAA &url=http%3A%2F%2Fclicky.com%2F&ei=5fvWUrbSIOiqyAHyr4Fo &usg=AFQjCNFOPVnz4LaWTBeOHfeCOADfndg5BA&sig2=vLGsm0mdBFXCn-4n_CQ11Q&bvm=bv.59378465,d.aWc&cad=rja

And then you are redirected to our web site. You can see they hide the search paramter ("q=") but because this intermediate page is handled with standard HTTP, at least we still know they came from Google. Better than a kick in the pants, right?

The problem with Bing's new secure search is that there is no intermediate page. This means that unless your site uses forced HTTPS across the board, not only will no searches get logged, you won't even know they came from Bing in the first place (because browsers don't send referrers when going from HTTPS -> HTTP). Since there will be no referrer, they will get logged as "direct" visitors.

For those of you with forced HTTPS on your site, you will still be able to see the actual search phrases used. But since very few sites use forced HTTPS, this will impact the vast majority of web sites out there.

This hasn't been technically "released" yet so things might change, but as they stand now, this is how it is. To be clear, it doesn't appear Microsoft is doing this to hide data from web site owners like Google is intentionally doing, but rather, as a way to protect users from NSA spying. That's a great reason for this change, I just really hope they also understand the negative impact this will have on most site owners and add some kind of way for us to detect that the visitor at least arrived from Bing. Hiding the search phrase is one thing, but hiding the source of traffic entirely is just really bad for site owners.

There is a small amount of hope they might make it so it works like Google: This change is really going to hurt Bing's marketshare numbers as reported by various services, including us. For this reason alone, I think Microsoft will change things around. They don't want to start seeing headlines about Bing's marketshare plummeting to zero.


I wanted to add that all of this secure search stuff impacts all trackers, not just Clicky. And I also wanted to let you know that we have a plan to deal with this, at least when we know the source of traffic is a search engine. As far as I know, no other tracker does anything special with these secure searches. We want to be the first to offer a solution, so I'm not going to go into details now, but I think our proposed solution will work well for most of our customers. Stay tuned.
2 comments |   Jan 15 2014 1:26pm

Authentication improvements to on-site analytics

There's been an authentication bug with the on-site analytics widget (OSA) for about a year. It only affected a small number of customers, so it was hard to track down. This was not a security bug, but rather, OSA would just simply not display for a small number of people.

While mucking around with heatmap updates earlier this week, I was also updating some of the OSA code and a lightbulb went off in my dang head as to the cause.

Here's how OSA works. When you login to Clicky, a cookie is set for "in.getclicky.com", which is our tracking domain. This cookie is used for authentication purposes. When you visit your own web site, the tracking code tracks your visit and sends that info to in.getclicky.com. If that cookie is set, then we check if it matches the owner of that site ID on our end. If so, then we call some code to display the OSA widget (and at the same time automatically ignore your own visit).

OSA was released when our domain was still getclicky.com. So at that time, browsers considered the cookie to be first party cookie because it was a sub-domain of the site you were already on. Life was good.

Two months after OSA was released, we acquired clicky.com. But we left *.getclicky.com as our tracking domains, so people wouldn't have to change their tracking code and everything could just be transparent.

This change meant that setting a cookie for "in.getclicky.com" when logged on to clicky.com was now a third party cookie as far as browsers were concerned. So when we started getting emails about OSA not working, we assumed that the cause was third party cookies were disabled in their browser. And in lots of cases, this was the cause. But there were a few people who, no matter what they did, could not get the widget to work.

So that lightbulb I was talking about earlier... when in.getclicky.com tells our tracking code to execute the OSA widget, the widget is loaded from clicky.com, not in.getclicky.com. We hadn't made any major updates to heatmaps or the widget since the domain change, until earlier this week when I was messing with that code, so I hadn't really dived into that block of code for a while. Sure, I had looked at the code since it was written, but you can't really get into the zen with existing code until you start playing with it.

The problem was that the third party cookie would authenticate you on the tracking domain, but unless you had checked the "remember me" box when logging in to clicky.com, you had no authentication to load the OSA widget from clicky.com, so it would just exit out. Most people use the "remember me" option so that's why this only affected a small number of people.

What we've done is now when your beacon (with proper cookie) is sent to in.getclicky.com, before calling the OSA widget, we tell the tracking code what your sitekey is. The sitekey is used to authenticate requests with the API and other things. So now when OSA tries to load by sending a request to clicky.com for the data, it's authenticated with the sitekey.

I'm always 100% of the time logged in to clicky.com with the "remember me" option set. Once I discovered this issue, I logged out and sure enough, OSA would not load for me anymore when I was on clicky.com, even though my cookie for in.getclicky.com was still set. Now after making this change, whaddaya know... It works! And it should work for you too if this was affecting you!
0 comments |   Jan 11 2014 2:41pm

Alerts for organizations

We had a customer request for being alerted by visits from certain organizations, whether generic in nature (the example given here was "school"), or something more specific.

Yeah we should have already had this. Took less than 30 minutes to add it. This will be extremely useful for those of you who use Clicky as a lead generation tool.

Another example would be, pretend you work at a newspaper and you have a feeling that a major competing newspaper is stealing your stories. Setup an alert to see when they visit, as shown in the screenshot below. (Not saying NYT does this of course, it's just an example).

This will match both organization names and hostnames.




UPDATE JAN 10: We just made another change to alerts, so the item that triggered the alert is now included in the alert notification. For example if you use wildcards in organization alerts, more than one organization can trigger that alert, but you won't know what the actual organization is until you click through and arrive at our web site. We've had this request before and now especially with the new organization alerts, we figured it was time we added this. This change affects all alert types of course, not just organizations.

While making this change we also found a bug with alerts for dynamic campaigns (ones that use "UTM" variables). Namely, alerts didn't work for these at all. This has been fixed!
5 comments |   Jan 09 2014 5:48pm

Heatmap updates: Longer storage, better scaling, and new filters

We just pushed some updates to heatmaps today.

First, we've extended the storage time to 6 months, up from 3. When we released these about 15 months ago, we were worried that they would take up too much space if we kept them around for too long, so we were limiting to 3 months of data. It took a while before "lots" of people were using them, so we couldn't analyze the impact on storage space fairly reliably until recently. Good news, they take up less space than we had anticipated because we made some good decisions on the design side of things. Of course, the data older than 3 months from right now has already been purged, so you don't have 6 months now. But 3 months from now, you will.

Second, we added a few new filter options when using heatmaps via the on site analytics widget. The new filters are under the "More..." menu, as shown below. These new filters allow you to view heatmaps for a page for just new visitors, returning visitors, visitors online now, or "registered" visitors. Registered means they have a "username" custom data field attached to them.

We'll probably add a few other filters as we think of them. What would you like to see here? One thing we considered was some kind of engagement filter, e.g. only visitors who bounced or only visitors who did NOT bounce. However, if you think about it, almost everyone who bounces would not have clicked anything on your site, so it wouldn't really tell you much.




Last, we changed the scaling of the heatmaps. The problem was that for pages with lots of clicks, there would generally be a few extremely "hot" areas, e.g. in the navigation bar, and they would be so much hotter than the rest of the page, the other clicks would not even be visible. We've changed it so the scale of hotness now only goes from 1 to 10, rather than 1 to whatever it would happen to be as stored in the database.

Here is an example of a heatmap for our homepage using the old method, for "all clicks":



Most visitors coming to our site are already customers, so the two login links are going to be the most clicked items, by about a thousand miles. Of course, there are plenty of other clicks on this page (e.g. new people signing up are going to click the big ass "register" button), but we can't see them unless we apply some filters first, such as only showing visitors who completed our "new user" goal. It would be real nice to be able to see ALL the clicks on this page though, wouldn't it?

So here's what the new scaling method does to the exact same pool of data in the last screenshot, most of which was completely invisible:



It's certainly noisier but the hot areas still stand out, and yet you can see every single click on the page, because the range from min to max is so much smaller.

Let us know what you think!
3 comments |   Jan 09 2014 3:21pm

Next Page »




Copyright © 2017, Roxr Software Ltd     Blog home   |   Clicky home   |   RSS