Forums » Suggestions & Feature requests



Track Spiders

I was wondering if there is any way to track spiders through clicky? It would be a nice feature to see which pages spiders are hitting and how often they are hitting them. It could be beneficial to see spiders crawling through the site and possibly identify problem pages where spiders are getting stuck or hitting a dead-end. I'm not sure if this "doable" but I figured I'd toss around the idea since it could be helpful for a lot of webmasters.

Posted Wed May 21 2008 7:51a by mafmi***


No, bots tend not to interact with external elements on your web site, especially javascript. The only way you can get reliable information about bot activity is by using a log analyzer, such as Webalyzer, AWStats, etc...

Posted Wed May 21 2008 12:18p by Your Friendly Clicky Admin


I can help you track live spiders very precisely. Man it is so easy, I thought clicky would have it already and this would be asynchronous as well. The data can be distinguished like "live visits" and "Bot Visits". But it will take me a week to develop such a code and information. Get in tough, would be great to work with you guys.

Posted Tue Jul 13 2010 9:12a by bloggersc***


If you use wordpress, they have a robot analytics plugin

Posted Tue Sep 21 2010 6:14p by paradigm***


It is actually possible to track bots with Clicky, if you use server side tracking with php or an equivalant.

Posted Tue Sep 21 2010 7:54p by ameer_da***


Agree this would be a great feature to help monitor bots - in particular rogue bots. Any update as to if and when this may happen? Webalyzer does not appear to offer real time so thats not a good solution.

Posted Wed Apr 3 2013 12:05p by b2byellowp***


You must be logged in to your account to post!