Hiraeth Posted May 7, 2008 Report Share Posted May 7, 2008 I'm pretty new to this stuff and have just got my site up and running. I can see visits from crawlers in my access log. I recognise google and yahoo but who or what are 'crawl-1.cuill.com' and 'cache-prs-ab05.proxy.aol.com'? Are these 'safe' crawlers and if not what could I do about it? Quote Link to comment Share on other sites More sharing options...
Smack Posted August 12, 2008 Report Share Posted August 12, 2008 Hiraeth,'Crawl-1.cuill.com' is a search engine spider from Cuil. They recently launched to scathing reviews. For example, see Kevin Ryan's comments at What's Wrong with Being Cuil?The other one is probably an AOL user.Greg Quote Link to comment Share on other sites More sharing options...
Rush Posted August 13, 2008 Report Share Posted August 13, 2008 I guess cuill finally named it's bot... Mine always showed as Twiceler bot.On a side note, some users and possible the AOL user might be participating in the MJ-12 distributed search engine project.Rush Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.