Jump to content
TNG Community
Sign in to follow this  
arnold

showlog.php and search engine robots

Recommended Posts

arnold

It has become impossible to see which IP Addresses are visiting our website due to the overwhelming number of visits by various search engine robots, especially Google.com's. I wrote to Darrin about this problem to see if he would add a way to keep the visits by robots from cluttering up the log file. He wrote back stating that that feature is already included. Here is what he wrote:

Just go to the log and copy a portion of the logged domain name or IP address (ie, "google.bot"), then go to Admin/Setup/Log Settings and paste that into the field for host names to exclude. You can separate multiple entries with commas. That should still allow those crawlers to visit your site, but their visits will no longer be logged.

After implementing Darrin's "fix," our log file now looks like this:

The Sprague Project Log File

The hundreds and hundreds of visits by various robots no longer appear.

Thank you, Darrin, as always.

Share this post


Link to post
Share on other sites
tmaguirre

I implemented this change several months ago so my access log wouldn't be clogged with visits by googlebot. It has worked great, and doesn't affect the webstats at all.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×