Rush Posted February 26, 2007 Report Share Posted February 26, 2007 I just had a comcast IP come cruising through my site, I think I know whats going on: Distributed Search Engine projects.I noticed in the logs the comcast IP was using a MJ12bot/v1.1.2 agent.Which is written by http://www.majestic12.co.uk/So folks run this on their computers, gather links for the search engine and gain ranking.All in all, pretty harmless.You can block it by putting:User-Agent: MJ12botDisallow: /In your robots.txt file.Rush Quote Link to comment Share on other sites More sharing options...
kpeirce Posted April 13, 2007 Report Share Posted April 13, 2007 Adding more and more IP address blocks via your server's control panel (to the .htaccess file) will eventually slow it down as it has more rules to search through.Rogue bots also ignore the robots.txt file, as I've had them come through web sites that I have.If you want to display a visitor's IP address to them, this bit of PHP code will do it.<?php echo (getenv("REMOTE_ADDR")); ?>If embedding into a php file, remove the (<?php) and ending (?>).I've found that a lot of bots and automated scanning is related to looking for vulnerabilities in web sites. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.