jseaman Posted March 1, 2007 Report Share Posted March 1, 2007 Yesterday, I needed to make available some files to someone I've been exchanging emails with. I was uncomfortable sending these rather large files via email so I created a new directory on my TNG website and uploaded the files to that area. I set the directory so just browsing to it would display a list of the contents - easier than providing links to each individual file!So today I was just messing around and happened to look at my server logs - Google had already found the newly created directory and taken a look at every file in it! HOW?The name of this directory does not exist anywhere as a link except in the email I sent to my friend. How did Google find the directory? It is located off my main TNG directory so a index file was present in its parent directory.I had thought the only way the search engines could find a directory (or file) was if there was a link to it somewhere in universe. (assuming the parent directory wasn't browseable). For that matter ... where does Google find the link to tngrss.php? I don't see one anywhere on my site!Ideas? Quote Link to comment Share on other sites More sharing options...
Rush Posted March 2, 2007 Report Share Posted March 2, 2007 I really wish I could give you a good answer on that...Maybe it's because Google's power is beyond our understanding The only thing I can think of is if your parent directory structure can be browsed, then Google will pick up it up.Rush Quote Link to comment Share on other sites More sharing options...
Russellgs Posted March 19, 2007 Report Share Posted March 19, 2007 Yesterday, I needed to make available some files to someone I've been exchanging emails with. I was uncomfortable sending these rather large files via email so I created a new directory on my TNG website and uploaded the files to that area. I set the directory so just browsing to it would display a list of the contents - easier than providing links to each individual file!So today I was just messing around and happened to look at my server logs - Google had already found the newly created directory and taken a look at every file in it! HOW?The name of this directory does not exist anywhere as a link except in the email I sent to my friend. How did Google find the directory? It is located off my main TNG directory so a index file was present in its parent directory.I had thought the only way the search engines could find a directory (or file) was if there was a link to it somewhere in universe. (assuming the parent directory wasn't browseable). For that matter ... where does Google find the link to tngrss.php? I don't see one anywhere on my site!Ideas?J,I have to questions.1 Have you created a robot.txt file?2 Do you have an .htaccess file?If you don't have either of these then the robots have free reign of you site. If you have a .htaccess file, you may need to deny access to the robots to that folder.Russ Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.