ThyElite Forums - Googles hidden files

ThyElite Forums (http://www.thyelite.com/forum/index.php)

- General Talk (http://www.thyelite.com/forum/forumdisplay.php?f=14)

- - Googles hidden files (http://www.thyelite.com/forum/showthread.php?t=1817)

MYMOMMAGOTBEEF

08-05-2006 05:55 PM

Googles hidden files

Just snooping around and found google's robot file. http://google.com/robots.txt

mjmorrell

08-05-2006 06:22 PM

Wow that is where every website's robot file has to be if they have it :l

Heff	08-05-2006 09:28 PM

Quote:

Originally Posted by YOMOMMAGOTBEEF

Just snooping around and found google's robot file. http://google.com/robots.txt

You are an idiot, everyone has a robots file to tell what the robots should crawl

All times are GMT -6. The time now is 10:38 AM.