seo_expert
05-24-2005, 06:30 AM
A robots.txt file is a short text file that resides in your home directory.
Before search engines spider your site, they look into this file to see which files/file types and/or directories they are not allowed to see.
You can use this tool to create your own Robot.txt file.
http://www.1-hit.com/all-in-one/tool-robots.txt-generator.htm
Before search engines spider your site, they look into this file to see which files/file types and/or directories they are not allowed to see.
You can use this tool to create your own Robot.txt file.
http://www.1-hit.com/all-in-one/tool-robots.txt-generator.htm