Robots.txt

Robots.txt is a text file that allows SEO's to control the access of search engine spiders on the website.

Back to the web design and SEO glossary