DreamTeam Money Making ForumRegister
Link To Us
Advertise on DTMHelpLog In
Welcome Guest ( Log In | Register )
What is the role of robot.txt file in SEO.Someone told me that robot.txt file is use to stop the crawling of search engine on particular website.
Robot.txt files cache your webpage & Google save it in library..
It tells search robots which files to ignore (or alternatively) which files to crawl. It also helps Search Engines to locate the Sitemap of the website and hence crawl the entire website in depth... helping in your rankings and traffic.Here is all about it:http://searchenginewatch.com/article/20644...Robots.txt-File
Thanks for sharing the fruitful info about this topic.
Robot.txt is text file which allow search engine which page crawl or which one not.
It concerns about all the indexing issues , and doorway pages mirror pages like these , help crawler to crawl
robots.txt is nothing but the text file which tells the search engine what page to crawl and what page not to crawl.