Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Robots.txt Generator for Quick Indexing on Search Engines

Generate Robots.txt for free according to your desired search engines. Many times Search engines like google, yahoo, etc are not able to read our site content due to faulty Robots.txt or blocker in it.

What do robots txt mean for SEO?

The robots.txt file, also known as the robots exclusion protocol or standard text file, this is a text file that tells search engine robots that which pages on your site are to be crawled and which not. Many websites don't need a robots.txt file. Because Google can usually find and is able to index all of the important pages on any site. And they'll also automatically NOT index the pages that aren't important or found duplicate versions of other pages.