Robots.txt Generator

<h1>Free Website SEO Tools</h1>

Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

The free robots.txt generator tools creates a robots.txt file to help ensure Google and other search engines can crawl and index your website properly.

Every website needs a robots.txt file to help Google and other search engine bots crawl and index your website so it appears in the Google search results. If a website does not have a robots.txt file it can make it difficult for search engines to crawl the website and this can have an impact on the websites ranking in Google.

How Does The Robots.txt generator tool work

The robots.txt generator is a free to use tool which allows you to easily generate a correctly formated robots.txt file which Google, Yahoo and Bing can understand.

All you need to do is choose the options inclued in the tool and then select the create robots.txt button.

The tool will cerate a robots.txt file which you need to upload to the root of your website or /public_hrml or /home directory so Google and other website can access the file and crawl the website.

For advice about the robots.txt read the support article at the Google website https://support.google.com/webmasters/answer/6062608?hl=en

Why Should I Use The Robots.txt File

As noted above the robots.txt file tells google and other search engines how to crawl your website. This helps google understand your website and access all the resources at the website. The file also allows you to tell Google not to crawl specific parts or your website which may include directories, files, images, Java scripts and more. The free tools at our website makes it easy for non technical people to quickly and easily generate robots.txt files with a click of a button.