Robots.txt Generator

Default - All Robots are

Crawl-Delay

Sitemap

Leave blank if you don't have.

Search Robots:

Google
Google Image
Google Mobile
MSN Search
Yahoo
Yahoo MM
Yahoo Blogs
Ask/Teoma
GigaBlast
DMOZ Checker
Nutch
Alexa/Wayback
Baidu
Naver
MSN PicSearch

Disallow Folders

The path is relative to the root and must contain a trailing slash "/".

URL Decode

JavaScript Beautifier

JSON to JSON Schema

Page Size Checker

WordPress Theme Detector

Htaccess Redirect Generator

Robots.txt Generator

Generate an SEO-friendly robots.txt file with our free Robots.txt Generator. This tool will help you control which parts of your site can be crawled by search engines, making sure only the right pages are indexed. Using a well-configured robots.txt file will go a long way in improving your website crawlability and overall SEO performance.