Create a custom robots.txt file for your website. Add user‑agent rules, allow/disallow paths, and include a sitemap URL. Copy or download the file instantly.
Control how search engines crawl your site. Block specific directories or files, set crawl delays, and point to your sitemap.
All generation happens in your browser. No data sent to any server.
Add multiple user‑agent rules, allow/disallow paths – see the output update in real time.
1️⃣ Add one or more user‑agent rules (e.g., *, Googlebot, Bingbot).
2️⃣ For each rule, specify paths to allow or disallow.
3️⃣ Optionally add your sitemap URL.
4️⃣ Click "Generate robots.txt" – the file content is displayed.
5️⃣ Copy the text or download as .txt to upload to your website root.
✅ A properly configured robots.txt helps search engines index your site correctly.
The robots.txt file tells search engine crawlers which pages or directories they can or cannot access. It’s essential for preventing duplicate content, hiding staging areas, or blocking sensitive data. Our generator lets you build a complete, standards‑compliant file without any technical hassle. All data stays in your browser – your rules are never sent elsewhere.
It applies to all search engine crawlers. You can also specify specific bots like Googlebot, Bingbot, etc.
Yes – order matters. Disallow directives take precedence over Allow in most bots, but we output them in the order you provide.
Place it in the root directory of your website (e.g., https://example.com/robots.txt).