Developer Tools
Tutorials
Build
Resume Builder
Create professional, ATS-friendly resumes
Form Builder
Forms, surveys, quizzes and exams
Analytics
Visualize and analyze your data
Work
Projects
Apply, submit work and get paid
Team
Meet the people behind StepsEasy
Books
Articles
Tech News
Log in
Register
Developer Tools
Tutorials
Build
Resume Builder
Form Builder
Analytics
Work
Projects
Team
Books
Articles
Tech News
Log in
Register
Tools
Robots.txt Generator
Robots.txt Generator
Create a robots.txt file with allow/disallow rules.
✓
Input
Rule Group 1
User-Agent
* (All bots)
Googlebot
Bingbot
Yahoo Slurp
DuckDuckBot
Baiduspider
YandexBot
Facebook
Twitterbot
AhrefsBot
Rules
Disallow
Allow
Add Rule
Add Rule Group
Additional Settings
Sitemap URL
Crawl-Delay (seconds)
Process
Copy
Clear
Output
Copy