Robots.txt can disallow and allow search engine to fetch some web pages, however, robots.txt file not be added in wordpress system defaultly.
In this tutorial, we will introduce how to create this file with Yoast SEO Plugin online.
Yoast SEO Plugin is a very popular wordpress seo plugin, most of wordpress sites have installed it. So create robots.txt by this plugin is very easy and acceptable.
How to create a robots.txt with Yoast?
Step 1. Open Yoast Tool
Step 2. Click File editor
Step 3. Create a robots.txt file
Step 4. Add content in robots.txt
