Create a robots.txt file

The robots.txt file is used to instruct search engine robots about
what pages on your website should be crawled and consequently indexed.
Most websites have files and folders that are not relevant for search
engines (like images or admin files) therefore creating a robots.txt...