If you have a lot of pages you'd like to prevent search engines from crawling, there are better ways to accomplish that besides adding them to the robots.txt, however, you should definitely ensure that this file isn't blocking important URLs for some reason.
Hope this serves as a quick sanity check. If you have further questions, please leave them in the comments and thanks for watching!
0 Comments