This website uses cookies to personalise ads and to analyse traffic ok
web design

Modify WordPress auto-generated robots.txt without plugin

WordPress generates automatically its own robots.txt and the best way to add your own rules in it, is to use the robots_txt filter.

Instead of using plugins or manually creating your robots.txt file, you can use WordPress robots_txt filter to modify the auto-generated robots.txt and add your rules.

Edit your theme (or child theme) functions.php and add the following snippet:

function add_to_robotstxt($output, $public) {
    $my_rules = "
# Added by robots_txt filter in functions
# Add your rules below, e.g.: 
Disallow: /wp-content/themes/
";
    return $output . $my_rules;
}

add_filter('robots_txt', 'add_to_robotstxt');

That’s it. That’s the preferred way to do it, since you don’t want a whole plugin just for a few lines of code and you don’t really want to create your own robots.txt file and miss any of the rules added by WordPress 😉

Divi WordPress Theme Monarch Social Sharing Plugin


3 Comments

Post a comment
  1. Mmal75
    Posted on January 29, 2023 at 05:56 Permalink

    Good post. You helped me 🙂

    Reply
    • Mmal75
      Posted on January 29, 2023 at 06:06 Permalink

      Should it be UTF8?

      Reply
  2. Surojit Dutta
    Posted on March 16, 2020 at 18:26 Permalink

    Very Good And Informative Post For All Webmaster. Here is a similar post About Robots.txt you can check here https://surojitdutta.com/optimize-wordpress-robots-txt-seo/

    Reply

Post a comment

Your e-mail is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>
You can write or copy/paste code directly in your comment using the <code> tag:
<code>code here...</code>
You may also use the data-lang attribute to determine the code language like so:
<code data-lang-html>, <code data-lang-css>, <code data-lang-js> and <code data-lang-php>

css.php