The Best Robot.txt File for SEO in WordPress

/in /by

Here is a sample WordPress robots.txt file that you can use for your wordpress powered site, because guess what… WordPress doesn’t come with a WordPress robots.txt file by default, yep thats right, unless you have added a WordPress robots.txt file there will not be one there at all. So, here you go (this one will work just fine but you can obviously customize it to suite your specific needs by adding more directories to disallow access to.

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /feed
Disallow: /comments
Disallow: /category/*/*
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /*?*
Disallow: /*?
Allow: /wp-content/uploads

# Google Image
User-agent: Googlebot-Image
Disallow:
Allow: /*

# Google AdSense
User-agent: Mediapartners-Google*
Disallow:
Allow: /*

# Internet Archiver Wayback Machine
User-agent: ia_archiver
Disallow: /

# digg mirror
User-agent: duggmirror
Disallow: /