Exclude your staging site from Google with robots.txt and not shoot yourself in the foot
If you want to exclude your staging site from Google using robots.txt
without running the risk to forget deleting the file once you go live, name the file robots.exclude.txt
instead.
In your Apache Vhost config, rewrite requests for the staging server only:
CopyRewriteEngine On RewriteCond %{HTTP_HOST} ^staging.project.com$ RewriteRule ^/robots.txt$ /robots.exclude.txt
Your robots.exclude.txt
looks like this:
Copy# This file is returned for /robots.txt on staging servers User-agent: * Disallow: /
Important Note: If your setup is incorrect and /robots.txt
is not accessible, it means there is no protection at all!
makandra has been working exclusively with Ruby on Rails since 2007. Our laser focus on a single technology has made us a leader in this space.