Redirects to Avoid Homepage Duplicate Content

Submitted by Adam King on 20 September, 2007 - 20:37.

There are many potential issues that can hurt your SEO when you set up your domains and websites. One of the more serious issues is duplicate content. This article provides solutions to issues with duplicate content with regard to arguably the most important page on a site - the homepage.

Types of Homepage Duplication, and Some Solutions

Please note that the following solutions are designed to be used on Apache systems and involve editing the .htaccess file. Similar solutions for sites on IIS are possible but are not within the scope of this article. For more information on mod_rewrite check out the Apache documentation.

1) WWW and Non-WWW Duplication

Scenario: You can access your site's homepage by typing your domain name with or without the www prefix (it doesn't redirect). You need to use a single version of your domain. How do you force the site to only use your chosen version?

Examples:

http://example.com/
http://www.example.com/

Solution:

To force the www version (http://www.example.com/):

 RewriteCond %{HTTP_HOST} ^example\.com [NC]
  RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L] 

To force the non-www version (http://example.com/):

 RewriteCond %{HTTP_HOST} ^www\.example\.com [NC]
  RewriteRule ^(.*)$ http://example.com/$1 [R=301,L] 

Source: http://www.webmasterworld.com/forum92/1435.htm

2) Multiple Domain Names

Scenario: You have purchased multiple versions of your domain name for the purposes of brand protection. You do not have seperate sites for each TLD, so you have a main .com domain and also other TLDs which all need to take people to the homepage of your site. How do you ensure each domain doesn't duplicate your content at the same time as leaving example.com as the single, canonical version of your homepage?

Examples:

http://www.example.co.uk/
http://www.example.com/
http://www.example.net/

Solution:

To ensure that all of your domain names redirect to your preferred domain by correctly using a 301 redirect:

 RewriteCond %{HTTP_HOST} ^(www\.)?example\.co\.uk [NC,OR]
  RewriteCond %{HTTP_HOST} ^(www\.)?example\.net [NC]
  RewriteCond %{QUERY_STRING} ^(.*)$
  RewriteRule ^(.*)$ http://www.example.com/$1?%1 [R=301,L]

Note: The above solution incorporates code to force the www version of the domain.

Source: http://www.webmasterworld.com/forum92/208.htm

3) Index-Page Duplication

Scenario: The homepage of your site is displayed/served by an index file in the root of your site. You can access the homepage using your domain name, but you can also call the index file directly. How do you ensure that the index file is redirected to the root domain without creating an infinite loop?

Examples:

http://www.example.com/
http://www.example.com/index.php

Solution:

This following code redirects all requests for /index.php to the root domain using a 301 redirect (and it doesn’t cause an infinite loop).

 RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.php\ HTTP/
  RewriteRule ^index\.php$ http://www.example.com/ [R=301,L] 

Source: http://www.webmasterworld.com/forum30/32672.htm

4) Sub-Domain Duplication

Scenario A: You don't use subdomains on your site but, due to your site's configuration, typing in a subdomain displays the main homepage of the site without any redirection. How do you correctly configure the subdomains to prevent content duplication?

Scenario B: You do use subdomains on your site but you can access the subdomains with or without using www. How do you prevent people accessing your subdomains using multiple versions of the url (with and without www)?

Examples:

http://forum.example.com/
http://forum.example.com/index.php
http://www.forum.example.com/
http://www.forum.example.com/index.php

Solutions for Scenario A:

To redirect all possible subdomains to the root domain:

  RewriteCond %{HTTP_HOST} .
  RewriteCond %{HTTP_HOST} !^www\.example\.com [NC]
  RewriteRule (.*) http://www.example.com/$1 [R=301,L] 

Source: http://www.webmasterworld.com/forum30/32672.htm

To redirect a specific subdomain to the root domain:

 RewriteCond %{HTTP_HOST} ^forum\.example\.com$ [NC,OR]
  RewriteCond %{HTTP_HOST} ^www\.forum\.example\.com$ [NC]
  RewriteRule ^(.*)$ http://www.example.com/ [R=301,L] 

Source: Unknown

Solutions for Scenario B:

To remove "www." from any subdomain requests:

  RewriteCond %{HTTP_HOST} ^www\.([^.]+)\.example\.com [NC]
  RewriteRule (.*) http://%1.example.com/$1 [R=301,L] 

Source: http://www.webmasterworld.com/forum92/4491.htm

Final Thoughts

In researching the issues surrounding duplicate content I found many good resources on the web which address these problems. The advice above was gleaned from the WebMasterWorld forums. I must thank jdMorgan for his seemingly boundless knowledge about all things involving .htaccess and his endless desire to help.

Bookmark/Search this post with: