Wordpress 2.7 Comment pages -Duplicate content issue & solution

Now, soon after the Wordpress 2.7 version was released, along with all the excitement, there was one problem that was not noticed in all the hype. A couple of webmasters complained that the post articles with more than 50 comments created multiple URLs for the same post, because the comments split itself into separate pages every time it crossed 50.

Obviously, this feature was included in Wordpress 2.7 for ease of navigation and better user experience. Wordpress would by default cut a page if the comment number exceeds 50 and create a new URL for comments starting from 51.

The settings can be adjusted in Wordpress >Discussion Area page.

QOT, have discussed this duplicate issue in this post, and also here at WP hacks, where it is said that in order to get out of this situation all you have to do is to uncheck this setting.

Now, I wish if the duplicate content issue ended here. But unfortunately, there was more.

Another webmaster friend of mine (He doesn’t want to disclose himself) complained about something similar.

He had this website that’s really popular in India with news articles. And in one of the sections of the site where he has Wordpress installed, there was something strange happening with comments.

With his popular traffic stats, Google picks up every new published story on blogsearch and Google.com within seconds of publishing. But lately, he found out that whenever someone commented (published after approval) on an already indexed, new story – the comment URL gets picked up by Google.

Ex:- site.com/category/story-commentpage1

And then on, its really strange. Soon after the comment page is indexed, the original URL is replaced in the index, so much that the original URL does not appear for an appropriate search term it used to rank for. Now, the comments URL is indexed and it does not rank for the appropriate term.

Adding more to the problem – Every new story that’s published in the category after this, does not get indexed on Google.com or Blog Search.

So that means, if he let one comment publish in any one story in the section,

- The original URL which used to rank for the appropriate term on Google vanishes.
- The comments URL replaces the original URL but does not rank for search terms it used to rank for.
- The comment url gets picked up in blogsearch.
- No new stories published after this gets indexed on Blog search (only comment URLs does).
- So instead of new posts getting indexed and ranked within minutes of
publishing, indexing of actual post URLs get delayed for hours, then
stops altogether.

Now, that’s a tricky situation right ?

How to fix duplicate comments issue in Wordpress 2.7 ?

This is how I saw it. Technically, as a new comment is published, it goes to the paginated mode, a new URL is made and Google indexes it. But a clear duplication happens here.

According to the search engines,

- There is one original URL, which got indexed first. (Ex:- site.com/category/story.htm)
- And then there’s this comment URL which acts up as duplicate content(Ex:- site.com/category/story-commentspage1.htm)

So technically, every new comment page shows up as duplicate content to Google. (Check screenshots from Google webmasters to prove this)

So how to get over this issue ?

- If only we could 301 redirect all the comment URLs to the original URL. (Lots of complicated commands on .htaccess, confusing and complicated.)
- If only we could stop the comments URL from appearing. (Done – But indexing issue for new posts remained, when u remove the comment pagination, this is done automatically. But
nothing else changes, problems remain.)
- If we could block the robots from indexing all duplicate comment pages (Done, but that’s too tedious a job).

And then came the canonical tags for duplicate content from Google (and others of course) !

With the canonical tags, it would be possible for us to add the canonical tag to all the duplicate comments page, so that even if it created duplicate instances of the same page, the robots would be directed to go back and look for the original content on the original URL. (Tadaaaa !)

Now, this really worked out. But adding canonical tags manually meant a whole lot of time wasted. Instead we used the Canonical Tag plugin from Yoast. This plugin, helped us add the canonical tags to every duplicate content page stating the original URL and now the problem is sorted.

- No duplicate comments URLs.
- No indexing issues for newer posts.
- Search results page ranks restored and working fine.

If you have only a couple of pages on your site with such a problem, you could also do this manually by adding the canonical tags to each of them, picking their URLs from Google webmasters. But if duplicate content keeps popping up, I’d recommend using this plugin so that they are taken care of automatically.

Related SEO Tips and Articles:

• Duplicate content ? Canonical tag to your rescue
  Duplicate content has been discussed and fought over on...
• Avoid duplicate content on Wordpress - How to ?
  Wordpress by default is search engine friendly - a...
• Avoiding duplicate content on wordpress blog
  Did you know that your wordpress blog is not...
• Wordpress comment notifier for Mac
  If you’re yet another Mac + Wordpress lover like...
• Curb Duplicate content with customizable Excerpt plugin for Wordpress
  We’ve discussed earlier about how cutting short the homepage...