Instruct search engines not to index revision history for posts?
 

Googlebot is picking up a bunch of internal 404 links on my Askbot site. Upon closer inspection, it turns out that those links are found in old revisions of certain posts.

I have since edited those posts to fix the broken links, but of course they are permanently stored in the revision history, which means they will never disappear from search indexers!

I was going to suggest automatically adding rel="nofollow" to links that appear in revision history, but the more I think about it, should post revision pages be indexed by search engines anyway? Maybe they should just be disallowed in robots.txt.

For example, see revision 2 on http://expats.qa/questions/59/revisions/:

While investigating [how to start a business in Costa Rica][1], I opened communication with a Costa Rican law firm.
...
[1]:  /21/hiring-employees-and-managing-an-office-for-a-us/

Note that the link is incorrect; it should be /question/21/.../, not /21/.../.

To enter a block of code:

  • enter empty line after your previous text
  • paste or type the code
  • select the code and press the button above
Preview: (hide)
todofixthis's avatar
1.3k
todofixthis
asked 12 years ago
Evgeny's avatar
13.2k
Evgeny
updated 11 years ago

Comments

Good point!

Chankey Pathak's avatar Chankey Pathak (11 years ago)
see more comments