Instruct search engines not to index revision history for posts?

Googlebot is picking up a bunch of internal 404 links on my Askbot site. Upon closer inspection, it turns out that those links are found in old revisions of certain posts.

I have since edited those posts to fix the broken links, but of course they are permanently stored in the revision history, which means they will never disappear from search indexers!

I was going to suggest automatically adding rel="nofollow" to links that appear in revision history, but the more I think about it, should post revision pages be indexed by search engines anyway? Maybe they should just be disallowed in robots.txt.

For example, see revision 2 on http://expats.qa/questions/59/revisions/:

While investigating [how to start a business in Costa Rica][1], I opened communication with a Costa Rican law firm.
...
[1]:  /21/hiring-employees-and-managing-an-office-for-a-us/

Note that the link is incorrect; it should be /question/21/.../, not /21/.../.

todofixthis's avatar
1.3k
todofixthis
asked 2013-01-02 12:09:30 -0500
Evgeny's avatar
13.2k
Evgeny
updated 2013-07-04 05:23:33 -0500
edit flag offensive 0 remove flag close merge delete

Comments

Good point!

Chankey Pathak's avatar Chankey Pathak (2013-07-04 05:09:12 -0500) edit
add a comment see more comments