First time here? Check out the FAQ!
0

Sitemap URL is example.com in robots.txt

When I view my site's robots.txt file in my browser, I notice that the Sitemap URL is incorrect:

http://expats.askbot.com/robots.txt:

User-agent: *
Disallow:

Sitemap: http://example.com/sitemap.xml

Note that the URL for the sitemap uses example.com as its domain.

Following the instructions in http://askbot.org/en/question/2715/sitemapxml-is-using-examplecom, I changed the domain for my website, and now http://expats.askbot.com/sitemap.xml is displaying the correct URLs, but robots.txt is still broken.

todofixthis's avatar
1.3k
todofixthis
asked 2012-03-23 09:31:44 -0500, updated 2012-04-14 15:39:55 -0500
edit flag offensive 0 remove flag close merge delete

Comments

1

Thanks for reporting this, will take a look now and fix it.

Fitoria's avatar Fitoria (2012-03-23 09:52:47 -0500) edit
add a comment see more comments

1 Answer

1

Fixed:

http://expats.askbot.com/robots.txt

User-agent: *
Disallow:


Sitemap: http://expats.askbot.com/sitemap.xml
Fitoria's avatar
1.1k
Fitoria
answered 2012-03-23 12:19:46 -0500
edit flag offensive 0 remove flag delete link

Comments

Thanks! Will this take effect for any domain name (i.e., as specified in Django admin)? Or will it need to be manually updated if/when I change the domain name for the site?

todofixthis's avatar todofixthis (2012-03-23 12:53:03 -0500) edit

for the moment this is set according to your domain name you cannot change it.

Fitoria's avatar Fitoria (2012-03-23 12:55:30 -0500) edit

Looks like this might be getting set globally; I accessed <http://expats.askbot.com/robots.txt> just now, and the Sitemap: entry is now http://help.simplecv.org/sitemap.xml. According to GWT, http://askcre.com/sitemap.xml has also appeared in robots.txt.

todofixthis's avatar todofixthis (2012-03-26 00:19:15 -0500) edit
add a comment see more comments