First time here? Check out the FAQ!
Ask Your Question
0

Sitemap URL is example.com in robots.txt
 
   
  
 
 
When I view my site's robots.txt file in my browser, I notice that the Sitemap URL is incorrect:
 
http://expats.askbot.com/robots.txt:
 
User-agent: *
Disallow:

Sitemap: http://example.com/sitemap.xml

 
Note that the URL for the sitemap uses example.com as its domain.
 
Following the instructions in http://askbot.org/en/question/2715/sitemapxml-is-using-examplecom, I changed the domain for my website, and now http://expats.askbot.com/sitemap.xml is displaying the correct URLs, but robots.txt is still broken.
 
 
To enter a block of code:
 
     
  • enter empty line after your previous text
    •  
  • paste or type the code
    •  
  • select the code and press the  button above
    •  
 
Preview: (hide)
  
 
 
 
 
 
 todofixthis's avatar

      1.3k
    
 
 
 
 todofixthis 
 
 
    asked 13 years ago,
        
    updated 13 years ago 
 
 
 
 
        
 
 
 
Comments
 
 
 
 
1
  
 
 
 
Thanks for reporting this, will take a look now and fix it.
 
 
 Fitoria's avatar Fitoria (13 years ago)   
 
 
  
 
  see more comments  
 
 
 
 
 
 1 Answer
  
  
  
 
 
 
 
1
 
 
  
 
 
 
 
 
 
Fixed:
 
http://expats.askbot.com/robots.txt
 
User-agent: *
Disallow:


Sitemap: http://expats.askbot.com/sitemap.xml

 
 
To enter a block of code:
 
     
  • enter empty line after your previous text
    •  
  • paste or type the code
    •  
  • select the code and press the  button above
    •  
 
Preview: (hide)
  
 
 
 
 
 
 Fitoria's avatar

      1.1k
    
 
 
 
 Fitoria 
 
 
    answered 13 years ago 
 
 
 
 
     link   
 
 
 
Comments
 
 
 
 
  
 
 
 
Thanks!  Will this take effect for any domain name (i.e., as specified in Django admin)?  Or will it need to be manually updated if/when I change the domain name for the site?
 
 
 todofixthis's avatar todofixthis (13 years ago)   
 
 
  
 
 
  
 
 
 
for the moment this is set according to your domain name you cannot change it.
 
 
 Fitoria's avatar Fitoria (13 years ago)   
 
 
  
 
 
  
 
 
 
Looks like this might be getting set globally; I accessed <http://expats.askbot.com/robots.txt> just now, and the Sitemap: entry is now http://help.simplecv.org/sitemap.xml.  According to GWT, http://askcre.com/sitemap.xml has also appeared in robots.txt.
 
 
 todofixthis's avatar todofixthis (13 years ago)   
 
 
  
 
  see more comments