First time here? Check out the FAQ!
2

"502 Bad Gateway" when visiting certain URLs, Part 3

Googlebot found another bad URL:

http://expats.qa/get-editor/

When viewing it on my hosted site, I get an Nginx 502 Bad Gateway error.

todofixthis's avatar
1.3k
todofixthis
asked 2012-10-25 00:21:49 -0500
edit flag offensive 0 remove flag close merge delete

Comments

add a comment see more comments

1 Answer

2

I've fixed this in the repository. Replaced with Access Forbidden, maybe "Bad Request" will be better. This request requires some parameters to work, it's a part of the internal api.

Evgeny's avatar
13.2k
Evgeny
answered 2012-10-25 00:24:32 -0500
edit flag offensive 0 remove flag delete link

Comments

Google sure is good at finding these URLs. Would it be prudent to start adding them to robots.txt?

todofixthis's avatar todofixthis (2012-11-15 09:19:49 -0500) edit
1

Yes, the non-content urls should not be crawled.

Evgeny's avatar Evgeny (2012-11-15 09:22:57 -0500) edit
add a comment see more comments