First time here? Check out the FAQ!
2

Askbot crashing on questions with åäö

Hi! Iv'e a clean intstall of Askbot on a CentOS-server with uWsgi. It's behind an Nginx (on an other server). Evrything works fine. Can create users, ask questions etc. Until I put a non ascii-character (Swedish åäö) in the title of a question.

The question is stored properly, And is possible to read on the /question/X/Testfrågan-url. I is also stored correctly in the databas in the askbot_post-table.

But I get a UnicodeDecodeError on /questions/ or /users/X/User for the user who has written the question. The UnicodeDecodeError complains about the title.

The string that could not be encoded/decoded was: estfr��gan<

I have tried with different setting for ALLOW_UNICODE_SLUGS and ASKBOT_TRANSLATE_URL, and droped the database and retried. Database is UTF-8, and so is the DATABASES.TEST_CHARSET, DATABASES.TEST_COLLATION.

What do I do wrong? How can I fix this?

Versions:
CentOS 7.2.1511
Askbot: 0.10.0
Django: 1.8.13
Python: 2.7.5
uWSgi 2.0.13.1
PostgreSQL: 9.3

Traceback:

File "/var/www/askbot/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/var/www/askbot/lib/python2.7/site-packages/askbot/views/readers.py" in questions
  287.         return render(request, 'main_page.html', template_data)
File "/var/www/askbot/lib/python2.7/site-packages/django/shortcuts.py" in render
  67.             template_name, context, request=request, using=using)
File "/var/www/askbot/lib/python2.7/site-packages/django/template/loader.py" in render_to_string
  99.         return template.render(context, request)
File "/var/www/askbot/lib/python2.7/site-packages/askbot/skins/template_backends.py" in render
  86.         return self.template.render(context)
File "/var/www/askbot/lib/python2.7/site-packages/coffin/template/__init__.py" in render
  55.         return super(Template, self).render(**context)
File "/var/www/askbot/lib/python2.7/site-packages/jinja2/environment.py" in render
  989.         return self.environment.handle_exception(exc_info, True)
File "/var/www/askbot/lib/python2.7/site-packages/jinja2/environment.py" in handle_exception
  754.         reraise(exc_type, exc_value, tb)
File "/var/www/askbot/lib/python2.7/site-packages/askbot/templates/main_page.html" in top-level template code
  2. {% import "macros.html" as macros %}
File "/var/www/askbot/lib/python2.7/site-packages/askbot/templates/two_column_body.html" in top-level template code
  1. {% extends "base.html" %}
File "/var/www/askbot/lib/python2.7/site-packages/askbot/templates/base.html" in top-level template code
  46.             {% block body %}
File "/var/www/askbot/lib/python2.7/site-packages/askbot/templates/two_column_body.html" in block "body"
  5.     {% block content%}
File "/var/www/askbot/lib/python2.7/site-packages/askbot/templates/main_page.html" in block "content"
  21.         {% include "main_page/questions_loop.html" %}
File "/var/www/askbot/lib/python2.7/site-packages/askbot/templates/main_page/questions_loop.html" in top-level template code
  6.         {{ thread.get_summary_html(search_state=search_state, visitor = request.user) }}

Exception Type: UnicodeDecodeError at /questions/
Exception Value: 'ascii' codec can't decode byte 0xc3 in position 1064: ordinal not in range(128)
Tvartom's avatar
31
Tvartom
asked 2016-06-13 16:58:34 -0500
edit flag offensive 0 remove flag close merge delete

Comments

Why not update Python to 2.7.11 or 2.7.12?

slelievre's avatar slelievre (2016-07-13 02:25:13 -0500) edit

Thanks for your suggest! I uppgraded to 2.7.12. Still no success! Anyone having any idea? Should I try Python 3.3?

Tvartom's avatar Tvartom (2016-12-01 19:46:30 -0500) edit
add a comment see more comments

2 Answers

1

Finally I found a solution! The problem is memcached !

Succedded to trace down the problem to

#File:askbot/models/question.py: Line 1678
html = self.get_cached_summary_html(visitor) or self.update_summary_html(visitor)

If I remove the "get_cached" part:

html = self.update_summary_html(visitor)

it work like a charm.

Stopped digging there, and with restored code I and stopped the memcache service, and suddenly everything worked.

Tvartom's avatar
31
Tvartom
answered 2016-12-01 21:02:43 -0500, updated 2016-12-21 09:26:38 -0500
edit flag offensive 0 remove flag delete link

Comments

@Tvartom what you are suggesting is to not cache the summary html fragments that are shown on the main page. This will make the main page load quite a bit slower.

Evgeny's avatar Evgeny (2016-12-20 13:46:15 -0500) edit
1

No, I'm not suggesting to not cache this, I just told you how I did find out.

Tvartom's avatar Tvartom (2016-12-21 09:26:08 -0500) edit

Ok, thanks, it did help to pinpoint this issue to caching.

Evgeny's avatar Evgeny (2016-12-21 09:30:00 -0500) edit
add a comment see more comments
0

What seems to fix the issue - add to settings.py:

import sys
reload(sys) #this brings method setdefaultencoding
sys.setdefaultencoding('utf-8')

As @Tvartom suggested - this issue is seen when the django memcache caching backend is used. I find it strange how django memcached cache backend works (even after the default encoding is set to utf-8):

>>> from django.core.cache import cache
>>> cache.set('key', u'\u1537')
>>> cache.get('key')
'\xe1\x94\xb7'

Note that the value comes back as encoded string, not as original value.

With Redis cache (and similarly with the LocMemCache) I get what I'd expect:

>>> cache.set('key', u'\u1537')
True
>>> cache.get('key')
u'\u1537'

Redis cache also works irrespective of the default encoding.

Because of that I would recommend to use django-redis-cache.

Evgeny's avatar
13.2k
Evgeny
answered 2016-12-20 13:45:06 -0500, updated 2016-12-21 06:49:45 -0500
edit flag offensive 0 remove flag delete link

Comments

add a comment see more comments