Fix UnicodeDecode errors when displaying the result page.

[sneridagh]

We experienced an issue on FormLikertFields in the result page when using unicode characters in the answers. It was caused because the result page is implicitly casting whatever it gets from the fields to str. I added an explicit cast and conversion to utf-8.

We need access to pypi for making a release as soon as possible. Please could you grant permissions to the "timo" pypi username?

Thanks!

[sneridagh]

@mauritsvanrees @smcmahon Could you please take a look at this?

Travis failed at the 4.2 build with no reason apparent. The 4.1 and python 2.6 also fails, because of setuptools I guess...

[tisto]

@smcmahon @mauritsvanrees sorry to bug you here but we really need to ship this fix to a client. We can fix and release on our own if we have permissions...

[mauritsvanrees]

I am looking into this now. I have meanwhile restarted the Travis build for 4.2, I did not see a log at all that I could look into.
When you are in a hurry, you can of course make an internal release and use that.

[mauritsvanrees]

I have fixed the 4.2 build by pinning i18ndude to 4.3. See collective/buildout.plonetest#32. That should not have been the original error, because I only released the conflicting i18ndude 5 yesterday.
If you rebase on 1.7-maintenance, Travis should pass. But I would say skip that: I will believe that your small fix does not somehow break 4.2.

But how do I reproduce the issue you are fixing? I added a LikertField, changed the last sample answer to Strôngly agrée and selected that as one of the values when filling in the form, and the thank-you page with the result showed up just fine.

The fix seems weird at first sight:

str(request.form.get(fname, 'No Input')).decode('utf-8')

• If the request input is a unicode with accented characters, then calling str on it will give an error.
• If the request input is a string, then calling str on it will have no effect. I guess calling decode should be fine.

[sneridagh]

@mauritsvanrees The thing is that in the case of a Likert field, we are casting a dict (the content of the field) to str directly. Then it fails, because of the encoding. It's strange that worked for you.

Here you can find where it's failing for me:
https://www.monitor-versorgungsforschung.de/Innofonds-umfrage

[mauritsvanrees]

Ah wait, I used a Thanks Page, and that does not use the template you fixed.
I see the error now.

[sneridagh]

@mauritsvanrees Mmm, regarding the implementation you're right, it fixes the dict use case, but it could fail if the field is str.

[mauritsvanrees]

Huh. Well, I see that on master this was already fixed in the same way in PR #200.
Fair enough. :-) Apparently it works without bad side effects.
Okay, I will merge and release.

[mauritsvanrees]

I have released 1.7.24 with this fix. Thank you!

cc @smcmahon

[sneridagh]

Oh, great! Thanks!

[tisto]

Thank you @mauritsvanrees!