Shibboleth SSO and umlauts

[mastermarv]

When shibboleth provides text with umlauts in any of the fields used by indico (affiliation, first_name, last_name) this causes an

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

Traceback (most recent call last):
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/flask/util.py", line 114, in wrapper
    return obj().process()
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/rh.py", line 270, in process
    res = self._do_process()
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/rh.py", line 249, in _do_process
    return self._process()
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/modules/auth/controllers.py", line 283, in _process
    moderate_accounts=handler.moderate_registrations)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/views.py", line 118, in render_template
    return cls(g.rh, *wp_args, **context).display()
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/views.py", line 232, in display
    body = to_unicode(self._display(params))
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/views.py", line 262, in _display
    return self._applyDecoration(self._getBody(params))
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/modules/auth/views.py", line 27, in _getBody
    return self._getPageContent(params)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/views.py", line 145, in _getPageContent
    return self.render_template_func(template, **params)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/modules/auth/templates/register.html", line 3, in top-level template code
    {% set show_extra_form = local and not must_verify_email %}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/templates/layout/base.html", line 1, in top-level template code
    {% block page %}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/templates/layout/base.html", line 47, in block "page"
    {% block content %}{% endblock %}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/modules/auth/templates/register.html", line 24, in block "content"
    {% call form_fieldset(_('User information'), render_as_fieldset=(show_extra_form or moderate_accounts)) %}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/jinja2/runtime.py", line 553, in _invoke
    rv = self._func(*arguments)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/templates/forms/_form.html", line 130, in template
    {{ caller() }}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/jinja2/runtime.py", line 553, in _invoke
    rv = self._func(*arguments)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/modules/auth/templates/register.html", line 26, in template
    {{ form_rows(form, fields=('first_name', 'last_name', 'affiliation', 'phone'), widget_attrs=widget_attrs) }}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/jinja2/runtime.py", line 553, in _invoke
    rv = self._func(*arguments)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/templates/forms/_form.html", line 97, in template
    disabled=(true if field.short_name in disable else none), placeholder=placeholder) }}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/jinja2/runtime.py", line 553, in _invoke
    rv = self._func(*arguments)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/templates/forms/_form.html", line 88, in template
    {{ form_field(field, classes, widget_attrs, disabled=disabled) }}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/jinja2/runtime.py", line 553, in _invoke
    rv = self._func(*arguments)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/templates/forms/_form.html", line 46, in template
    {{ _render_field(field, widget_attrs, disabled) }}
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/forms/jinja_helpers.py", line 100, in render_field
    return field(**args)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/wtforms/fields/core.py", line 153, in __call__
    return self.meta.render_field(self, kwargs)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/wtforms/meta.py", line 56, in render_field
    return field.widget(field, **render_kw)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/indico/web/forms/widgets.py", line 174, in __call__
    return self.default_widget(field, **kwargs)
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/wtforms/widgets/core.py", line 163, in __call__
    kwargs['value'] = field._value()
  File "/opt/indico/.venv/local/lib/python2.7/site-packages/wtforms/fields/core.py", line 533, in _value
    return text_type(self.data) if self.data is not None else ''
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

[ThiefMaster]

Can you check which Flask-Multipass version you have (pip freeze | grep -i multipass)? Because this should be fixed in indico/flask-multipass@520b2aa, which is part of flask-multipass since 0.0.dev7, so if you use indico 2.0 (or even a recent 1.9.11.dev) you should not be affected by it.

[mastermarv]

pip freeze | grep -i multipass
Flask-Multipass==0.1
pip freeze | grep -i indico
indico==2.0a1

[ThiefMaster]

found the problem... https://github.com/indico/flask-multipass/blob/master/flask_multipass/providers/shibboleth.py#L22

obviously that check should be isinstance(s, str) instead of isinstance(s, unicode) 🤦‍♂️

could you please try patching this locally and let me know if it works? I'll then release an updated flask-multipass version and make sure to require the new flask-multipass version in the next Indico release. the file to edit is /opt/indico/.venv/lib/python2.7/site-packages/flask_multipass/providers/shibboleth.py

[mastermarv]

it works! thank you.