Removed organization breaks search facets

[abulte]

Detailed Description

When an organization is removed and purged, some search result pages (including list ones) will be broken with the traceback below.

This is probably due to the organization still being indexed somewhere in ElasticSearch.

Steps to reproduce:

• create an organization
• create a dataset in this organization
• remove the organization
• purge the organization (job)
• visit /datasets, you should get an error 400 - "None" is not valid identifier

Traceback (most recent call last):
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/Users/alexandre/Developer/Etalab/udata/udata/frontend/views.py", line 36, in dispatch_request
    return super(BaseView, self).dispatch_request(*args, **kwargs)
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/flask/views.py", line 149, in dispatch_request
    return meth(*args, **kwargs)
  File "/Users/alexandre/Developer/Etalab/udata/udata/frontend/views.py", line 93, in get
    return self.render()
  File "/Users/alexandre/Developer/Etalab/udata/udata/frontend/views.py", line 25, in render
    return theme.render(self.get_template_name(), **context)
  File "/Users/alexandre/Developer/Etalab/udata/udata/theme/__init__.py", line 132, in render
    return render_theme_template(get_theme(theme), template, **context)
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/flask_themes2/__init__.py", line 141, in render_theme_template
    return render_template(template_name, **context)
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/dataset/list.html", line 14, in top-level template code
    {% set bundle = 'search' %}
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/layouts/1-column.html", line 4, in top-level template code
    {% set toolbar_class = toolbar_class|default('') %}
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/base.html", line 1, in top-level template code
    {% extends theme('raw.html') %}
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/raw.html", line 29, in top-level template code
    {% block body %}{% endblock %}
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/base.html", line 19, in block "body"
    {% block content %}{% endblock %}
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/layouts/1-column.html", line 14, in block "content"
    {% block main_content %}{% endblock %}
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/dataset/list.html", line 59, in block "main_content"
    {% include theme('dataset/search-panel.html') %}
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/dataset/search-panel.html", line 4, in top-level template code
    {{ s.model_terms_facet(datasets,
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/jinja2/runtime.py", line 553, in _invoke
    rv = self._func(*arguments)
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/macros/search.html", line 138, in template
    {% call facet_panel(result, name, label, icon, url) %}
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/jinja2/runtime.py", line 553, in _invoke
    rv = self._func(*arguments)
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/macros/search.html", line 99, in template
    {{ caller() }}
  File "/Users/alexandre/Developer/Etalab/udata/pyenv/lib/python2.7/site-packages/jinja2/runtime.py", line 553, in _invoke
    rv = self._func(*arguments)
  File "/Users/alexandre/Developer/Etalab/udata/udata/templates/macros/search.html", line 142, in template
    {{ result.query.facets[name].labelize(obj) }}
  File "/Users/alexandre/Developer/Etalab/udata/udata/search/fields.py", line 52, in labelize
    return labelize(value)
  File "/Users/alexandre/Developer/Etalab/udata/udata/search/fields.py", line 137, in default_labelizer
    self.validate_parameter(value)
  File "/Users/alexandre/Developer/Etalab/udata/udata/search/fields.py", line 151, in validate_parameter
    raise ValueError('"{0}" is not valid identifier'.format(value))
ValueError: "None" is not valid identifier

Possible Implementation

Debug or implement the reindexation after an organization is removed and/or purged.

Your Environment

dev branch, local. Also happens on master in production.

[abulte]

Traceback in production (slightly different):

[u'DoesNotExist: Organization matching query does not exist.', u'  File 
"flask/app.py", line 1982, in wsgi_app', u'  File "flask/app.py", line 
1614, in full_dispatch_request', u'  File "flask_restplus/api.py", line 
536, in error_router', u'  File "flask/app.py", line 1517, in 
handle_user_exception', u'  File "flask/app.py", line 1612, in 
full_dispatch_request', u'  File "flask/app.py", line 1598, in 
dispatch_request', u'  File "udata/core/topic/views.py", line 73, in 
datasets', u'  File "udata/theme.py", line 147, in render', u'  File 
"flask_themes2/__init__.py", line 141, in render_theme_template', u'  
File "flask/templating.py", line 134, in render_template', u'  File 
"flask/templating.py", line 116, in _render', u'  File 
"jinja2/environment.py", line 969, in render', u'  File 
"jinja2/environment.py", line 742, in handle_exception', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/topic/datasets.html",
 line 13, in top-level template code', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata_gouvfr/theme/templates/topic/display_base.html",
 line 1, in top-level template code', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/topic/display_base.html",
 line 12, in top-level template code', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata_gouvfr/theme/templates/base.html",
 line 1, in top-level template code', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/base.html",
 line 1, in top-level template code', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata_gouvfr/theme/templates/raw.html",
 line 1, in top-level template code', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/raw.html",
 line 29, in top-level template code', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/base.html",
 line 19, in block "body"', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/topic/display_base.html",
 line 15, in block "content"', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/macros/breadcrumb.html",
 line 5, in template', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/topic/datasets.html",
 line 26, in block "breadcrumb"', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/dataset/search-labels.html",
 line 3, in top-level template code', u'  File 
"/srv/udata/local/lib/python2.7/site-packages/udata/templates/macros/search.html",
 line 76, in template', u'  File "udata/search/fields.py", line 50, in 
labelize', u'  File "udata/search/fields.py", line 50, in 
<genexpr>', u'  File "udata/search/fields.py", line 139, in 
default_labelizer', u'  File "mongoengine/queryset/base.py", line 242, 
in get']

[abulte]

The broken facet is:

    {{ s.model_terms_facet(datasets,
        'organization', _('Organizations'), ficon('fa-building-o'), url=url
    ) }}

[abulte]

Workaround: udata search reindex -t Dataset

[abulte]

Fixed in 1.1.0