Corriger erreur 500 sur upload de fichier

[maudetes]

Voir https://mattermost.incubateur.net/betagouv/pl/bbziyxojzpb1dg11tm48de9k5e

Soucis d'upload de fichiers dans certains cas, exemple sur https://www.data.gouv.fr/fr/admin/dataset/64060c2ac773dcf3fabbe5d2/.

Logs:

[2023-05-22 15:31:00,152] ERROR in app: Exception on /api/1/datasets/64060c2ac773dcf3fabbe5d2/upload/ [POST]
Traceback (most recent call last):
  File "/srv/udata/lib/python3.7/site-packages/mongoengine/queryset/base.py", line 583, in update
    result = update_func(query, update, upsert=upsert)
  File "/srv/udata/lib/python3.7/site-packages/pymongo/collection.py", line 1052, in update_one
    comment=comment,
  File "/srv/udata/lib/python3.7/site-packages/pymongo/collection.py", line 837, in _update_retryable
    (write_concern or self.write_concern).acknowledged and not multi, _update, session
  File "/srv/udata/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1476, in _retryable_write
    return self._retry_with_session(retryable, func, s, None)
  File "/srv/udata/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1349, in _retry_with_session
    return self._retry_internal(retryable, func, session, bulk)
  File "/srv/udata/lib/python3.7/site-packages/pymongo/_csot.py", line 105, in csot_wrapper
    return func(self, *args, **kwargs)
  File "/srv/udata/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1390, in _retry_internal
    return func(session, sock_info, retryable)
  File "/srv/udata/lib/python3.7/site-packages/pymongo/collection.py", line 833, in _update
    comment=comment,
  File "/srv/udata/lib/python3.7/site-packages/pymongo/collection.py", line 782, in _update
    _check_write_command_response(result)
  File "/srv/udata/lib/python3.7/site-packages/pymongo/helpers.py", line 217, in _check_write_command_response
    _raise_last_write_error(write_errors)
  File "/srv/udata/lib/python3.7/site-packages/pymongo/helpers.py", line 190, in _raise_last_write_error
    raise WriteError(error.get("errmsg"), error.get("code"), error)
pymongo.errors.WriteError: The field 'resources' must be an array but is of type null in document {_id: ObjectId('64060c2ac773dcf3fabbe5d2')}, full error: {'index': 0, 'code': 2, 'errmsg': "The field 'resources' must be an array but is of type null in document {_id: ObjectId('64060c2ac773dcf3fabbe5d2')}"}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/udata/lib/python3.7/site-packages/flask/app.py", line 1523, in full_dispatch_request
    rv = self.dispatch_request()
  File "/srv/udata/lib/python3.7/site-packages/flask/app.py", line 1509, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/srv/udata/lib/python3.7/site-packages/udata/api/__init__.py", line 132, in wrapper
    return func(*args, **kwargs)
  File "/srv/udata/lib/python3.7/site-packages/flask_restx/api.py", line 404, in wrapper
    resp = resource(*args, **kwargs)
  File "/srv/udata/lib/python3.7/site-packages/flask/views.py", line 84, in view
    return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs)
  File "/srv/udata/lib/python3.7/site-packages/flask_restx/resource.py", line 46, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/srv/udata/lib/python3.7/site-packages/udata/api/__init__.py", line 123, in wrapper
    return func(*args, **kwargs)
  File "/srv/udata/lib/python3.7/site-packages/flask_restx/marshalling.py", line 244, in wrapper
    resp = f(*args, **kwargs)
  File "/srv/udata/lib/python3.7/site-packages/udata/core/dataset/api.py", line 384, in post
    dataset.add_resource(resource)
  File "/srv/udata/lib/python3.7/site-packages/udata/core/dataset/models.py", line 760, in add_resource
    '$position': 0
  File "/srv/udata/lib/python3.7/site-packages/mongoengine/document.py", line 640, in update
    return self._qs.filter(**self._object_key).update_one(**kwargs)
  File "/srv/udata/lib/python3.7/site-packages/mongoengine/queryset/base.py", line 648, in update_one
    **update,
  File "/srv/udata/lib/python3.7/site-packages/mongoengine/queryset/base.py", line 594, in update
    raise OperationError("Update failed (%s)" % err)
mongoengine.errors.OperationError: Update failed (The field 'resources' must be an array but is of type null in document {_id: ObjectId('64060c2ac773dcf3fabbe5d2')}, full error: {'index': 0, 'code': 2, 'errmsg': "The field 'resources' must be an array but is of type null in document {_id: ObjectId('64060c2ac773dcf3fabbe5d2')}"})

Question bonus : Pourquoi pas d'erreur sentry associée ?

[quaxsze]

Erreur liée au fait que monoengine semble mettre un ListField vide en champ unset ou null.
De nombreuses discussions semblent exister sur le sujet :
MongoEngine/mongoengine#267
MongoEngine/mongoengine#761
et une MR en attente depuis 2 ans :
MongoEngine/mongoengine#2517

Le probleme a été résolu de notre côté en passant la commande suivante sur notre base Mongo afin d'expliciter les listes vides :
db.dataset.updateMany({"resources": null}, {"$set": {"resources": []}})