external pyCSW Service breaks upload of Documents and Datasets

[mwallschlaeger]

Expected Behavior

When configuring an external pycsw service via the environment variables: CATALOGUE_ENGINE and CATALOGUE_URL. Uploading a document or dataset through the webUI should work.

Actual Behavior

When I use the described setup above I get the following error:

Internal Server Error: /documents/upload
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
geonode.documents.models.Document.DoesNotExist: Document matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/usr/local/lib/python3.10/dist-packages/django/template/base.py", line 170, in render
    return self._render(context)
  File "/usr/local/lib/python3.10/dist-packages/django/template/base.py", line 162, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python3.10/dist-packages/django/template/base.py", line 938, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.10/dist-packages/django/template/base.py", line 905, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.10/dist-packages/django/template/loader_tags.py", line 183, in render
    template = context.template.engine.select_template(template_name)
  File "/usr/local/lib/python3.10/dist-packages/django/template/engine.py", line 170, in select_template
    raise TemplateDoesNotExist("No template names provided")
django.template.exceptions.TemplateDoesNotExist: No template names provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/geonode/./geonode/documents/views.py", line 127, in post
    return super().post(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/views/generic/edit.py", line 172, in post
    return super().post(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/views/generic/edit.py", line 142, in post
    return self.form_valid(form)
  File "/usr/src/geonode/./geonode/documents/views.py", line 170, in form_valid
    self.object = resource_manager.create(
  File "/usr/src/geonode/./geonode/resource/manager.py", line 329, in create
    _resource, _created = resource_type.objects.get_or_create(uuid=uuid, defaults=defaults)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/modeltranslation/manager.py", line 419, in get_or_create
    return super().get_or_create(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 588, in get_or_create
    return self.create(**params), True
  File "/usr/local/lib/python3.10/dist-packages/modeltranslation/manager.py", line 411, in create
    return super().create(**kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/src/geonode/./geonode/base/models.py", line 1089, in save
    super().save(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/polymorphic/models.py", line 87, in save
    return super().save(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 787, in save_base
    post_save.send(
  File "/usr/local/lib/python3.10/dist-packages/django/dispatch/dispatcher.py", line 180, in send
    return [
  File "/usr/local/lib/python3.10/dist-packages/django/dispatch/dispatcher.py", line 181, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "/usr/src/geonode/./geonode/catalogue/models.py", line 49, in catalogue_post_save
    catalogue.create_record(instance)
  File "/usr/src/geonode/./geonode/catalogue/backends/generic.py", line 298, in create_record
    md_link = self.catalogue.create_from_dataset(item)
  File "/usr/src/geonode/./geonode/catalogue/backends/generic.py", line 146, in create_from_dataset
    response = self.csw_request(layer, "catalogue/transaction_insert.xml")  # noqa
  File "/usr/src/geonode/./geonode/catalogue/backends/generic.py", line 141, in csw_request
    md_doc = self.csw_gen_xml(layer, template)
  File "/usr/src/geonode/./geonode/catalogue/backends/generic.py", line 132, in csw_gen_xml
    md_doc = tpl.render(context=ctx)
  File "/usr/local/lib/python3.10/dist-packages/django/template/backends/django.py", line 63, in render
    reraise(exc, self.backend)
  File "/usr/local/lib/python3.10/dist-packages/django/template/backends/django.py", line 84, in reraise
    raise new from exc
django.template.exceptions.TemplateDoesNotExist: No template names provided

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.10/dist-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/src/geonode/./geonode/documents/views.py", line 131, in post
    json.dumps(exception_response.data),
AttributeError: 'NoneType' object has no attribute 'data'
[pid: 261|app: 0|req: 1/1] 10.244.0.1 () {76 vars in 1530 bytes} [Mon Jul 31 12:57:45 2023] POST /documents/upload?no__redirect=true => generated 318559 bytes in 355 msecs (HTTP/1.1 500) 8 headers in 295 bytes (2 switches on core 0)

Steps to Reproduce the Problem

1. setup a pycsw (I used pycsw docker v2.6.0)
2. configure the geonode to use this pycsw endpoint
3. upload a document or dataset

Specifications

This is part of a feature development in geonode-k8s to run pycsw as an external service within the kubernetes deployment. (zalf-rdm/geonode-k8s#27, zalf-rdm/geonode-k8s#34). It can be tested using the geonode-k8s minikube installation

• GeoNode version: 4.1.x
• Installation type (vanilla, geonode-project): kubernetes
• Installation method (manual, docker): kubernetes
• Platform:
• Additional details: