Error passing upload RPM

[fao89]

Author: wibbit (wibbit)

Redmine Issue: 9551, https://pulp.plan.io/issues/9551

---

After uploading an RPM (rpm -qp --all .rpm works fine on the source RPM), I'm greeted with the following error.

{'child_tasks': [],
 'created_resources': [],
 'error': {'description': 'RPM file cannot be parsed for metadata.',
           'traceback': '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", '
                        'line 272, in _perform_task\n'
                        '    result = func(*args, **kwargs)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/app/tasks/base.py", '
                        'line 34, in general_create\n'
                        '    serializer.is_valid(raise_exception=True)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
                        'line 220, in is_valid\n'
                        '    self._validated_data = '
                        'self.run_validation(self.initial_data)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/rest_framework/serializers.py", '
                        'line 422, in run_validation\n'
                        '    value = self.validate(value)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/serializers/content.py", '
                        'line 104, in validate\n'
                        '    data = self.deferred_validate(data)\n'
                        '  File '
                        '"/opt/bats/lib/python3.8/site-packages/pulp_rpm/app/serializers/package.py", '
                        'line 240, in deferred_validate\n'
                        '    raise NotAcceptable(detail="RPM file cannot be '
                        'parsed for metadata.")\n'},
 'finished_at': datetime.datetime(2021, 11, 2, 19, 12, 12, 382621, tzinfo=tzutc()),
 'logging_cid': '3c5f6c83815947bfae0e71cb661ec91f',
 'name': 'pulpcore.app.tasks.base.general_create',
 'parent_task': None,
 'progress_reports': [],
 'pulp_created': datetime.datetime(2021, 11, 2, 19, 12, 10, 711408, tzinfo=tzutc()),
 'pulp_href': '/pulp/api/v3/tasks/452c9c7e-1715-4751-9462-0f68a649feb3/',
 'reserved_resources_record': ['/pulp/api/v3/artifacts/58eb9338-26f8-4f55-a0dc-ddca290fa1fb/'],
 'started_at': datetime.datetime(2021, 11, 2, 19, 12, 10, 792329, tzinfo=tzutc()),
 'state': 'failed',
 'task_group': None,
 'worker': None}

[fao89]

From: wibbit (wibbit)
Date: 2021-11-02T19:35:41Z

---

For reference this RPM is 1.5GB in size.

[fao89]

From: @goosemania (ttereshc)
Date: 2021-11-11T18:29:00Z

---

Pulp needs to show the real issue or at least log the traceback and not hide the root cause.

In the meantime, you can patch your setup to see the full traceback by removing try-except block :

diff --git a/pulp_rpm/app/serializers/package.py b/pulp_rpm/app/serializers/package.py
index 3d7660f7..b6628da2 100644
--- a/pulp_rpm/app/serializers/package.py
+++ b/pulp_rpm/app/serializers/package.py
@@ -234,11 +234,7 @@ class PackageSerializer(SingleArtifactContentUploadSerializer, ContentChecksumSe
         """
         data = super().deferred_validate(data)
         # export META from rpm and prepare dict as saveable format
-        try:
-            new_pkg = _prepare_package(data["artifact"], data["relative_path"])
-        except OSError:
-            raise NotAcceptable(detail="RPM file cannot be parsed for metadata.")
-
+        new_pkg = _prepare_package(data["artifact"], data["relative_path"])
         attrs = {key: new_pkg[key] for key in Package.natural_key_fields()}
         package = Package.objects.filter(**attrs)

[fao89]

From: @goosemania (ttereshc)
Date: 2021-11-15T13:28:47Z

---

The issue was running out of disc space but it is unexpected.
The suspicion is that the /tmp is used instead of the worker working directory.

Please give this patch a try

diff --git a/pulp_rpm/app/shared_utils.py b/pulp_rpm/app/shared_utils.py
index 8771c1de..fc52aae1 100644
--- a/pulp_rpm/app/shared_utils.py
+++ b/pulp_rpm/app/shared_utils.py
@@ -23,7 +23,7 @@ def _prepare_package(artifact, filename):
         filename: name of file uploaded by user
     """
     artifact_file = storage.open(artifact.file.name)
-    with tempfile.NamedTemporaryFile("wb", suffix=filename) as temp_file:
+    with tempfile.NamedTemporaryFile("wb", dir=".", suffix=filename) as temp_file:
         shutil.copyfileobj(artifact_file, temp_file)
         temp_file.flush()
         cr_pkginfo = createrepo_c.package_from_rpm(temp_file.name)

[fao89]

From: wibbit (wibbit)
Date: 2021-11-15T13:42:12Z

---

I can confirm that this was the issue, and by including the dir="." allowed the package to get uploaded and the rest of the process to complete as expected.

[fao89]

From: pulpbot (pulpbot)
Date: 2021-11-15T18:11:09Z

---

PR: #2178

[fao89]

From: @goosemania (ttereshc)
Date: 2021-11-23T15:11:45Z

---

Applied in changeset commit:fdcc1b4bb05c94829d17a9e8c43250bea76bddf2.

[dralley]

#2319 (comment)