Skip to content

Commit

Permalink
Loosened import version-restriction to x.y instead of x.y.z.
Browse files Browse the repository at this point in the history
closes #2269
  • Loading branch information
ggainey authored and dralley committed Apr 12, 2022
1 parent 4cdba44 commit 962d28a
Show file tree
Hide file tree
Showing 4 changed files with 91 additions and 21 deletions.
1 change: 1 addition & 0 deletions CHANGES/2269.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Loosened the version-restrictions on PulpImport to only require X.Y matching.
65 changes: 49 additions & 16 deletions docs/workflows/import-export.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,22 +37,6 @@ In order to minimize space utilization, import/export operates on sets of
:term:`Artifacts<Artifact>` only once per-export, rather than once for each
:term:`Repository` being exported.

.. note::

Export will not operate on :term:`RepositoryVersions<RepositoryVersion>` that have
been synchronized using ``policy=on_demand``. :term:`Artifacts<Artifact>` must actually
exist in order to be exported - this is, after all the only way for the Downstream Pulp
instance to gain access to them!

.. note::

Import and Export strictly control which directories may be read from/written to via
the settings options ``ALLOWED_IMPORT_PATHS`` and ``ALLOWED_EXPORT_PATHS``.
These default to empty, if not explicitly set attempts to import or export will fail
with a validation error like

``"Path '/tmp/exports/' is not an allowed export path"``

Definitions
^^^^^^^^^^^
Upstream
Expand Down Expand Up @@ -87,6 +71,55 @@ Import order
ModelResources be imported in order. Plugins are responsible for specifying the
import-order of the ModelResources they own

Assumptions
^^^^^^^^^^^

The import/export workflow operates on a set of assumptions. Violating them will result
in error-messages as described below.

On-Demand content not supported
-------------------------------

Export will not operate on :term:`RepositoryVersions<RepositoryVersion>` that have
been synchronized using ``policy=on_demand`` or ``policy=streamed``. :term:`Artifacts<Artifact>`
must actually exist in order to be exported - this is, after
all the only way for the Downstream Pulp instance to gain access to them!

If a repository is specified for export that utilized on-demand/streamed syncing, the
export will fail with a RuntimeError:

``Remote artifacts cannot be exported.``

Export/Import Directories must be explicitly allowed
----------------------------------------------------

Import and Export strictly control which directories may be read from/written to via
the settings options ``ALLOWED_IMPORT_PATHS`` and ``ALLOWED_EXPORT_PATHS``.
These default to empty - if they not explicitly set, attempts to import or export will fail
with a validation error like

``"Path '/tmp/exports/' is not an allowed export path"``

Installed plugins must match
----------------------------

A Downstream must support the complete set of plugins present in a given export. If the
export includes plugins that are not installed in the Downstream, an import attempt will
fail with a validation error like

``Export uses pulp_rpm which is not installed.``

Version-compatibility required
------------------------------

The export-to-import workflow is built on the assumption that the Upstream and
Downstream instances are running "compatible" versions of pulpcore and plugins. In this
context, "compatible" is defined as **"share the same X.Y version"**. If this is not the
case, an import attempt will fail with a validation error like

``Export version 3.14.15 of pulpcore incompatible with installed version 3.16.3.``


Exporting
^^^^^^^^^

Expand Down
18 changes: 13 additions & 5 deletions pulpcore/app/tasks/importer.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,12 @@ def _import_file(fpath, resource_class, retry=False):


def _check_versions(version_json):
"""Compare the export version_json to the installed components."""
"""
Compare the export version_json to the installed components.
An upstream whose db-metadata doesn't match the downstream won't import successfully; check
for compatibility and raise a ValidationError if incompatible versions are found.
"""
error_messages = []
for component in version_json:
try:
Expand All @@ -101,10 +106,13 @@ def _check_versions(version_json):
_("Export uses {} which is not installed.").format(component["component"])
)
else:
if version != component["version"]:
# Check that versions are compatible. Currently, "compatible" is defined as "same X.Y".
# Versions are strings that generally look like "X.Y.Z" or "X.Y.Z.dev"; we check that
# first two places are the same.
if version.split(".")[:2] != component["version"].split(".")[:2]:
error_messages.append(
_(
"Export version {export_ver} of {component} does not match "
"Export version {export_ver} of {component} incompatible with "
"installed version {ver}."
).format(
export_ver=component["version"],
Expand All @@ -113,8 +121,8 @@ def _check_versions(version_json):
)
)

if error_messages:
raise ValidationError((" ".join(error_messages)))
if error_messages:
raise ValidationError((" ".join(error_messages)))


def import_repository_version(importer_pk, destination_repo_pk, source_repo_name, tar_path):
Expand Down
28 changes: 28 additions & 0 deletions pulpcore/tests/unit/test_import_checks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
from unittest.mock import patch

from django.test import TestCase
from rest_framework.serializers import ValidationError

from pulpcore.app.tasks.importer import _check_versions


class TestObject:
version = "1.2.3" # Every component is vers 1.2.3


class TestCheckVersions(TestCase):
@patch("pulpcore.app.tasks.importer.get_distribution", return_value=TestObject())
def test_vers_check(self, mock_get_distribution):
export_json = [{"component": "xyz", "version": "1.2.3"}]
_check_versions(export_json)

export_json = [{"component": "xy", "version": "1.2"}]
_check_versions(export_json)

export_json = [{"component": "x_noty_z", "version": "1.4.3"}]
with self.assertRaises(ValidationError):
_check_versions(export_json)

export_json = [{"component": "notx_y_z", "version": "2.2.3"}]
with self.assertRaises(ValidationError):
_check_versions(export_json)

0 comments on commit 962d28a

Please sign in to comment.