diff --git a/docs/tutorial_advanced.rst b/docs/tutorial_advanced.rst index 775864447c..0cee1a05a5 100644 --- a/docs/tutorial_advanced.rst +++ b/docs/tutorial_advanced.rst @@ -671,13 +671,14 @@ ReFrame can be used also to test applications that run inside a container. First, we need to enable the container platform support in ReFrame's configuration and, specifically, at the partition configuration level: .. literalinclude:: ../tutorials/config/settings.py - :lines: 38-58 - :emphasize-lines: 15-20 + :lines: 38-62 + :emphasize-lines: 15-24 For each partition, users can define a list of container platforms supported using the :js:attr:`container_platforms` `configuration parameter `__. -In this case, we define the `Singularity `__ platform, for which we set the :js:attr:`modules` parameter in order to instruct ReFrame to load the ``singularity`` module, whenever it needs to run with this container platform. +In this case, we define the `Sarus `__ platform for which we set the :js:attr:`modules` parameter in order to instruct ReFrame to load the ``sarus`` module, whenever it needs to run with this container platform. +Similarly, we add an entry for the `Singularity `__ platform. -The following test will use a Singularity container to run: +The following parameterized test, will create two tests, one for each of the supported container platforms: .. code-block:: console @@ -690,23 +691,36 @@ The following test will use a Singularity container to run: A container-based test can be written as :class:`RunOnlyRegressionTest ` that sets the :attr:`container_platform ` attribute. This attribute accepts a string that corresponds to the name of the container platform that will be used to run the container for this test. -In this case, the test will be using `Singularity `__ as a container platform. If such a platform is not `configured `__ for the current system, the test will fail. -As soon as the container platform to be used is defined, you need to specify the container image to use and the commands to run inside the container by setting the :attr:`image ` and the :attr:`commands ` container platform attributes. -These two attributes are mandatory for container-based checks. +As soon as the container platform to be used is defined, you need to specify the container image to use by setting the :attr:`image `. +In the ``Singularity`` test variant, we add the ``docker://`` prefix to the image name, in order to instruct ``Singularity`` to pull the image from `DockerHub `__. +The default command that the container runs can be overwritten by setting the :attr:`command ` attribute of the container platform. + +The :attr:`image ` is the only mandatory attribute for container-based checks. It is important to note that the :attr:`executable ` and :attr:`executable_opts ` attributes of the actual test are ignored in case of container-based tests. -ReFrame will run the container as follows: +ReFrame will run the container according to the given platform as follows: + +.. code-block:: bash + + # Sarus + sarus run --mount=type=bind,source="/path/to/test/stagedir",destination="/rfm_workdir" ubuntu:18.04 bash -c 'cat /etc/os-release | tee /rfm_workdir/release.txt' + + # Singularity + singularity exec -B"/path/to/test/stagedir:/rfm_workdir" docker://ubuntu:18.04 bash -c 'cat /etc/os-release | tee /rfm_workdir/release.txt' -.. code-block:: console - singularity exec -B"/path/to/test/stagedir:/workdir" docker://ubuntu:18.04 bash -c 'cd rfm_workdir; pwd; ls; cat /etc/os-release' +In the ``Sarus`` case, ReFrame will prepend the following command in order to pull the container image before running the container: -By default ReFrame will mount the stage directory of the test under ``/rfm_workdir`` inside the container and it will always prepend a ``cd`` command to that directory. -The user commands are then run from that directory one after the other. +.. code-block:: bash + + sarus pull ubuntu:18.04 + + +This is the default behavior of ReFrame, which can be changed if pulling the image is not desired by setting the :attr:`pull_image ` attribute to :class:`False`. +By default ReFrame will mount the stage directory of the test under ``/rfm_workdir`` inside the container. Once the commands are executed, the container is stopped and ReFrame goes on with the sanity and performance checks. -Users may also change the default mount point of the stage directory by using :attr:`workdir ` attribute: Besides the stage directory, additional mount points can be specified through the :attr:`mount_points ` attribute: .. code-block:: python @@ -714,5 +728,29 @@ Besides the stage directory, additional mount points can be specified through th self.container_platform.mount_points = [('/path/to/host/dir1', '/path/to/container/mount_point1'), ('/path/to/host/dir2', '/path/to/container/mount_point2')] + +The container filesystem is ephemeral, therefore, ReFrame mounts the stage directory under ``/rfm_workdir`` inside the container where the user can copy artifacts as needed. +These artifacts will therefore be available inside the stage directory after the container execution finishes. +This is very useful if the artifacts are needed for the sanity or performance checks. +If the copy is not performed by the default container command, the user can override this command by settings the :attr:`command ` attribute such as to include the appropriate copy commands. +In the current test, the output of the ``cat /etc/os-release`` is available both in the standard output as well as in the ``release.txt`` file, since we have used the command: + +.. code-block:: bash + + bash -c 'cat /etc/os-release | tee /rfm_workdir/release.txt' + + +and ``/rfm_workdir`` corresponds to the stage directory on the host system. +Therefore, the ``release.txt`` file can now be used in the subsequent sanity checks: + +.. code-block:: python + + os_release_pattern = r'18.04.\d+ LTS \(Bionic Beaver\)' + self.sanity_patterns = sn.all([ + sn.assert_found(os_release_pattern, 'release.txt'), + sn.assert_found(os_release_pattern, self.stdout) + ]) + + For a complete list of the available attributes of a specific container platform, please have a look at the :ref:`container-platforms` section of the :doc:`regression_test_api` guide. On how to configure ReFrame for running containerized tests, please have a look at the :ref:`container-platform-configuration` section of the :doc:`config_reference`. diff --git a/docs/tutorial_basics.rst b/docs/tutorial_basics.rst index c7bb1edd3f..79439d747d 100644 --- a/docs/tutorial_basics.rst +++ b/docs/tutorial_basics.rst @@ -323,7 +323,7 @@ Note that you should *not* edit this configuration file in place. Here is how the new configuration file looks like with the needed additions highlighted: .. literalinclude:: ../tutorials/config/settings.py - :lines: 10-24,76-97,130- + :lines: 10-24,80-101,134- :emphasize-lines: 3-15,31-42 Here we define a system named ``catalina`` that has one partition named ``default``. @@ -807,7 +807,7 @@ Let's extend our configuration file for Piz Daint. .. literalinclude:: ../tutorials/config/settings.py - :lines: 10-45,58-66,73- + :lines: 10-45,62-70,77- :emphasize-lines: 16-48,70-101,114-120 diff --git a/reframe/core/containers.py b/reframe/core/containers.py index f1a8a34eee..a57d2ba63a 100644 --- a/reframe/core/containers.py +++ b/reframe/core/containers.py @@ -10,25 +10,65 @@ from reframe.core.exceptions import ContainerError +_STAGEDIR_MOUNT = '/rfm_workdir' + + class ContainerPlatform(abc.ABC): '''The abstract base class of any container platform.''' + #: The default mount location of the test case stage directory inside the + #: container + #: The container image to be used for running the test. #: #: :type: :class:`str` or :class:`None` #: :default: :class:`None` image = fields.TypedField(str, type(None)) + #: The command to be executed within the container. + #: + #: If no command is given, then the default command of the corresponding + #: container image is going to be executed. + #: + #: .. versionadded:: 3.5.0 + #: Changed the attribute name from `commands` to `command` and its type + #: to a string. + #: + #: :type: :class:`str` or :class:`None` + #: :default: :class:`None` + command = fields.TypedField(str, type(None)) + + _commands = fields.TypedField(typ.List[str]) #: The commands to be executed within the container. #: + #: .. deprecated:: 3.5.0 + #: Please use the `command` field instead. + #: #: :type: :class:`list[str]` #: :default: ``[]`` - commands = fields.TypedField(typ.List[str]) + commands = fields.DeprecatedField( + _commands, + 'The `commands` field is deprecated, please use the `command` field ' + 'to set the command to be executed by the container.', + fields.DeprecatedField.OP_SET, from_version='3.5.0' + ) + + #: Pull the container image before running. + #: + #: This does not have any effect for the `Singularity` container platform. + #: + #: .. versionadded:: 3.5 + #: + #: :type: :class:`bool` + #: :default: ``True`` + pull_image = fields.TypedField(bool) #: List of mount point pairs for directories to mount inside the container. #: #: Each mount point is specified as a tuple of - #: ``(/path/in/host, /path/in/container)``. + #: ``(/path/in/host, /path/in/container)``. The stage directory of the + #: ReFrame test is always mounted under ``/rfm_workdir`` inside the + #: container, independelty of this field. #: #: :type: :class:`list[tuple[str, str]]` #: :default: ``[]`` @@ -40,25 +80,40 @@ class ContainerPlatform(abc.ABC): #: :default: ``[]`` options = fields.TypedField(typ.List[str]) + _workdir = fields.TypedField(str, type(None)) #: The working directory of ReFrame inside the container. #: #: This is the directory where the test's stage directory is mounted inside #: the container. This directory is always mounted regardless if #: :attr:`mount_points` is set or not. #: + #: .. deprecated:: 3.5 + #: Please use the `options` field to set the working directory. + #: #: :type: :class:`str` #: :default: ``/rfm_workdir`` - workdir = fields.TypedField(str, type(None)) + workdir = fields.DeprecatedField( + _workdir, + 'The `workdir` field is deprecated, please use the `options` field to ' + 'set the container working directory', + fields.DeprecatedField.OP_SET, from_version='3.5.0' + ) def __init__(self): self.image = None - self.commands = [] + self.command = None + + # NOTE: Here we set the target fields directly to avoid the deprecation + # warnings + self._commands = [] + self._workdir = _STAGEDIR_MOUNT + self.mount_points = [] self.options = [] - self.workdir = '/rfm_workdir' + self.pull_image = True @abc.abstractmethod - def emit_prepare_commands(self): + def emit_prepare_commands(self, stagedir): '''Returns commands for preparing this container for running. Such a command could be for pulling the container image from a @@ -70,10 +125,12 @@ def emit_prepare_commands(self): platform backends. :meta private: + + :arg stagedir: The stage directory of the test. ''' @abc.abstractmethod - def launch_command(self): + def launch_command(self, stagedir): '''Returns the command for running :attr:`commands` with this container platform. @@ -82,15 +139,14 @@ def launch_command(self): platforms. :meta private: + + :arg stagedir: The stage directory of the test. ''' def validate(self): if self.image is None: raise ContainerError('no image specified') - if not self.commands: - raise ContainerError('no commands specified') - def __str__(self): return type(self).__name__ @@ -102,17 +158,24 @@ class Docker(ContainerPlatform): '''Container platform backend for running containers with `Docker `__.''' - def emit_prepare_commands(self): - return [] + def emit_prepare_commands(self, stagedir): + return [f'docker pull {self.image}'] if self.pull_image else [] - def launch_command(self): - super().launch_command() - run_opts = ['-v "%s":"%s"' % mp for mp in self.mount_points] + def launch_command(self, stagedir): + super().launch_command(stagedir) + mount_points = self.mount_points + [(stagedir, _STAGEDIR_MOUNT)] + run_opts = [f'-v "{mp[0]}":"{mp[1]}"' for mp in mount_points] run_opts += self.options - run_cmd = 'docker run --rm %s %s bash -c ' % (' '.join(run_opts), - self.image) - return run_cmd + "'" + '; '.join( - ['cd ' + self.workdir] + self.commands) + "'" + + if self.command: + return (f'docker run --rm {" ".join(run_opts)} ' + f'{self.image} {self.command}') + + if self.commands: + return (f"docker run --rm {' '.join(run_opts)} {self.image} " + f"bash -c 'cd {self.workdir}; {'; '.join(self.commands)}'") + + return f'docker run --rm {" ".join(run_opts)} {self.image}' class Sarus(ContainerPlatform): @@ -130,27 +193,34 @@ def __init__(self): self.with_mpi = False self._command = 'sarus' - def emit_prepare_commands(self): + def emit_prepare_commands(self, stagedir): # The format that Sarus uses to call the images is # //:. If an image was loaded # locally from a tar file, the is 'load'. - if self.image.startswith('load/'): + if not self.pull_image or self.image.startswith('load/'): return [] - - return [self._command + ' pull %s' % self.image] - - def launch_command(self): - super().launch_command() - run_opts = ['--mount=type=bind,source="%s",destination="%s"' % - mp for mp in self.mount_points] + else: + return [f'{self._command} pull {self.image}'] + + def launch_command(self, stagedir): + super().launch_command(stagedir) + mount_points = self.mount_points + [(stagedir, _STAGEDIR_MOUNT)] + run_opts = [f'--mount=type=bind,source="{mp[0]}",destination="{mp[1]}"' + for mp in mount_points] if self.with_mpi: run_opts.append('--mpi') run_opts += self.options - run_cmd = self._command + ' run %s %s bash -c ' % (' '.join(run_opts), - self.image) - return run_cmd + "'" + '; '.join( - ['cd ' + self.workdir] + self.commands) + "'" + + if self.command: + return (f'{self._command} run {" ".join(run_opts)} {self.image} ' + f'{self.command}') + + if self.commands: + return (f"{self._command} run {' '.join(run_opts)} {self.image} " + f"bash -c 'cd {self.workdir}; {'; '.join(self.commands)}'") + + return f'{self._command} run {" ".join(run_opts)} {self.image}' class Shifter(Sarus): @@ -177,20 +247,26 @@ def __init__(self): super().__init__() self.with_cuda = False - def emit_prepare_commands(self): + def emit_prepare_commands(self, stagedir): return [] - def launch_command(self): - super().launch_command() - run_opts = ['-B"%s:%s"' % mp for mp in self.mount_points] + def launch_command(self, stagedir): + super().launch_command(stagedir) + mount_points = self.mount_points + [(stagedir, _STAGEDIR_MOUNT)] + run_opts = [f'-B"{mp[0]}:{mp[1]}"' for mp in mount_points] if self.with_cuda: run_opts.append('--nv') run_opts += self.options - run_cmd = 'singularity exec %s %s bash -c ' % (' '.join(run_opts), - self.image) - return run_cmd + "'" + '; '.join( - ['cd ' + self.workdir] + self.commands) + "'" + if self.command: + return (f'singularity exec {" ".join(run_opts)} ' + f'{self.image} {self.command}') + + if self.commands: + return (f"singularity exec {' '.join(run_opts)} {self.image} " + f"bash -c 'cd {self.workdir}; {'; '.join(self.commands)}'") + + return f'singularity run {" ".join(run_opts)} {self.image}' class ContainerPlatformField(fields.TypedField): @@ -203,6 +279,6 @@ def __set__(self, obj, value): value = globals()[value]() except KeyError: raise ValueError( - 'unknown container platform: %s' % value) from None + f'unknown container platform: {value}') from None super().__set__(obj, value) diff --git a/reframe/core/pipeline.py b/reframe/core/pipeline.py index 41d052dc5c..c9a8946b6d 100644 --- a/reframe/core/pipeline.py +++ b/reframe/core/pipeline.py @@ -138,6 +138,7 @@ class RegressionMixin(metaclass=RegressionTestMeta): .. versionadded:: 3.4.2 ''' + def __getattribute__(self, name): try: return super().__getattribute__(name) @@ -1289,9 +1290,6 @@ def run(self): 'on the current partition: %s' % e) from None self.container_platform.validate() - self.container_platform.mount_points += [ - (self._stagedir, self.container_platform.workdir) - ] # We replace executable and executable_opts in case of containers self.executable = self.container_platform.launch_command() diff --git a/tutorials/advanced/containers/container_test.py b/tutorials/advanced/containers/container_test.py index b2f469c36c..d47bef17ac 100644 --- a/tutorials/advanced/containers/container_test.py +++ b/tutorials/advanced/containers/container_test.py @@ -7,20 +7,20 @@ import reframe.utility.sanity as sn -@rfm.simple_test +@rfm.parameterized_test(['Sarus'], ['Singularity']) class ContainerTest(rfm.RunOnlyRegressionTest): - def __init__(self): - self.descr = 'Run commands inside a container' + def __init__(self, platform): + self.descr = f'Run commands inside a container using {platform}' self.valid_systems = ['daint:gpu'] self.valid_prog_environs = ['builtin'] - self.container_platform = 'Singularity' - self.container_platform.image = 'docker://ubuntu:18.04' - self.container_platform.commands = [ - 'pwd', 'ls', 'cat /etc/os-release' - ] - self.container_platform.workdir = '/workdir' + image_prefix = 'docker://' if platform == 'Singularity' else '' + self.container_platform = platform + self.container_platform.image = f'{image_prefix}ubuntu:18.04' + self.container_platform.command = ( + "bash -c 'cat /etc/os-release | tee /rfm_workdir/release.txt'" + ) + os_release_pattern = r'18.04.\d+ LTS \(Bionic Beaver\)' self.sanity_patterns = sn.all([ - sn.assert_found(r'^' + self.container_platform.workdir, - self.stdout), - sn.assert_found(r'18.04.\d+ LTS \(Bionic Beaver\)', self.stdout), + sn.assert_found(os_release_pattern, 'release.txt'), + sn.assert_found(os_release_pattern, self.stdout) ]) diff --git a/tutorials/config/settings.py b/tutorials/config/settings.py index 1e415d7361..1a2748d990 100644 --- a/tutorials/config/settings.py +++ b/tutorials/config/settings.py @@ -50,6 +50,10 @@ } ], 'container_platforms': [ + { + 'type': 'Sarus', + 'modules': ['sarus'] + }, { 'type': 'Singularity', 'modules': ['singularity'] diff --git a/unittests/test_containers.py b/unittests/test_containers.py index 5910a50057..846bda0498 100644 --- a/unittests/test_containers.py +++ b/unittests/test_containers.py @@ -6,14 +6,16 @@ import pytest import reframe.core.containers as containers +import reframe.core.warnings as warn from reframe.core.exceptions import ContainerError @pytest.fixture(params=[ - 'Docker', - 'Sarus', 'Sarus+mpi', 'Sarus+localimage', - 'Shifter', 'Shifter+mpi', 'Shifter+localimage', - 'Singularity', 'Singularity+cuda' + 'Docker', 'Docker+nocommand', 'Docker+nopull', + 'Sarus', 'Sarus+nocommand', 'Sarus+nopull', 'Sarus+mpi', 'Sarus+load', + 'Shifter', 'Shifter+nocommand', 'Shifter+mpi', 'Shifter+nopull', + 'Shifter+load', + 'Singularity', 'Singularity+nocommand', 'Singularity+cuda' ]) def container_variant(request): return request.param @@ -23,68 +25,102 @@ def container_variant(request): def container_platform(container_variant): name = container_variant.split('+')[0] ret = containers.__dict__[name]() + if '+nocommand' not in container_variant: + ret.command = 'cmd' + if '+mpi' in container_variant: ret.with_mpi = True if '+cuda' in container_variant: ret.with_cuda = True - if '+localimage' in container_variant: + if '+load' in container_variant: ret.image = 'load/library/image:tag' else: ret.image = 'image:tag' + if '+nopull' in container_variant: + ret.pull_image = False + return ret @pytest.fixture def expected_cmd_mount_points(container_variant): - if container_variant == 'Docker': - return ('docker run --rm -v "/path/one":"/one" -v "/path/two":"/two" ' - "image:tag bash -c 'cd /stagedir; cmd1; cmd2'") - elif container_variant == 'Sarus': + if container_variant in {'Docker', 'Docker+nopull'}: + return ('docker run --rm -v "/path/one":"/one" ' + '-v "/path/two":"/two" ' + '-v "/foo":"/rfm_workdir" image:tag cmd') + elif container_variant == 'Docker+nocommand': + return ('docker run --rm -v "/path/one":"/one" ' + '-v "/path/two":"/two" ' + '-v "/foo":"/rfm_workdir" image:tag') + elif container_variant in {'Sarus', 'Sarus+nopull'}: + return ('sarus run ' + '--mount=type=bind,source="/path/one",destination="/one" ' + '--mount=type=bind,source="/path/two",destination="/two" ' + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + 'image:tag cmd') + elif container_variant == 'Sarus+nocommand': return ('sarus run ' '--mount=type=bind,source="/path/one",destination="/one" ' '--mount=type=bind,source="/path/two",destination="/two" ' - "image:tag bash -c 'cd /stagedir; cmd1; cmd2'") + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + 'image:tag') elif container_variant == 'Sarus+mpi': return ('sarus run ' '--mount=type=bind,source="/path/one",destination="/one" ' '--mount=type=bind,source="/path/two",destination="/two" ' - "--mpi image:tag bash -c 'cd /stagedir; cmd1; cmd2'") - elif container_variant == 'Sarus+localimage': + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--mpi image:tag cmd') + elif container_variant == 'Sarus+load': return ('sarus run ' '--mount=type=bind,source="/path/one",destination="/one" ' '--mount=type=bind,source="/path/two",destination="/two" ' - "load/library/image:tag bash -c 'cd /stagedir; cmd1; cmd2'") - elif container_variant == 'Singularity': - return ('singularity exec -B"/path/one:/one" -B"/path/two:/two" ' - "image:tag bash -c 'cd /stagedir; cmd1; cmd2'") - elif container_variant == 'Singularity+cuda': - return ('singularity exec -B"/path/one:/one" -B"/path/two:/two" ' - "--nv image:tag bash -c 'cd /stagedir; cmd1; cmd2'") - elif container_variant == 'Shifter': + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + 'load/library/image:tag cmd') + elif container_variant in {'Shifter', 'Shifter+nopull'}: return ('shifter run ' '--mount=type=bind,source="/path/one",destination="/one" ' '--mount=type=bind,source="/path/two",destination="/two" ' - "image:tag bash -c 'cd /stagedir; cmd1; cmd2'") - elif container_variant == 'Shifter+localimage': + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + 'image:tag cmd') + elif container_variant == 'Shifter+nocommand': return ('shifter run ' '--mount=type=bind,source="/path/one",destination="/one" ' '--mount=type=bind,source="/path/two",destination="/two" ' - "load/library/image:tag bash -c 'cd /stagedir; cmd1; cmd2'") + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + 'image:tag') elif container_variant == 'Shifter+mpi': return ('shifter run ' '--mount=type=bind,source="/path/one",destination="/one" ' '--mount=type=bind,source="/path/two",destination="/two" ' - "--mpi image:tag bash -c 'cd /stagedir; cmd1; cmd2'") + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--mpi image:tag cmd') + elif container_variant == 'Shifter+load': + return ('shifter run ' + '--mount=type=bind,source="/path/one",destination="/one" ' + '--mount=type=bind,source="/path/two",destination="/two" ' + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + 'load/library/image:tag cmd') + elif container_variant in {'Singularity', 'Singularity+nopull'}: + return ('singularity exec -B"/path/one:/one" ' + '-B"/path/two:/two" -B"/foo:/rfm_workdir" image:tag cmd') + elif container_variant == 'Singularity+cuda': + return ('singularity exec -B"/path/one:/one" ' + '-B"/path/two:/two" -B"/foo:/rfm_workdir" --nv image:tag cmd') + elif container_variant == 'Singularity+nocommand': + return ('singularity run -B"/path/one:/one" ' + '-B"/path/two:/two" -B"/foo:/rfm_workdir" image:tag') @pytest.fixture def expected_cmd_prepare(container_variant): - if container_variant in ('Shifter', 'Shifter+mpi'): + if container_variant in {'Docker', 'Docker+nocommand'}: + return ['docker pull image:tag'] + elif container_variant in {'Shifter', 'Shifter+nocommand', 'Shifter+mpi'}: return ['shifter pull image:tag'] - elif container_variant in ('Sarus', 'Sarus+mpi'): + elif container_variant in {'Sarus', 'Sarus+nocommand', 'Sarus+mpi'}: return ['sarus pull image:tag'] else: return [] @@ -92,73 +128,166 @@ def expected_cmd_prepare(container_variant): @pytest.fixture def expected_cmd_run_opts(container_variant): - if container_variant == 'Docker': - return ('docker run --rm -v "/path/one":"/one" --foo --bar ' - "image:tag bash -c 'cd /stagedir; cmd'") - elif container_variant == 'Shifter': + if container_variant in {'Docker', 'Docker+nopull'}: + return ('docker run --rm -v "/path/one":"/one" ' + '-v "/foo":"/rfm_workdir" --foo --bar image:tag cmd') + if container_variant == 'Docker+nocommand': + return ('docker run --rm -v "/path/one":"/one" ' + '-v "/foo":"/rfm_workdir" --foo --bar image:tag') + elif container_variant in {'Shifter', 'Shifter+nopull'}: + return ('shifter run ' + '--mount=type=bind,source="/path/one",destination="/one" ' + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--foo --bar image:tag cmd') + elif container_variant == 'Shifter+nocommand': return ('shifter run ' '--mount=type=bind,source="/path/one",destination="/one" ' - "--foo --bar image:tag bash -c 'cd /stagedir; cmd'") + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--foo --bar image:tag') elif container_variant == 'Shifter+mpi': return ('shifter run ' '--mount=type=bind,source="/path/one",destination="/one" ' - "--mpi --foo --bar image:tag bash -c 'cd /stagedir; cmd'") - elif container_variant == 'Shifter+localimage': - return ( - 'shifter run ' - '--mount=type=bind,source="/path/one",destination="/one" ' - "--foo --bar load/library/image:tag bash -c 'cd /stagedir; cmd'" - ) + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--mpi --foo --bar image:tag cmd') + elif container_variant == 'Shifter+load': + return ('shifter run ' + '--mount=type=bind,source="/path/one",destination="/one" ' + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--foo --bar load/library/image:tag cmd') + elif container_variant in {'Sarus', 'Sarus+nopull'}: + return ('sarus run ' + '--mount=type=bind,source="/path/one",destination="/one" ' + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--foo --bar image:tag cmd') elif container_variant == 'Sarus': return ('sarus run ' '--mount=type=bind,source="/path/one",destination="/one" ' - "--foo --bar image:tag bash -c 'cd /stagedir; cmd'") + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--foo --bar image:tag cmd') + elif container_variant == 'Sarus+load': + return ('sarus run ' + '--mount=type=bind,source="/path/one",destination="/one" ' + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--foo --bar load/library/image:tag cmd') + elif container_variant == 'Sarus+nocommand': + return ('sarus run ' + '--mount=type=bind,source="/path/one",destination="/one" ' + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--foo --bar image:tag') elif container_variant == 'Sarus+mpi': return ('sarus run ' '--mount=type=bind,source="/path/one",destination="/one" ' - "--mpi --foo --bar image:tag bash -c 'cd /stagedir; cmd'") - elif container_variant == 'Sarus+localimage': - return ( - 'sarus run ' - '--mount=type=bind,source="/path/one",destination="/one" ' - "--foo --bar load/library/image:tag bash -c 'cd /stagedir; cmd'" - ) - elif container_variant == 'Singularity': - return ('singularity exec -B"/path/one:/one" ' - "--foo --bar image:tag bash -c 'cd /stagedir; cmd'") + '--mount=type=bind,source="/foo",destination="/rfm_workdir" ' + '--mpi --foo --bar image:tag cmd') + elif container_variant in {'Singularity'}: + return ('singularity exec -B"/path/one:/one" -B"/foo:/rfm_workdir" ' + '--foo --bar image:tag cmd') elif container_variant == 'Singularity+cuda': - return ('singularity exec -B"/path/one:/one" ' - "--nv --foo --bar image:tag bash -c 'cd /stagedir; cmd'") + return ('singularity exec -B"/path/one:/one" -B"/foo:/rfm_workdir" ' + '--nv --foo --bar image:tag cmd') + elif container_variant == 'Singularity+nocommand': + return ('singularity run -B"/path/one:/one" -B"/foo:/rfm_workdir" ' + '--foo --bar image:tag') def test_mount_points(container_platform, expected_cmd_mount_points): container_platform.mount_points = [('/path/one', '/one'), ('/path/two', '/two')] - container_platform.commands = ['cmd1', 'cmd2'] - container_platform.workdir = '/stagedir' - assert container_platform.launch_command() == expected_cmd_mount_points + cmd = container_platform.launch_command('/foo') + assert cmd == expected_cmd_mount_points def test_missing_image(container_platform): container_platform.image = None - container_platform.commands = ['cmd'] - with pytest.raises(ContainerError): - container_platform.validate() - - -def test_missing_commands(container_platform): - container_platform.image = 'image:tag' with pytest.raises(ContainerError): container_platform.validate() def test_prepare_command(container_platform, expected_cmd_prepare): - assert container_platform.emit_prepare_commands() == expected_cmd_prepare + commands = container_platform.emit_prepare_commands('/foo') + assert commands == expected_cmd_prepare def test_run_opts(container_platform, expected_cmd_run_opts): - container_platform.commands = ['cmd'] container_platform.mount_points = [('/path/one', '/one')] - container_platform.workdir = '/stagedir' container_platform.options = ['--foo', '--bar'] - assert container_platform.launch_command() == expected_cmd_run_opts + assert container_platform.launch_command('/foo') == expected_cmd_run_opts + + +# Everything from this point is testing deprecated behavior + +@pytest.fixture(params=['Docker', 'Singularity', 'Sarus', 'Shifter']) +def container_variant_noopt(request): + return request.param + + +@pytest.fixture +def container_platform_noopt(container_variant_noopt): + ret = containers.__dict__[container_variant_noopt]() + ret.image = 'image:tag' + ret.options = ['--foo'] + return ret + + +@pytest.fixture +def expected_run_with_commands(container_variant_noopt): + if container_variant_noopt == 'Docker': + return ("docker run --rm -v \"/foo\":\"/rfm_workdir\" " + "--foo image:tag bash -c 'cd /rfm_workdir; cmd1; cmd2'") + elif container_variant_noopt == 'Sarus': + return ( + "sarus run " + "--mount=type=bind,source=\"/foo\",destination=\"/rfm_workdir\" " + "--foo image:tag bash -c 'cd /rfm_workdir; cmd1; cmd2'" + ) + elif container_variant_noopt == 'Shifter': + return ( + "shifter run " + "--mount=type=bind,source=\"/foo\",destination=\"/rfm_workdir\" " + "--foo image:tag bash -c 'cd /rfm_workdir; cmd1; cmd2'" + ) + elif container_variant_noopt == 'Singularity': + return ("singularity exec -B\"/foo:/rfm_workdir\" " + "--foo image:tag bash -c 'cd /rfm_workdir; cmd1; cmd2'") + + +@pytest.fixture +def expected_run_with_workdir(container_variant_noopt): + if container_variant_noopt == 'Docker': + return ("docker run --rm -v \"/foo\":\"/rfm_workdir\" " + "--foo image:tag bash -c 'cd foodir; cmd1; cmd2'") + elif container_variant_noopt == 'Sarus': + return ( + "sarus run " + "--mount=type=bind,source=\"/foo\",destination=\"/rfm_workdir\" " + "--foo image:tag bash -c 'cd foodir; cmd1; cmd2'" + ) + elif container_variant_noopt == 'Shifter': + return ( + "shifter run " + "--mount=type=bind,source=\"/foo\",destination=\"/rfm_workdir\" " + "--foo image:tag bash -c 'cd foodir; cmd1; cmd2'" + ) + elif container_variant_noopt == 'Singularity': + return ("singularity exec -B\"/foo:/rfm_workdir\" --foo image:tag " + "bash -c 'cd foodir; cmd1; cmd2'") + + +def test_run_with_commands(container_platform_noopt, + expected_run_with_commands): + with pytest.warns(warn.ReframeDeprecationWarning): + container_platform_noopt.commands = ['cmd1', 'cmd2'] + + found_commands = container_platform_noopt.launch_command('/foo') + assert found_commands == expected_run_with_commands + + +def test_run_with_workdir(container_platform_noopt, expected_run_with_workdir): + with pytest.warns(warn.ReframeDeprecationWarning): + container_platform_noopt.commands = ['cmd1', 'cmd2'] + + with pytest.warns(warn.ReframeDeprecationWarning): + container_platform_noopt.workdir = 'foodir' + + found_commands = container_platform_noopt.launch_command('/foo') + assert found_commands == expected_run_with_workdir diff --git a/unittests/test_pipeline.py b/unittests/test_pipeline.py index 3d4af3cfcc..4334bb9096 100644 --- a/unittests/test_pipeline.py +++ b/unittests/test_pipeline.py @@ -99,8 +99,9 @@ def local_user_exec_ctx(user_system): try: environ = partition.environs[0] except IndexError: - pytest.skip('no environments configured for partition: %s' % - partition.fullname) + pytest.skip( + f'no environments configured for partition: {partition.fullname}' + ) yield partition, environ @@ -114,8 +115,9 @@ def remote_exec_ctx(user_system): try: environ = partition.environs[0] except IndexError: - pytest.skip('no environments configured for partition: %s' % - partition.fullname) + pytest.skip( + f'no environments configured for partition: {partition.fullname}' + ) yield partition, environ @@ -125,7 +127,7 @@ def container_remote_exec_ctx(remote_exec_ctx): def _container_exec_ctx(platform): partition = remote_exec_ctx[0] if platform not in partition.container_environs.keys(): - pytest.skip('%s is not configured on the system' % platform) + pytest.skip(f'{platform} is not configured on the system') yield from remote_exec_ctx @@ -137,7 +139,7 @@ def container_local_exec_ctx(local_user_exec_ctx): def _container_exec_ctx(platform): partition = local_user_exec_ctx[0] if platform not in partition.container_environs.keys(): - pytest.skip('%s is not configured on the system' % platform) + pytest.skip(f'{platform} is not configured on the system') yield from local_user_exec_ctx @@ -487,7 +489,7 @@ def set_resources(self): _run(test, partition, environ) expected_job_options = {'--gres=gpu:2', '#DW jobdw capacity=100GB', - '#DW stage_in source=%s' % test.stagedir, + f'#DW stage_in source={test.stagedir}', '--foo'} assert expected_job_options == set(test.job.options) @@ -775,7 +777,7 @@ def __init__(self, a): self.a = a def __repr__(self): - return 'C(%s)' % self.a + return f'C({self.a})' class MyTest(rfm.RegressionTest): def __init__(self, a, b): @@ -1100,7 +1102,7 @@ def extract_perf(patt, tag): sn.extractsingle(patt, perf_file, tag, float) ) with open('perf.log', 'a') as fp: - fp.write('%s=%s' % (tag, val)) + fp.write(f'{tag}={val}') return val @@ -1136,14 +1138,12 @@ def __init__(self): self.valid_systems = ['*'] self.container_platform = platform self.container_platform.image = image - self.container_platform.commands = [ - 'pwd', 'ls', 'cat /etc/os-release' - ] - self.container_platform.workdir = '/workdir' + self.container_platform.command = ( + "bash -c 'cd /rfm_workdir; pwd; ls; cat /etc/os-release'" + ) self.prerun_cmds = ['touch foo'] self.sanity_patterns = sn.all([ - sn.assert_found( - r'^' + self.container_platform.workdir, self.stdout), + sn.assert_found(r'^/rfm_workdir', self.stdout), sn.assert_found(r'^foo', self.stdout), sn.assert_found( r'18\.04\.\d+ LTS \(Bionic Beaver\)', self.stdout),