[feat] Auto-detect cpu topology

docs/config_reference.rst

@@ -341,9 +341,13 @@ System Partition Configuration
    :default: ``{}``
 
    Processor information for this partition stored in a `processor info object <#processor-info>`__.
+   If not set, ReFrame will try to auto-detect this information (see :ref:`proc-autodetection` for more information).
 
    .. versionadded:: 3.5.0
 
+   .. versionchanged:: 3.7.0
+      ReFrame is now able to detect the processor information automatically.
+
 
 .. js:attribute:: .systems[].partitions[].devices
 
@@ -1201,6 +1205,28 @@ General Configuration
    The command-line option sets the configuration option to ``false``.
 
 
+.. js:attribute:: .general[].remote_detect
+
+   :required: No
+   :default: ``false``
+
+   Try to auto-detect processor information of remote partitions as well.
+   This may slow down the initialization of the framework, since it involves submitting auto-detection jobs to the remote partitions.
+   For more information on how ReFrame auto-detects processor information, you may refer to :ref:`proc-autodetection`.
+
+   .. versionadded:: 3.7.0
+
+
+.. js:attribute:: .general[].remote_workdir
+
+   :required: No
+   :default: ``"."``
+
+   The temporary directory prefix that will be used to create a fresh ReFrame clone, in order to auto-detect the processor information of a remote partition.
+
+   .. versionadded:: 3.7.0
+
+
 .. js:attribute:: .general[].ignore_check_conflicts
 
    :required: No

docs/configure.rst

@@ -397,3 +397,43 @@ Let's see some concrete examples:
      "CC"
 
   If you explicitly query a configuration value which is not defined in the configuration file, ReFrame will print its default value.
+
+
+.. _proc-autodetection:
+
+Auto-detecting processor information
+------------------------------------
+
+.. versionadded:: 3.7.0
+
+.. |devices| replace:: :attr:`devices`
+.. _devices: config_reference.html#.systems[].partitions[].devices
+.. |processor| replace:: :attr:`processor`
+.. _processor: config_reference.html#.systems[].partitions[].processor
+.. |detect_remote_system_topology| replace:: :attr:`detect_remote_system_topology`
+.. _detect_remote_system_topology: config_reference.html#.general[].detect_remote_system_topology
+
+ReFrame is able to detect the processor topology of both local and remote partitions automatically.
+The processor and device information are made available to the tests through the corresponding attributes of the :attr:`~reframe.core.pipeline.RegressionTest.current_partition` allowing a test to modify its behavior accordingly.
+Currently, ReFrame supports auto-detection of the local or remote processor information only.
+It does not support auto-detection of devices, in which cases users should explicitly specify this information using the |devices|_ configuration option.
+The processor information auto-detection works as follows:
+
+#. If the |processor|_ configuration is option is defined, then no auto-detection is attempted.
+
+#. If the |processor|_ configuration option is not defined, ReFrame will look for a processor configuration metadata file in ``{configdir}/_meta/{system}-{part}/processor.json`` or in ``~/.reframe/topology/{system}-{part}/processor.json`` in case of the builtin configuration file.
+   If the file is found, the topology information is loaded from there.
+   These files are generated automatically by ReFrame from previous runs.
+
+#. If the corresponding metadata files are not found, the processor information will be auto-detected.
+   If the system partition is local (i.e., ``local`` scheduler + ``local`` launcher), the processor information is auto-detected unconditionally and stored in the corresponding metadata file for this partition.
+   If the partition is remote, ReFrame will not try to auto-detect it unless the :envvar:`RFM_REMOTE_DETECT` or the |detect_remote_system_topology|_ configuration option is set.
+   In that case, the steps to auto-detect the remote processor information are the following:
+
+     a. ReFrame creates a fresh clone of itself in a temporary directory created under ``.`` by default.
+        This temporary directory prefix can be changed by setting the :envvar:`RFM_REMOTE_WORKDIR` environment variable.
+     b. ReFrame changes to that directory and launches a job that will first bootstrap the fresh clone and then run that clone with ``{launcher} ./bin/reframe --detect-host-topology=topo.json``.
+        The :option:`--detect-host-topology` option causes ReFrame to detect the topology of the current host,
+        which in this case would be the remote compute nodes.
+
+   In case of errors during auto-detection, ReFrame will simply issue a warning and continue.

docs/manpage.rst

@@ -576,6 +576,16 @@ Miscellaneous options
 
    This option can also be set using the :envvar:`RFM_SYSTEM` environment variable.
 
+.. _--detect-host-topology:
+
+.. option:: --detect-host-topology[=FILE]
+
+   Detect the local host processor topology, store it to ``FILE`` and exit.
+   If no ``FILE`` is specified, the standard output will be used.
+
+   .. versionadded:: 3.7.0
+
+
 .. option:: --failure-stats
 
    Print failure statistics at the end of the run.
@@ -698,6 +708,36 @@ Here is an alphabetical list of the environment variables recognized by ReFrame:
       ================================== ==================
 
 
+.. envvar:: RFM_REMOTE_DETECT
+
+   Auto-detect processor information of remote partitions as well.
+
+   .. table::
+      :align: left
+
+      ================================== ==================
+      Associated command line option     N/A
+      Associated configuration parameter :js:attr:`remote_detect` general configuration parameter
+      ================================== ==================
+
+.. versionadded:: 3.7.0
+
+
+.. envvar:: RFM_REMOTE_WORKDIR
+
+   The temporary directory prefix that will be used to create a fresh ReFrame clone, in order to auto-detect the processor information of a remote partition.
+
+   .. table::
+      :align: left
+
+      ================================== ==================
+      Associated command line option     N/A
+      Associated configuration parameter :js:attr:`remote_workdir` general configuration parameter
+      ================================== ==================
+
+.. versionadded:: 3.7.0
+
+
 .. envvar:: RFM_GRAYLOG_ADDRESS
 
    The address of the Graylog server to send performance logs.
@@ -920,7 +960,7 @@ Here is an alphabetical list of the environment variables recognized by ReFrame:
       :align: left
 
       ================================== ==================
-      Associated command line option     n/a
+      Associated command line option     N/A
       Associated configuration parameter :js:attr:`resolve_module_conflicts` general configuration parameter
       ================================== ==================
 

docs/requirements.txt

@@ -1,3 +1,4 @@
+archspec==0.1.2
 docutils==0.16 # https://github.com/sphinx-doc/sphinx/issues/9001
 jsonschema==3.2.0
 semver==2.13.0

reframe/core/config.py

@@ -101,6 +101,11 @@ def __getitem__(self, key):
     def __getattr__(self, attr):
         return getattr(self._pick_config(), attr)
 
+    @property
+    def schema(self):
+        '''Configuration schema'''
+        return self._schema
+
     def add_sticky_option(self, option, value):
         self._sticky_options[option] = value
 

reframe/core/systems.py

@@ -5,12 +5,12 @@
 
 import json
 
-import reframe.utility as utility
+import reframe.utility as util
 import reframe.utility.jsonext as jsonext
 from reframe.core.backends import (getlauncher, getscheduler)
+from reframe.core.environments import (Environment, ProgEnvironment)
 from reframe.core.logging import getlogger
 from reframe.core.modules import ModulesSystem
-from reframe.core.environments import (Environment, ProgEnvironment)
 
 
 class ProcessorType(jsonext.JSONSerializable):
@@ -232,7 +232,7 @@ def access(self):
 
         :type: :class:`List[str]`
         '''
-        return utility.SequenceView(self._access)
+        return util.SequenceView(self._access)
 
     @property
     def descr(self):
@@ -249,7 +249,7 @@ def environs(self):
         :type: :class:`List[ProgEnvironment]`
         '''
 
-        return utility.SequenceView(self._environs)
+        return util.SequenceView(self._environs)
 
     @property
     def container_environs(self):
@@ -258,7 +258,7 @@ def container_environs(self):
         :type: :class:`Dict[str, Environment]`
         '''
 
-        return utility.MappingView(self._container_environs)
+        return util.MappingView(self._container_environs)
 
     @property
     def fullname(self):
@@ -315,7 +315,7 @@ def resources(self):
 
         '''
 
-        return utility.MappingView(self._resources)
+        return util.MappingView(self._resources)
 
     @property
     def scheduler(self):
@@ -661,7 +661,7 @@ def partitions(self):
 
         :type: :class:`List[SystemPartition]`
         '''
-        return utility.SequenceView(self._partitions)
+        return util.SequenceView(self._partitions)
 
     def __eq__(self, other):
         if not isinstance(other, type(self)):