Make DALI tests compatible with Python 3.12

Acknowledgements.txt

@@ -4285,4 +4285,128 @@ JAX
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
-   limitations under the License.
+   limitations under the License.
+
+==============================================================================
+Python
+
+A. HISTORY OF THE SOFTWARE
+==========================
+
+Python was created in the early 1990s by Guido van Rossum at Stichting
+Mathematisch Centrum (CWI, see https://www.cwi.nl) in the Netherlands
+as a successor of a language called ABC.  Guido remains Python's
+principal author, although it includes many contributions from others.
+
+In 1995, Guido continued his work on Python at the Corporation for
+National Research Initiatives (CNRI, see https://www.cnri.reston.va.us)
+in Reston, Virginia where he released several versions of the
+software.
+
+In May 2000, Guido and the Python core development team moved to
+BeOpen.com to form the BeOpen PythonLabs team.  In October of the same
+year, the PythonLabs team moved to Digital Creations, which became
+Zope Corporation.  In 2001, the Python Software Foundation (PSF, see
+https://www.python.org/psf/) was formed, a non-profit organization
+created specifically to own Python-related Intellectual Property.
+Zope Corporation was a sponsoring member of the PSF.
+
+All Python releases are Open Source (see https://opensource.org for
+the Open Source Definition).  Historically, most, but not all, Python
+releases have also been GPL-compatible; the table below summarizes
+the various releases.
+
+    Release         Derived     Year        Owner       GPL-
+                    from                                compatible? (1)
+
+    0.9.0 thru 1.2              1991-1995   CWI         yes
+    1.3 thru 1.5.2  1.2         1995-1999   CNRI        yes
+    1.6             1.5.2       2000        CNRI        no
+    2.0             1.6         2000        BeOpen.com  no
+    1.6.1           1.6         2001        CNRI        yes (2)
+    2.1             2.0+1.6.1   2001        PSF         no
+    2.0.1           2.0+1.6.1   2001        PSF         yes
+    2.1.1           2.1+2.0.1   2001        PSF         yes
+    2.1.2           2.1.1       2002        PSF         yes
+    2.1.3           2.1.2       2002        PSF         yes
+    2.2 and above   2.1.1       2001-now    PSF         yes
+
+Footnotes:
+
+(1) GPL-compatible doesn't mean that we're distributing Python under
+    the GPL.  All Python licenses, unlike the GPL, let you distribute
+    a modified version without making your changes open source.  The
+    GPL-compatible licenses make it possible to combine Python with
+    other software that is released under the GPL; the others don't.
+
+(2) According to Richard Stallman, 1.6.1 is not GPL-compatible,
+    because its license has a choice of law clause.  According to
+    CNRI, however, Stallman's lawyer has told CNRI's lawyer that 1.6.1
+    is "not incompatible" with the GPL.
+
+Thanks to the many outside volunteers who have worked under Guido's
+direction to make these releases possible.
+
+
+B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
+===============================================================
+
+Python software and documentation are licensed under the
+Python Software Foundation License Version 2.
+
+Starting with Python 3.8.6, examples, recipes, and other code in
+the documentation are dual licensed under the PSF License Version 2
+and the Zero-Clause BSD license.
+
+Some software incorporated into Python is under different licenses.
+The licenses are listed with code falling under that license.
+
+
+PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
+--------------------------------------------
+
+1. This LICENSE AGREEMENT is between the Python Software Foundation
+("PSF"), and the Individual or Organization ("Licensee") accessing and
+otherwise using this software ("Python") in source or binary form and
+its associated documentation.
+
+2. Subject to the terms and conditions of this License Agreement, PSF hereby
+grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
+analyze, test, perform and/or display publicly, prepare derivative works,
+distribute, and otherwise use Python alone or in any derivative version,
+provided, however, that PSF's License Agreement and PSF's notice of copyright,
+i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
+2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023 Python Software Foundation;
+All Rights Reserved" are retained in Python alone or in any derivative version
+prepared by Licensee.
+
+3. In the event Licensee prepares a derivative work that is based on
+or incorporates Python or any part thereof, and wants to make
+the derivative work available to others as provided herein, then
+Licensee hereby agrees to include in any such work a brief summary of
+the changes made to Python.
+
+4. PSF is making Python available to Licensee on an "AS IS"
+basis.  PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
+IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
+DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
+FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
+INFRINGE ANY THIRD PARTY RIGHTS.
+
+5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
+FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
+A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
+OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
+
+6. This License Agreement will automatically terminate upon a material
+breach of its terms and conditions.
+
+7. Nothing in this License Agreement shall be deemed to create any
+relationship of agency, partnership, or joint venture between PSF and
+Licensee.  This License Agreement does not grant permission to use PSF
+trademarks or trade name in a trademark sense to endorse or promote
+products or services of Licensee, or any third party.
+
+8. By copying, installing or otherwise using Python, Licensee
+agrees to be bound by the terms and conditions of this License
+Agreement.

qa/nose_wrapper/__main__.py

@@ -1,4 +1,86 @@
 import sys
+
+if sys.version_info >= (3, 12):
+    from importlib import machinery, util
+    from importlib._bootstrap import _exec, _load
+    import modulefinder
+    import types
+    import unittest
+
+    # the below are based on https://github.com/python/cpython/blob/3.11/Lib/imp.py
+    # based on PSF license
+    def find_module(name, path):
+        return modulefinder.ModuleFinder(path).find_module(name, path)
+
+    def load_module(name, file, filename, details):
+        PY_SOURCE = 1
+        class _HackedGetData:
+
+            """Compatibility support for 'file' arguments of various load_*()
+            functions."""
+
+            def __init__(self, fullname, path, file=None):
+                super().__init__(fullname, path)
+                self.file = file
+
+            def get_data(self, path):
+                """Gross hack to contort loader to deal w/ load_*()'s bad API."""
+                if self.file and path == self.path:
+                    # The contract of get_data() requires us to return bytes. Reopen the
+                    # file in binary mode if needed.
+                    file = None
+                    if not self.file.closed:
+                        file = self.file
+                        if 'b' not in file.mode:
+                            file.close()
+                    if self.file.closed:
+                        self.file = file = open(self.path, 'rb')
+
+                    with file:
+                        return file.read()
+                else:
+                    return super().get_data(path)
+
+        class _LoadSourceCompatibility(_HackedGetData, machinery.SourceFileLoader):
+
+            """Compatibility support for implementing load_source()."""
+
+        _, mode, type_ = details
+        if mode and (not mode.startswith('r') or '+' in mode):
+            raise ValueError('invalid file open mode {!r}'.format(mode))
+        elif file is None and type_ in {PY_SOURCE, PY_COMPILED}:
+            msg = 'file object required for import (type code {})'.format(type_)
+            raise ValueError(msg)
+        assert type_ == PY_SOURCE, "load_module replacement supports only PY_SOURCE file type"
+        loader = _LoadSourceCompatibility(name, filename, file)
+        spec = util.spec_from_file_location(name, filename, loader=loader)
+        if name in sys.modules:
+            module = _exec(spec, sys.modules[name])
+        else:
+            module = _load(spec)
+        # To allow reloading to potentially work, use a non-hacked loader which
+        # won't rely on a now-closed file object.
+        module.__loader__ = machinery.SourceFileLoader(name, filename)
+        module.__spec__.loader = module.__loader__
+        return module
+
+    def acquire_lock():
+        pass
+
+    def release_lock():
+        pass
+
+    context = {
+        "find_module": find_module,
+        "load_module": load_module,
+        "acquire_lock": acquire_lock,
+        "release_lock": release_lock,
+    }
+    imp_module = types.ModuleType("imp", "Mimics old imp module")
+    imp_module.__dict__.update(context)
+    sys.modules["imp"] = imp_module
+    unittest._TextTestResult = unittest.TextTestResult
+
 from nose.core import run_exit
 import collections
 import nose.case

qa/setup_packages.py

@@ -476,11 +476,10 @@ def get_pyvers_name(self, url, cuda_version):
 
 
 all_packages = [
-    PlainPackage("numpy", [">=1.17,<1.24"]),
-    PlainPackage("opencv-python", [PckgVer("4.8.1.78", dependencies=["numpy<1.24"])]),
+    PlainPackage("opencv-python", [PckgVer("4.8.1.78")]),
     CudaPackage(
         "cupy",
-        {"118": [PckgVer("12.2.0", python_min_ver="3.8", dependencies=["numpy<1.24"])]},
+        {"118": [PckgVer("12.2.0", python_min_ver="3.8")]},
         "cupy-cuda11x",
     ),
     CudaPackage(
@@ -494,7 +493,6 @@ def get_pyvers_name(self, url, cuda_version):
                     alias="tensorflow",
                     dependencies=[
                         "protobuf<4",
-                        "numpy<1.24",
                         "urllib3<2.0",
                     ],
                 ),
@@ -504,7 +502,6 @@ def get_pyvers_name(self, url, cuda_version):
                     alias="tensorflow",
                     dependencies=[
                         "protobuf<4",
-                        "numpy<1.24",
                         "urllib3<2.0",
                     ],
                 ),
@@ -516,7 +513,6 @@ def get_pyvers_name(self, url, cuda_version):
                     alias="tensorflow",
                     dependencies=[
                         "protobuf<4",
-                        "numpy<1.24",
                         "urllib3<2.0",
                     ],
                 ),
@@ -526,7 +522,6 @@ def get_pyvers_name(self, url, cuda_version):
                     alias="tensorflow",
                     dependencies=[
                         "protobuf<4",
-                        "numpy<1.24",
                         "urllib3<2.0",
                     ],
                 ),
@@ -536,7 +531,6 @@ def get_pyvers_name(self, url, cuda_version):
                     alias="tensorflow",
                     dependencies=[
                         "protobuf<4",
-                        "numpy<1.24",
                         "urllib3<2.0",
                     ],
                 ),
@@ -545,17 +539,17 @@ def get_pyvers_name(self, url, cuda_version):
     ),
     CudaPackageExtraIndex(
         "torch",
-        {"118": [PckgVer("2.1.0", python_min_ver="3.8", dependencies=["numpy<1.24"])]},
+        {"118": [PckgVer("2.1.0", python_min_ver="3.8")]},
         extra_index="https://download.pytorch.org/whl/cu{cuda_v}/",
     ),
     CudaPackageExtraIndex(
         "torchvision",
-        {"118": [PckgVer("0.16.0", python_min_ver="3.8", dependencies=["numpy<1.24"])]},
+        {"118": [PckgVer("0.16.0", python_min_ver="3.8")]},
         extra_index="https://download.pytorch.org/whl/cu{cuda_v}/",
     ),
     CudaPackageExtraIndex(
         "paddlepaddle-gpu",
-        {"110": [PckgVer("2.5.2.post117", dependencies=["protobuf<4", "numpy<1.24"])]},
+        {"110": [PckgVer("2.5.2.post117", dependencies=["protobuf<4"])]},
         links_index="https://www.paddlepaddle.org.cn/" "whl/linux/mkl/avx/stable.html",
     ),
     CudaPackageExtraIndex(
@@ -575,7 +569,7 @@ def get_pyvers_name(self, url, cuda_version):
         links_index=("https://storage.googleapis.com/" "jax-releases/jax_cuda_releases.html"),
     ),
     CudaPackage(
-        "numba", {"110": [PckgVer("0.57.0", python_min_ver="3.8", dependencies=["numpy<1.24"])]}
+        "numba", {"110": [PckgVer("0.57.0", python_min_ver="3.8")]}
     ),
 ]