qutip 4.7.3 QobjEvo test failures on Python 3.12

[drew-parsons]

Bug Description

Building qutip 4.7.3 for Debian unstable, all tests are passing with python 3.11. But a dozen tests using qobjevo fail with python 3.12:

=========================== short test summary info ============================
FAILED qutip/tests/test_brmesolve_td.py::test_time_dependent_spline_in_c_ops
FAILED qutip/tests/test_brmesolve_td.py::test_split_operators_maintain_answer[_2_tuple_split]
FAILED qutip/tests/test_correlation.py::TestTimeDependence::test_varying_coefficient_hamiltonian_2ls[string]
FAILED qutip/tests/test_correlation.py::TestTimeDependence::test_varying_coefficient_hamiltonian_2ls[numpy]
FAILED qutip/tests/test_mesolve.py::TestMESolverConstDecay::testMEDecayAsStrList
FAILED qutip/tests/test_mesolve.py::TestMESolveTDDecay::testMETDDecayAsStrList
FAILED qutip/tests/test_mesolve.py::TestMESolveSuperInit::testMETDDecayAsStrList
FAILED qutip/tests/test_mesolve.py::TestMESolveStepFuncCoeff::test_array_str_coeff
FAILED qutip/tests/test_mesolve.py::TestMESolveStepFuncCoeff::test_array_str_py_coeff
FAILED qutip/tests/test_openmp.py::test_openmp_mesolve_td - Exception: Could ...
FAILED qutip/tests/test_propagator.py::testPropHOStrTd - Exception: Could not...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_call - Exception: Could not ...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_call_args - Exception: Could...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_mul_vec - Exception: Could n...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_mul_mat - Exception: Could n...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_expect_psi - Exception: Coul...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_expect_rho - Exception: Coul...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_with_state - Exception: Coul...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_pickle - Exception: Could no...
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_safepickle - Exception: Coul...
FAILED qutip/tests/test_sesolve.py::TestSESolve::test_04_1_state_with_list_str_H
FAILED qutip/tests/test_sesolve.py::TestSESolve::test_04_2_unitary_with_list_func_H
FAILED qutip/tests/test_sesolve.py::TestSESolve::test_06_4_compare_state_and_unitary_list_str
= 23 failed, 3077 passed, 101 skipped, 68 deselected, 1387 warnings in 1058.36s (0:17:38) =

#1084 notes that a major overhaul of qobjevo has been done for qutip v5. Perhaps the issue is fixed. On the hand it could be an issue with python 3.12 that still needs to be addressed in v5, so I though I should mention in.

Code to Reproduce the Bug

python3.12 -c "import qutip.testing; qutip.testing.run(pytest_options=[['-k', 'mesolve_td or test_varying_coefficient_hamiltonian_2ls or testMEDecayAsStrList or test_array_str or test_qobjevo or test_04 or test_06_4_compare_state_and_unitary_list_str or testPropHOStrTd']])"

Code Output

$ python3.12 -c "import qutip.testing; qutip.testing.run(pytest_options=[['-k', 'mesolve_td or test_varying_coefficient_hamiltonian_2ls or testMEDecayAsStrList or test_array_str or test_qobjevo or test_04 or test_06_4_compare_state_and_unitary_list_str or testPropHOStrTd']])"

QuTiP: Quantum Toolbox in Python
================================
Copyright (c) QuTiP team 2011 and later.
Current admin team: Alexander Pitchford, Nathan Shammah, Shahnawaz Ahmed, Neill Lambert, Eric Giguère, Boxi Li, Jake Lishman, Simon Cross and Asier Galicia.
Board members: Daniel Burgarth, Robert Johansson, Anton F. Kockum, Franco Nori and Will Zeng.
Original developers: R. J. Johansson & P. D. Nation.
Previous lead developers: Chris Granade & A. Grimsmo.
Currently developed through wide collaboration. See https://github.com/qutip for details.

QuTiP Version:      4.7.3
Numpy Version:      1.24.2
Scipy Version:      1.10.1
Cython Version:     0.29.36
Matplotlib Version: 3.6.3
Python Version:     3.12.1
Number of CPUs:     8
BLAS Info:          OPENBLAS
OPENMP Installed:   True
INTEL MKL Ext:      False
Platform Info:      Linux (x86_64)
Installation path:  /usr/lib/python3/dist-packages/qutip
================================================================================
Please cite QuTiP in your publication.
================================================================================
For your convenience a bibtex reference can be easily generated using `qutip.cite()`
================================================================================================================================== test session starts ===================================================================================================================================
platform linux -- Python 3.12.1, pytest-7.4.4, pluggy-1.3.0 -- /usr/bin/python3.12
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/home/drew/projects/misc/build/.hypothesis/examples'))
rootdir: /home/drew/projects/misc/build/qutip
plugins: remotedata-0.4.1, asyncio-0.20.3, filter-subpackage-0.1.2, doctestplus-1.1.0, flaky-3.7.0, arraydiff-0.6.1, xvfb-3.0.0, astropy-0.11.0, mpi-0.6, astropy-header-0.2.2, cov-4.1.0, mock-3.12.0, openfiles-0.5.0, hypothesis-6.92.2
asyncio: mode=Mode.STRICT
collected 3269 items / 3233 deselected / 36 selected                                                                                                                                                                                                                                     

qutip/tests/test_brmesolve_td.py::test_time_dependent_spline_in_c_ops FAILED                                                                                                                                                                                                       [  2%]
qutip/tests/test_brmesolve_td.py::test_split_operators_maintain_answer[_2_tuple_split] FAILED                                                                                                                                                                                      [  5%]
qutip/tests/test_correlation.py::TestTimeDependence::test_varying_coefficient_hamiltonian_2ls[string] FAILED                                                                                                                                                                       [  8%]
qutip/tests/test_correlation.py::TestTimeDependence::test_varying_coefficient_hamiltonian_2ls[numpy] FAILED                                                                                                                                                                        [ 11%]
qutip/tests/test_correlation.py::TestTimeDependence::test_varying_coefficient_hamiltonian_2ls[function] PASSED                                                                                                                                                                     [ 13%]
qutip/tests/test_krylovsolve.py::TestKrylovSolve::test_04_simple_check_states_e_ops_H_sho PASSED                                                                                                                                                                                   [ 16%]
qutip/tests/test_mesolve.py::TestMESolverConstDecay::testMEDecayAsStrList FAILED                                                                                                                                                                                                   [ 19%]
qutip/tests/test_mesolve.py::TestMESolveStepFuncCoeff::test_array_str_coeff FAILED                                                                                                                                                                                                 [ 22%]
qutip/tests/test_mesolve.py::TestMESolveStepFuncCoeff::test_array_str_py_coeff FAILED                                                                                                                                                                                              [ 25%]
qutip/tests/test_openmp.py::test_openmp_mesolve_td FAILED                                                                                                                                                                                                                          [ 27%]
qutip/tests/test_propagator.py::testPropHOStrTd FAILED                                                                                                                                                                                                                             [ 30%]
qutip/tests/test_qobjevo.py::test_QobjEvo_call FAILED                                                                                                                                                                                                                              [ 33%]
qutip/tests/test_qobjevo.py::test_QobjEvo_call_args FAILED                                                                                                                                                                                                                         [ 36%]
qutip/tests/test_qobjevo.py::test_QobjEvo_step_coeff PASSED                                                                                                                                                                                                                        [ 38%]
qutip/tests/test_qobjevo.py::test_QobjEvo_copy PASSED                                                                                                                                                                                                                              [ 41%]
qutip/tests/test_qobjevo.py::test_QobjEvo_to_list PASSED                                                                                                                                                                                                                           [ 44%]
qutip/tests/test_qobjevo.py::test_QobjEvo_math_arithmetic PASSED                                                                                                                                                                                                                   [ 47%]
qutip/tests/test_qobjevo.py::test_QobjEvo_unitary PASSED                                                                                                                                                                                                                           [ 50%]
qutip/tests/test_qobjevo.py::test_QobjEvo_tidyup PASSED                                                                                                                                                                                                                            [ 52%]
qutip/tests/test_qobjevo.py::test_QobjEvo_compress PASSED                                                                                                                                                                                                                          [ 55%]
qutip/tests/test_qobjevo.py::test_QobjEvo_shift PASSED                                                                                                                                                                                                                             [ 58%]
qutip/tests/test_qobjevo.py::test_QobjEvo_apply PASSED                                                                                                                                                                                                                             [ 61%]
qutip/tests/test_qobjevo.py::test_QobjEvo_apply_decorator PASSED                                                                                                                                                                                                                   [ 63%]
qutip/tests/test_qobjevo.py::test_QobjEvo_mul_vec FAILED                                                                                                                                                                                                                           [ 66%]
qutip/tests/test_qobjevo.py::test_QobjEvo_mul_mat FAILED                                                                                                                                                                                                                           [ 69%]
qutip/tests/test_qobjevo.py::test_QobjEvo_expect_psi FAILED                                                                                                                                                                                                                        [ 72%]
qutip/tests/test_qobjevo.py::test_QobjEvo_expect_rho FAILED                                                                                                                                                                                                                        [ 75%]
qutip/tests/test_qobjevo.py::test_QobjEvo_with_state FAILED                                                                                                                                                                                                                        [ 77%]
qutip/tests/test_qobjevo.py::test_QobjEvo_pickle FAILED                                                                                                                                                                                                                            [ 80%]
qutip/tests/test_qobjevo.py::test_QobjEvo_safepickle FAILED                                                                                                                                                                                                                        [ 83%]
qutip/tests/test_qobjevo.py::test_QobjEvo_superoperator PASSED                                                                                                                                                                                                                     [ 86%]
qutip/tests/test_qobjevo.py::test_QobjEvo_tidyup_default[1] PASSED                                                                                                                                                                                                                 [ 88%]
qutip/tests/test_qobjevo.py::test_QobjEvo_tidyup_default[1e-15] PASSED                                                                                                                                                                                                             [ 91%]
qutip/tests/test_sesolve.py::TestSESolve::test_04_1_state_with_list_str_H FAILED                                                                                                                                                                                                   [ 94%]
qutip/tests/test_sesolve.py::TestSESolve::test_04_2_unitary_with_list_func_H FAILED                                                                                                                                                                                                [ 97%]
qutip/tests/test_sesolve.py::TestSESolve::test_06_4_compare_state_and_unitary_list_str FAILED                                                                                                                                                                                      [100%]

======================================================================================================================================== FAILURES ========================================================================================================================================
__________________________________________________________________________________________________________________________ test_time_dependent_spline_in_c_ops ___________________________________________________________________________________________________________________________

    def test_time_dependent_spline_in_c_ops():
        N = 10
        a = qutip.destroy(N)
        H = a.dag()*a
        psi0 = qutip.basis(N, 9)
        times = np.linspace(0, 10, 100)
        kappa = 0.2
        exact = 9 * np.exp(-2 * kappa * (1 - np.exp(-times)))
        a_ops = [[a + a.dag(), _string_w_interpolating_t(kappa, times)]]
        collapse_points = np.sqrt(kappa) * np.exp(-0.5*times)
        c_ops = [[a, qutip.Cubic_Spline(times[0], times[-1], collapse_points)]]
>       brme = qutip.brmesolve(H, psi0, times,
                               a_ops, e_ops=[a.dag()*a], c_ops=c_ops)

/usr/lib/python3/dist-packages/qutip/tests/test_brmesolve_td.py:163: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3/dist-packages/qutip/bloch_redfield.py:200: in brmesolve
    output = _td_brmesolve(H, psi0, tlist, a_ops=a_ops, e_ops=e_ops,
/usr/lib/python3/dist-packages/qutip/bloch_redfield.py:482: in _td_brmesolve
    exec(code, globals())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   ModuleNotFoundError: No module named 'rhs4968610'

<string>:1: ModuleNotFoundError
__________________________________________________________________________________________________________________ test_split_operators_maintain_answer[_2_tuple_split] __________________________________________________________________________________________________________________

collapse_operators = <function _2_tuple_split at 0x7fe9f0ca6520>

    @pytest.mark.parametrize("collapse_operators", [
            _2_tuple_split,
            pytest.param(_4_tuple_split, marks=pytest.mark.slow),
            pytest.param(_2_tuple_splines, marks=pytest.mark.slow),
            pytest.param(_2_list_entries_2_tuple_split, marks=pytest.mark.slow),
        ])
    def test_split_operators_maintain_answer(collapse_operators):
        N = 10
        w0 = 1.0 * 2*np.pi
        g = 0.05 * w0
        kappa = 0.15
    
        a = qutip.destroy(N)
        H = w0*a.dag()*a + g*(a+a.dag())
        psi0 = (qutip.basis(N, 4) + qutip.basis(N, 2) + qutip.basis(N, 0)).unit()
        psi0 = qutip.ket2dm(psi0)
        times = np.linspace(0, 25, 1000)
        e_ops = [a.dag()*a, a+a.dag()]
    
        me_c_ops, brme_c_ops, a_ops = collapse_operators(N, kappa, times)
        me = qutip.mesolve(H, psi0, times, me_c_ops, e_ops)
>       brme = qutip.brmesolve(H, psi0, times, a_ops, e_ops, brme_c_ops)

/usr/lib/python3/dist-packages/qutip/tests/test_brmesolve_td.py:256: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3/dist-packages/qutip/bloch_redfield.py:200: in brmesolve
    output = _td_brmesolve(H, psi0, tlist, a_ops=a_ops, e_ops=e_ops,
/usr/lib/python3/dist-packages/qutip/bloch_redfield.py:482: in _td_brmesolve
    exec(code, globals())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   ModuleNotFoundError: No module named 'rhs4968611'

<string>:1: ModuleNotFoundError
__________________________________________________________________________________________________________ TestTimeDependence.test_varying_coefficient_hamiltonian_2ls[string] ___________________________________________________________________________________________________________

code = '#!python\n#cython: language_level=3\n# This file is generated automatically by QuTiP.\n\nimport numpy as np\ncimport ...f double t_off = self.t_off\n        cdef double tp = self.tp\n\n        coeff[0] = exp(-(t-t_off)**2 / (2 * tp*tp))\n'
basefilename = 'cqobjevo_compiled_coeff_', obj_name = 'CompiledStrCoeff', cythonfile = True

    def _import_str(code, basefilename, obj_name, cythonfile=False):
        """
        Import 'obj_name' defined in 'code'.
        Using a temporary file starting by 'basefilename'.
        """
        filename = (basefilename + str(hash(code))[1:4] +
                    str(os.getpid()) + time.strftime("%M%S"))
        tries = 0
        import_list = []
        ext = ".pyx" if cythonfile else ".py"
        e = ImportError() # used if import_list remains empty
        if os.getcwd() not in sys.path:
            sys.path.insert(0, os.getcwd())
        while not import_list and tries < 3:
            try_file = filename + str(tries)
            file_ = open(try_file+ext, "w")
            file_.writelines(code)
            file_.close()
            if not os.access(try_file, os.R_OK):
                time.sleep(0.1)
            codeString = str("from " + try_file +
                             " import " + obj_name + '\n' +
                             "import_list.append(" + obj_name + ")")
            try:
                import_code = compile(codeString, '<string>', 'exec')
>               exec(import_code, locals())
E               ModuleNotFoundError: No module named 'cqobjevo_compiled_coeff_46049686147042'

/usr/lib/python3/dist-packages/qutip/qobjevo_codegen.py:45: ModuleNotFoundError

The above exception was the direct cause of the following exception:

self = <qutip.tests.test_correlation.TestTimeDependence object at 0x7fe9f0e896d0>, dependence_2ls = 'exp(-(t-t_off)**2 / (2 * tp*tp))'

    def test_varying_coefficient_hamiltonian_2ls(self, dependence_2ls):
        H = [[_2ls_args['H0'], dependence_2ls]]
>       assert abs(_2ls_g2_0(H, []) - 0.575) < 1e-2

/usr/lib/python3/dist-packages/qutip/tests/test_correlation.py:167: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3/dist-packages/qutip/tests/test_correlation.py:135: in _2ls_g2_0
    correlation = qutip.correlation_3op_2t(H, start, times, times, [sp],
/usr/lib/python3/dist-packages/qutip/correlation.py:324: in correlation_3op_2t
    return _correlation_2t(H, state0, tlist, taulist,
/usr/lib/python3/dist-packages/qutip/correlation.py:1030: in _correlation_2t
    return _correlation_me_2t(H, state0, tlist, taulist,
/usr/lib/python3/dist-packages/qutip/correlation.py:1069: in _correlation_me_2t
    rho_t = mesolve(H, rho0, tlist, c_ops, [],
/usr/lib/python3/dist-packages/qutip/mesolve.py:231: in mesolve
    ss = _mesolve_QobjEvo(H, c_ops, tlist, args, options)
/usr/lib/python3/dist-packages/qutip/mesolve.py:287: in _mesolve_QobjEvo
    L_td.compile(omp=nthread)
/usr/lib/python3/dist-packages/qutip/qobjevo.py:1641: in compile
    self.coeff_get, Code, file_ = _compiled_coeffs(
/usr/lib/python3/dist-packages/qutip/qobjevo_codegen.py:94: in _compiled_coeffs
    coeff_obj, filename = _import_str(code, "cqobjevo_compiled_coeff_",
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

code = '#!python\n#cython: language_level=3\n# This file is generated automatically by QuTiP.\n\nimport numpy as np\ncimport ...f double t_off = self.t_off\n        cdef double tp = self.tp\n\n        coeff[0] = exp(-(t-t_off)**2 / (2 * tp*tp))\n'
basefilename = 'cqobjevo_compiled_coeff_', obj_name = 'CompiledStrCoeff', cythonfile = True

    def _import_str(code, basefilename, obj_name, cythonfile=False):
        """
        Import 'obj_name' defined in 'code'.
        Using a temporary file starting by 'basefilename'.
        """
        filename = (basefilename + str(hash(code))[1:4] +
                    str(os.getpid()) + time.strftime("%M%S"))
        tries = 0
        import_list = []
        ext = ".pyx" if cythonfile else ".py"
        e = ImportError() # used if import_list remains empty
        if os.getcwd() not in sys.path:
            sys.path.insert(0, os.getcwd())
        while not import_list and tries < 3:
            try_file = filename + str(tries)
            file_ = open(try_file+ext, "w")
            file_.writelines(code)
            file_.close()
            if not os.access(try_file, os.R_OK):
                time.sleep(0.1)
            codeString = str("from " + try_file +
                             " import " + obj_name + '\n' +
                             "import_list.append(" + obj_name + ")")
            try:
                import_code = compile(codeString, '<string>', 'exec')
                exec(import_code, locals())
            except (ModuleNotFoundError, ImportError) as e:
                time.sleep(0.05)
                tries += 1
                _try_remove(try_file+ext)
                err = e
        if not import_list:
>           raise Exception("Could not convert string to importable function, "
                            "tmpfile:" + try_file + ext) from err
E           Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_46049686147042.pyx

/usr/lib/python3/dist-packages/qutip/qobjevo_codegen.py:52: Exception
...etc etc...
________________________________________________________________________________________________________________ TestSESolve.test_06_4_compare_state_and_unitary_list_str ________________________________________________________________________________________________________________

code = '#!python\n#cython: language_level=3\n# This file is generated automatically by QuTiP.\n\nimport numpy as np\ncimport ...id _call_core(self, double t, complex * coeff):\n        cdef double w_a = self.w_a\n\n        coeff[0] = cos(w_a*t)\n'
basefilename = 'cqobjevo_compiled_coeff_', obj_name = 'CompiledStrCoeff', cythonfile = True

    def _import_str(code, basefilename, obj_name, cythonfile=False):
        """
        Import 'obj_name' defined in 'code'.
        Using a temporary file starting by 'basefilename'.
        """
        filename = (basefilename + str(hash(code))[1:4] +
                    str(os.getpid()) + time.strftime("%M%S"))
        tries = 0
        import_list = []
        ext = ".pyx" if cythonfile else ".py"
        e = ImportError() # used if import_list remains empty
        if os.getcwd() not in sys.path:
            sys.path.insert(0, os.getcwd())
        while not import_list and tries < 3:
            try_file = filename + str(tries)
            file_ = open(try_file+ext, "w")
            file_.writelines(code)
            file_.close()
            if not os.access(try_file, os.R_OK):
                time.sleep(0.1)
            codeString = str("from " + try_file +
                             " import " + obj_name + '\n' +
                             "import_list.append(" + obj_name + ")")
            try:
                import_code = compile(codeString, '<string>', 'exec')
>               exec(import_code, locals())
E               ModuleNotFoundError: No module named 'cqobjevo_compiled_coeff_98549686147162'

/usr/lib/python3/dist-packages/qutip/qobjevo_codegen.py:45: ModuleNotFoundError

The above exception was the direct cause of the following exception:

self = <qutip.tests.test_sesolve.TestSESolve object at 0x7fe9e7e576e0>

    def test_06_4_compare_state_and_unitary_list_str(self):
        "sesolve: compare state and unitary operator evo - list str td"
        eps = 0.2 * 2*np.pi
        delta = 1.0 * 2*np.pi   # atom frequency
        w0 = 0.5*eps
        w1 = 0.5*delta
        H0 = w0*sigmaz()
        H1 = w1*sigmax()
        w_a = w0
    
        td_args = {'w_a':w_a}
        H = [H0, [H1, 'cos(w_a*t)']]
    
        psi0 = basis(2, 0)        # initial state
        tlist = np.linspace(0, 20, 200)
    
>       self.compare_evolution(H, psi0, tlist,
                        normalize=False, td_args=td_args, tol=5e-5)

/usr/lib/python3/dist-packages/qutip/tests/test_sesolve.py:322: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3/dist-packages/qutip/tests/test_sesolve.py:223: in compare_evolution
    out_s = sesolve(H, psi0, tlist, [sigmax(), sigmay(), sigmaz()],
/usr/lib/python3/dist-packages/qutip/sesolve.py:124: in sesolve
    ss = _sesolve_QobjEvo(H, tlist, args, options)
/usr/lib/python3/dist-packages/qutip/sesolve.py:155: in _sesolve_QobjEvo
    H_td.compile(omp=nthread)
/usr/lib/python3/dist-packages/qutip/qobjevo.py:1641: in compile
    self.coeff_get, Code, file_ = _compiled_coeffs(
/usr/lib/python3/dist-packages/qutip/qobjevo_codegen.py:94: in _compiled_coeffs
    coeff_obj, filename = _import_str(code, "cqobjevo_compiled_coeff_",
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

code = '#!python\n#cython: language_level=3\n# This file is generated automatically by QuTiP.\n\nimport numpy as np\ncimport ...id _call_core(self, double t, complex * coeff):\n        cdef double w_a = self.w_a\n\n        coeff[0] = cos(w_a*t)\n'
basefilename = 'cqobjevo_compiled_coeff_', obj_name = 'CompiledStrCoeff', cythonfile = True

    def _import_str(code, basefilename, obj_name, cythonfile=False):
        """
        Import 'obj_name' defined in 'code'.
        Using a temporary file starting by 'basefilename'.
        """
        filename = (basefilename + str(hash(code))[1:4] +
                    str(os.getpid()) + time.strftime("%M%S"))
        tries = 0
        import_list = []
        ext = ".pyx" if cythonfile else ".py"
        e = ImportError() # used if import_list remains empty
        if os.getcwd() not in sys.path:
            sys.path.insert(0, os.getcwd())
        while not import_list and tries < 3:
            try_file = filename + str(tries)
            file_ = open(try_file+ext, "w")
            file_.writelines(code)
            file_.close()
            if not os.access(try_file, os.R_OK):
                time.sleep(0.1)
            codeString = str("from " + try_file +
                             " import " + obj_name + '\n' +
                             "import_list.append(" + obj_name + ")")
            try:
                import_code = compile(codeString, '<string>', 'exec')
                exec(import_code, locals())
            except (ModuleNotFoundError, ImportError) as e:
                time.sleep(0.05)
                tries += 1
                _try_remove(try_file+ext)
                err = e
        if not import_list:
>           raise Exception("Could not convert string to importable function, "
                            "tmpfile:" + try_file + ext) from err
E           Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_98549686147162.pyx

/usr/lib/python3/dist-packages/qutip/qobjevo_codegen.py:52: Exception
================================================================================================================================ short test summary info =================================================================================================================================
FAILED qutip/tests/test_brmesolve_td.py::test_time_dependent_spline_in_c_ops - ModuleNotFoundError: No module named 'rhs4968610'
FAILED qutip/tests/test_brmesolve_td.py::test_split_operators_maintain_answer[_2_tuple_split] - ModuleNotFoundError: No module named 'rhs4968611'
FAILED qutip/tests/test_correlation.py::TestTimeDependence::test_varying_coefficient_hamiltonian_2ls[string] - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_46049686147042.pyx
FAILED qutip/tests/test_correlation.py::TestTimeDependence::test_varying_coefficient_hamiltonian_2ls[numpy] - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_26849686147052.pyx
FAILED qutip/tests/test_mesolve.py::TestMESolverConstDecay::testMEDecayAsStrList - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_60949686147062.pyx
FAILED qutip/tests/test_mesolve.py::TestMESolveStepFuncCoeff::test_array_str_coeff - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_57149686147072.pyx
FAILED qutip/tests/test_mesolve.py::TestMESolveStepFuncCoeff::test_array_str_py_coeff - Exception: Could not convert string to importable function, tmpfile:td_Qobj_single_str38349686147072.pyx
FAILED qutip/tests/test_openmp.py::test_openmp_mesolve_td - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_00149686147082.pyx
FAILED qutip/tests/test_propagator.py::testPropHOStrTd - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_91749686147092.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_call - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_45749686147092.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_call_args - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_45749686147102.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_mul_vec - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_45749686147112.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_mul_mat - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_45749686147112.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_expect_psi - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_45749686147122.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_expect_rho - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_45749686147122.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_with_state - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_76649686147132.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_pickle - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_11349686147132.pyx
FAILED qutip/tests/test_qobjevo.py::test_QobjEvo_safepickle - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_11349686147142.pyx
FAILED qutip/tests/test_sesolve.py::TestSESolve::test_04_1_state_with_list_str_H - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_96649686147152.pyx
FAILED qutip/tests/test_sesolve.py::TestSESolve::test_04_2_unitary_with_list_func_H - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_96649686147152.pyx
FAILED qutip/tests/test_sesolve.py::TestSESolve::test_06_4_compare_state_and_unitary_list_str - Exception: Could not convert string to importable function, tmpfile:cqobjevo_compiled_coeff_98549686147162.pyx
============================================================================================================== 21 failed, 15 passed, 3233 deselected, 33 warnings in 15.37s ==============================================================================================================

Expected Behaviour

Tests should pass with python3.12.

Your Environment

$ python3.12 -c "import qutip; qutip.about()"

QuTiP: Quantum Toolbox in Python
================================
Copyright (c) QuTiP team 2011 and later.
Current admin team: Alexander Pitchford, Nathan Shammah, Shahnawaz Ahmed, Neill Lambert, Eric Giguère, Boxi Li, Jake Lishman, Simon Cross and Asier Galicia.
Board members: Daniel Burgarth, Robert Johansson, Anton F. Kockum, Franco Nori and Will Zeng.
Original developers: R. J. Johansson & P. D. Nation.
Previous lead developers: Chris Granade & A. Grimsmo.
Currently developed through wide collaboration. See https://github.com/qutip for details.

QuTiP Version:      4.7.3
Numpy Version:      1.24.2
Scipy Version:      1.10.1
Cython Version:     0.29.36
Matplotlib Version: 3.6.3
Python Version:     3.12.1
Number of CPUs:     8
BLAS Info:          OPENBLAS
OPENMP Installed:   True
INTEL MKL Ext:      False
Platform Info:      Linux (x86_64)
Installation path:  /usr/lib/python3/dist-packages/qutip
================================================================================
Please cite QuTiP in your publication.
================================================================================
For your convenience a bibtex reference can be easily generated using `qutip.cite()`

Additional Context

cython3-legacy 0.29.36-3

[hodgestar]

@Ericgig It would be good if this was fixed in 4.7.4.
@drew-parsons Could you try with the qutip-4.7.X branch which has some fixes?

[hodgestar]

We don't currently test or build for 3.12 on the qutip-4.7.X branch. Probably we should since 5 isn't fully released yet.

[Ericgig]

Cython 0.29 is not compatible with python 3.12.
On debian, they have the module python3-zombie-imp which is a copy of the removed imp module, but it does not seems available on pip or conda. Thus cython jit (pyximport) is not working on 3.12 for almost everyone.

Adding cython3 to 4.7.X is not a quick fix and we already choose not to do it.

The best we can do is to run as if cython was not installed at runtime on 3.12 and release 5.0.0 soon.

[hodgestar]

If the issue is Cython support for 3.12, let's just leave it for now as you suggest.

[drew-parsons]

In any case, these tests still fail with HEAD on the qutip-4.7.X branch (7a883d3)

[drew-parsons]

I disabled cython support on python 3.12 with this patch:

Index: qutip/qutip/__init__.py
===================================================================
--- qutip.orig/qutip/__init__.py	2024-01-16 03:52:59.780085342 +0100
+++ qutip/qutip/__init__.py	2024-01-16 03:52:59.780085342 +0100
@@ -53,25 +53,31 @@
 except ImportError:
     pass
 else:
-    from qutip.utilities import _version2int
-    _cy_require = "0.29.20"
-    _cy_unsupported = "3.0.0"
-    if _version2int(_Cython.__version__) < _version2int(_cy_require):
-        warnings.warn(
-            "Old version of Cython detected: needed {}, got {}."
-            .format(_cy_require, _Cython.__version__)
-        )
-    if _version2int(_Cython.__version__) >= _version2int(_cy_unsupported):
-        warnings.warn(
-            "The new version of Cython, (>= 3.0.0) is not supported."
-            .format(_Cython.__version__)
-        )
+    from sys import version_info as sys_pyver
+    if sys_pyver.major == 3 and sys_pyver.minor >= 12:
+        # cython3 not supported on python 3.12
+        # see https://github.com/qutip/qutip/issues/2293
+        print(f"qutip does not support Cython on Python 3.{sys_pyver.minor} (Issue#2293)")
     else:
-        # Setup pyximport
-        import qutip.cy.pyxbuilder as _pyxbuilder
-        _pyxbuilder.install()
-        del _pyxbuilder, _Cython, _version2int
-        qutip.settings.has_cython = True
+        from qutip.utilities import _version2int
+        _cy_require = "0.29.20"
+        _cy_unsupported = "3.0.0"
+        if _version2int(_Cython.__version__) < _version2int(_cy_require):
+            warnings.warn(
+                "Old version of Cython detected: needed {}, got {}."
+                .format(_cy_require, _Cython.__version__)
+            )
+        if _version2int(_Cython.__version__) >= _version2int(_cy_unsupported):
+            warnings.warn(
+                "The new version of Cython, (>= 3.0.0) is not supported."
+                .format(_Cython.__version__)
+            )
+        else:
+            # Setup pyximport
+            import qutip.cy.pyxbuilder as _pyxbuilder
+            _pyxbuilder.install()
+            del _pyxbuilder, _Cython, _version2int
+            qutip.settings.has_cython = True
 
 
 # -----------------------------------------------------------------------------
Index: qutip/qutip/tests/test_brmesolve_td.py
===================================================================
--- qutip.orig/qutip/tests/test_brmesolve_td.py	2024-01-16 03:52:59.780085342 +0100
+++ qutip/qutip/tests/test_brmesolve_td.py	2024-01-16 03:52:59.780085342 +0100
@@ -148,7 +148,8 @@
     brme = qutip.brmesolve(H, psi0, times, a_ops, e_ops=[a.dag()*a])
     assert np.mean(np.abs(brme.expect[0] - exact) / exact) < 1e-5
 
-
+@pytest.mark.skipif(not qutip.settings.has_cython,
+                    reason="cython support is not available")
 def test_time_dependent_spline_in_c_ops():
     N = 10
     a = qutip.destroy(N)
@@ -232,6 +233,8 @@
              [(a, a.dag()), (spectrum, '1', '1')]])
 
 
+@pytest.mark.skipif(not qutip.settings.has_cython,
+                    reason="cython support is not available")
 @pytest.mark.parametrize("collapse_operators", [
         _2_tuple_split,
         pytest.param(_4_tuple_split, marks=pytest.mark.slow),
Index: qutip/qutip/about.py
===================================================================
--- qutip.orig/qutip/about.py	2024-01-16 03:52:59.780085342 +0100
+++ qutip/qutip/about.py	2024-01-16 03:54:39.029027082 +0100
@@ -44,7 +44,8 @@
         cython_ver = Cython.__version__
     except ImportError:
         cython_ver = 'None'
-    print("Cython Version:     %s" % cython_ver)
+    print("Cython Version:     %s%s" % (cython_ver,
+            "" if cython_ver == 'None' or qutip.settings.has_cython else " (unsupported)"))
     try:
         import matplotlib
         matplotlib_ver = matplotlib.__version__

[Ericgig]

Should be fixed with the 4.7.5 release.