Add support for Python 3.12 #2711

weiji14 · 2023-10-02T20:36:52Z

Description of proposed changes

Python 3.12 has been released on 2 Oct 2023, changelog is at https://docs.python.org/3.12/whatsnew/3.12.html

Previous PR for Python 3.11 at #2172

Fixes #

Reminders

Run make format and make check to make sure the code follows the style guide.
Add tests for new features or tests that would have caught the bug that you're fixing.
Add new public functions/methods/classes to doc/api/index.rst.
Write detailed docstrings for all functions/methods.
If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
If adding new functionality, add an example to docstrings or tutorials.
Use underscores (not hyphens) in names of Python files and directories.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash commands are:

/format: automatically format and lint the code
/test-gmt-dev: run full tests on the latest GMT development version

Python 3.12 has been released on 2 Oct 2023, changelog is at https://docs.python.org/3.12/whatsnew/3.12.html

.github/workflows/ci_docs.yml

weiji14 · 2023-10-03T21:22:44Z

Waiting for a new build of rasterio on conda-forge that supports Python 3.12, lots of migrations happening at https://conda-forge.org/status/#python312. Needs to go through GDAL first probably.

Current traceback from https://github.com/GenericMappingTools/pygmt/actions/runs/6385364893/job/17329957379?pr=2711#step:3:55:

  error    libmamba Could not solve for environment specs
      The following packages are incompatible
      ├─ contextily is installable with the potential options
      │  ├─ contextily [0.9.0|0.9.2] would require
      │  │  └─ python 2.7* , which can be installed;
      │  ├─ contextily 0.9.2 would require
      │  │  └─ python 3.4* , which can be installed;
      │  ├─ contextily 0.9.2 would require
      │  │  └─ python 3.5* , which can be installed;
      │  └─ contextily [0.9.2|0.99.0|...|1.3.0] would require
      │     └─ rasterio with the potential options
      │        ├─ rasterio [1.0.23|1.0.24|...|1.1.3] would require
      │        │  └─ python >=2.7,<2.8.0a0 , which can be installed;
      │        ├─ rasterio [1.0.23|1.0.24|...|1.1.3] would require
      │        │  └─ python >=3.6,<3.7.0a0 , which can be installed;
      │        ├─ rasterio [1.0.23|1.0.24|...|1.1.3] would require
      │        │  └─ python >=3.7,<3.8.0a0 , which can be installed;
      │        ├─ rasterio [1.1.1|1.1.2|1.1.3] would require
      │        │  └─ python >=3.8,<3.9.0a0 , which can be installed;
      │        ├─ rasterio [1.1.4|1.1.5|...|1.2.9] would require
      │        │  └─ python_abi 3.6.* *_cp36m, which can be installed;
      │        ├─ rasterio [1.1.4|1.1.5|...|1.2.9] would require
      │        │  └─ python_abi 3.7.* *_cp37m, which can be installed;
      │        ├─ rasterio [1.1.4|1.1.5|...|1.3.8] would require
      │        │  └─ python_abi 3.8.* *_cp38, which can be installed;
      │        ├─ rasterio [1.1.7|1.1.8|...|1.3.8] would require
      │        │  └─ python_abi 3.9.* *_cp39, which can be installed;
      │        ├─ rasterio [1.2.10|1.3.0|...|1.3.8] would require
      │        │  └─ python_abi 3.10.* *_cp310, which can be installed;
      │        └─ rasterio [1.3.3|1.3.4|1.3.6|1.3.7|1.3.8] would require
      │           └─ python_abi 3.11.* *_cp311, which can be installed;
      └─ python 3.12**  is not installable because it requires
         └─ python_abi 3.12.* *_cp312, which conflicts with any installable versions previously reported.

weiji14 · 2023-10-11T20:36:14Z

Also getting this error on flakeheaven with Python 3.12 at https://github.com/GenericMappingTools/pygmt/actions/runs/6487636754/job/17618364397#step:5:23:

FLAKEHEAVEN_CACHE_TIMEOUT=0 flakeheaven lint pygmt doc/conf.py examples
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.0/x64/bin/flakeheaven", line 8, in <module>
    sys.exit(entrypoint())
             ^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flakeheaven/_cli.py", line 40, in entrypoint
    exit_code, msg = main(argv)
                     ^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flakeheaven/_cli.py", line 32, in main
    return COMMANDS[command_name](argv=argv[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flakeheaven/commands/_lint.py", line 12, in lint_command
    app.run(argv)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flake8/main/application.py", line 375, in run
    self._run(argv)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flake8/main/application.py", line 363, in _run
    self.initialize(argv)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flake8/main/application.py", line 343, in initialize
    self.find_plugins(config_finder)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flakeheaven/_patched/_app.py", line 229, in find_plugins
    self.check_plugins = FlakeHeavenCheckers(local_plugins.extension)  # this line is changed
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flakeheaven/_patched/_plugins.py", line 65, in __init__
    self.manager = FlakeHeavenPluginManager(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flakeheaven/_patched/_plugins.py", line 47, in __init__
    self._load_entrypoint_plugins()
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/flake8/plugins/manager.py", line 261, in _load_entrypoint_plugins
    eps = importlib_metadata.entry_points().get(self.namespace, ())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'EntryPoints' object has no attribute 'get'
make: *** [Makefile:73: check] Error 1
Error: Process completed with exit code 2.

It seems like the error is on flake8 actually (see PyCQA/flake8#1701). But flakeheaven pins to flake8 ^4.0.1 at https://github.com/flakeheaven/flakeheaven/blob/3.3.0/pyproject.toml#L45

weiji14 · 2023-10-11T20:41:21Z

Looks like rasterio has Python 3.12 support now - conda-forge/rasterio-feedstock#284. Next error is with NumPy 1.25 and Python 3.12 at https://github.com/GenericMappingTools/pygmt/actions/runs/6487636757/job/17618365052?pr=2711#step:3:60

       The following packages are incompatible
      ├─ numpy 1.25**  is installable with the potential options
      │  ├─ numpy [1.25.0|1.25.1|1.25.2] would require
      │  │  └─ python_abi 3.10.* *_cp310, which can be installed;
      │  ├─ numpy [1.25.0|1.25.1|1.25.2] would require
      │  │  └─ python_abi 3.11.* *_cp311, which can be installed;
      │  ├─ numpy [1.25.0|1.25.1|1.25.2] would require
      │  │  └─ pypy3.9 >=7.3.11 , which can be installed;
      │  └─ numpy [1.25.0|1.25.1|1.25.2] would require
      │     └─ python_abi 3.9.* *_cp39, which can be installed;
      └─ python 3.12**  is not installable because it requires
         └─ python_abi 3.12.* *_cp312, which conflicts with any installable versions previously reported.

May need to use NumPy 1.26 (https://anaconda.org/conda-forge/numpy/files?version=1.26.0), so need to merge the PR at #2692? But that will involve a ghostscript update too 🙂

seisman · 2023-10-12T01:48:53Z

It seems like the error is on flake8 actually (see PyCQA/flake8#1701). But flakeheaven pins to flake8 ^4.0.1 at https://github.com/flakeheaven/flakeheaven/blob/3.3.0/pyproject.toml#L45

xref #2741

This reverts commit 6bded07.

weiji14 · 2023-10-15T23:44:00Z

Ok, Style Checks working on Python 3.12 now after #2747, but we'll still need to merge the NumPy 1.26 PR at #2692 first for ci_tests.yml as mentioned at #2711 (comment).

weiji14 · 2023-10-25T03:40:59Z

Ok, dependencies can be installed now with Python 3.12 and NumPy 1.26. Getting this AttributeError now for one test at https://github.com/GenericMappingTools/pygmt/actions/runs/6635357023/job/18026155793#step:8:713:

 =================================== FAILURES ===================================
____________________________ test_load_libgmt_fails ____________________________

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fa5291267b0>

    @pytest.mark.skipif(sys.platform == "win32", reason="run on UNIX platforms only")
    def test_load_libgmt_fails(monkeypatch):
        """
        Test that GMTCLibNotFoundError is raised when GMT's shared library cannot
        be found.
        """
        with monkeypatch.context() as mpatch:
            mpatch.setattr(sys, "platform", "win32")  # pretend to be on Windows
            mpatch.setattr(
                subprocess, "check_output", lambda cmd, encoding: "libfakegmt.so"
            )
            with pytest.raises(GMTCLibNotFoundError):
>               check_libgmt(load_libgmt())

../pygmt/tests/test_clib_loading.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../pygmt/clib/loading.py:47: in load_libgmt
    for libname in lib_fullnames:
../pygmt/clib/loading.py:135: in clib_full_names
    libfullpath = find_library(libname)
../../../../micromamba/envs/pygmt/lib/python3.12/ctypes/util.py:351: in find_library
    _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
../../../../micromamba/envs/pygmt/lib/python3.12/ctypes/util.py:110: in _findLib_gcc
    c_compiler = shutil.which('gcc')
../../../../micromamba/envs/pygmt/lib/python3.12/shutil.py:1543: in which
    if sys.platform == "win32" and _win_path_needs_curdir(cmd, mode):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cmd = 'gcc', mode = 1

    def _win_path_needs_curdir(cmd, mode):
        """
        On Windows, we can use NeedCurrentDirectoryForExePath to figure out
        if we should add the cwd to PATH when searching for executables if
        the mode is executable.
        """
>       return (not (mode & os.X_OK)) or _winapi.NeedCurrentDirectoryForExePath(
                    os.fsdecode(cmd))
E       AttributeError: 'NoneType' object has no attribute 'NeedCurrentDirectoryForExePath'

../../../../micromamba/envs/pygmt/lib/python3.12/shutil.py:1497: AttributeError

Seems related to something on CPython? I found python/cpython#103179, but not sure if it's relevant because the test above is failing on Ubuntu, not Windows.

seisman · 2023-10-25T04:09:50Z

The test test_load_libgmt_fails was initially added in commit 243ad13.

The test is skipped on Windows, but we also have mpatch.setattr(sys, "platform", "win32") to pretend the test is run on Windows. This explains why we're having the error:

    if sys.platform == "win32" and _win_path_needs_curdir(cmd, mode):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cmd = 'gcc', mode = 1

    def _win_path_needs_curdir(cmd, mode):
        """
        On Windows, we can use NeedCurrentDirectoryForExePath to figure out
        if we should add the cwd to PATH when searching for executables if
        the mode is executable.
        """
>       return (not (mode & os.X_OK)) or _winapi.NeedCurrentDirectoryForExePath(
                    os.fsdecode(cmd))
E       AttributeError: 'NoneType' object has no attribute 'NeedCurrentDirectoryForExePath'

../../../../micromamba/envs/pygmt/lib/python3.12/shutil.py:1497: AttributeError

It's still UNIX, but we force sys.platform to be win32, so the if test passes but the _winapi, from the name I guess it's Windows only, doesn't exist on UNIX.

seisman · 2023-10-25T04:11:49Z

We also need to address the following deprecation warnings:

../../../../micromamba/envs/pygmt/lib/python3.12/site-packages/dateutil/tz/tz.py:37
  /home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/dateutil/tz/tz.py:37: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    EPOCH = datetime.datetime.utcfromtimestamp(0)

pygmt/tests/test_clib_put_vector.py::test_put_vector_string_dtype
  /home/runner/work/pygmt/pygmt/pygmt/tests/test_clib_put_vector.py:125: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    f"{datetime.utcnow().strftime('%Y-%m-%d')}T04:50:06",

Replace deprecated datetime.datetime.utcnow() with datetime.datetime.now(tz=datetime.UTC).

weiji14 · 2023-10-25T05:03:00Z

It's still UNIX, but we force sys.platform to be win32, so the if test passes but the _winapi, from the name I guess it's Windows only, doesn't exist on UNIX.

Ah yes, so _winapi is None on UNIX, so the 'pretend-to-be-on-Windows' monkeypatch won't work anymore. Might need some time to find a workaround to keep that test.

seisman · 2023-10-25T05:12:37Z

It's still UNIX, but we force sys.platform to be win32, so the if test passes but the _winapi, from the name I guess it's Windows only, doesn't exist on UNIX.

Ah yes, so _winapi is None on UNIX, so the 'pretend-to-be-on-Windows' monkeypatch won't work anymore. Might need some time to find a workaround to keep that test.

Actually I'm a little confused why we have this line:

mpatch.setattr(sys, "platform", "win32")  # pretend to be on Windows

Workaround for AttributeError: 'NoneType' object has no attribute 'NeedCurrentDirectoryForExePath' when calling`_winapi.NeedCurrentDirectoryForExePath`, where `_winapi` is None on UNIX. See #2711 (comment)

weiji14 · 2023-10-26T01:08:24Z

It's still UNIX, but we force sys.platform to be win32, so the if test passes but the _winapi, from the name I guess it's Windows only, doesn't exist on UNIX.

Ah yes, so _winapi is None on UNIX, so the 'pretend-to-be-on-Windows' monkeypatch won't work anymore. Might need some time to find a workaround to keep that test.

Actually I'm a little confused why we have this line:
mpatch.setattr(sys, "platform", "win32")  # pretend to be on Windows

I tried to remove that line and ran the test on Linux, and it failed because GMTCLibNotFoundError was not raised. Looking at commit 243ad13 in #702 a bit more closely, I think the idea was to try and raise GMTCLibNotFoundError by pretending to be on Windows (so that the script looks for gmt.dll), but have subprocess.check_output(["gmt", "--show-library"], ...) return libfakegmt.so (that doesn't actually exist). That libfakegmt.so will then try to be loaded with ctypes here:

pygmt/pygmt/clib/loading.py

Lines 47 to 59 in fddf0a3

    
           for libname in lib_fullnames: 
        
               try: 
        
                   if libname not in failing_libs:  # skip the lib if it's known to fail 
        
                       libgmt = ctypes.CDLL(libname) 
        
                       check_libgmt(libgmt) 
        
                       error = False 
        
                       break 
        
               except (OSError, GMTCLibError) as err: 
        
                   error_msg.append(f"Error loading GMT shared library at '{libname}'.\n{err}") 
        
                   failing_libs.append(libname) 
        
           if error: 
        
               raise GMTCLibNotFoundError("\n".join(error_msg))

But since libfakegmt.so doesn't exist, the GMTCLibNotFoundError will be raised. I think I might have a workaround, which is to pretend to be on macOS instead of Windows (a048c86), so that _winapi is not used 😆

pygmt/tests/test_clib_put_vector.py

Make the test compatible with Python<3.11

weiji14 · 2023-10-26T01:46:15Z

pygmt/tests/test_clib_loading.py

@@ -72,7 +72,7 @@ def test_load_libgmt_fails(monkeypatch):
    be found.
    """
    with monkeypatch.context() as mpatch:
-        mpatch.setattr(sys, "platform", "win32")  # pretend to be on Windows
+        mpatch.setattr(sys, "platform", "darwin")  # pretend to be on macOS


Ok, this workaround I applied in a048c86 doesn't actually work on macOS...

Your workaround a048c86 fails on macOS CI, because on macOS, PyGMT will find the gmt.dylib successfully in the system default path (I think conda's lib path is in the default search path)

Two possible workarounds:

1. Monkeypatch `sys.platform` so it always returns the wrong platform, e.g., return `linux` on macOS/Windows and `darwin` on Linux 2. Instead of monkeypatching `sys.platform`, we can monkeypath `clib_names` to return invalid library names.

Ok, trying out Option 1 in 99fdbf8. Edit: Yep, it worked!

weiji14 · 2023-10-26T01:49:03Z

.github/workflows/ci_tests_dev.yaml

@@ -100,7 +100,7 @@ jobs:
          cache-downloads: true
          cache-environment: true
          create-args: >-
-            python=3.11
+            python=3.12


GMT Dev Tests failing because pygit2 doesn't have Python 3.12 wheels, and building from source fails at https://github.com/GenericMappingTools/pygmt/actions/runs/6648200102/job/18064869372#step:7:570:

Building wheel for pygit2 (pyproject.toml): started Building wheel for pygit2 (pyproject.toml): finished with status 'error' error: subprocess-exited-with-error × Building wheel for pygit2 (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [66 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-312 creating build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/index.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/ffi.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/submodule.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/credentials.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/_run.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/utils.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/config.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/errors.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/_build.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/refspec.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/repository.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/blame.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/packbuilder.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/remote.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/callbacks.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/settings.py -> build/lib.linux-x86_64-cpython-312/pygit2 copying pygit2/__init__.py -> build/lib.linux-x86_64-cpython-312/pygit2 creating build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/transport.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/errors.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/blame.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/checkout.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/net.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/strarray.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/merge.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/diff.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/graph.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/describe.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/refspec.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/buffer.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/proxy.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/commit.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/revert.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/config.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/common.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/oid.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/attr.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/repository.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/remote.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/types.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/pack.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/callbacks.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/clone.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/indexer.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/stash.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/index.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/decl/submodule.h -> build/lib.linux-x86_64-cpython-312/pygit2/decl copying pygit2/_pygit2.pyi -> build/lib.linux-x86_64-cpython-312/pygit2 running build_ext generating cffi module 'build/temp.linux-x86_64-cpython-312/pygit2._libgit2.c' creating build/temp.linux-x86_64-cpython-312 building 'pygit2._pygit2' extension creating build/temp.linux-x86_64-cpython-312/src gcc -pthread -B /home/runner/micromamba/envs/pygmt/compiler_compat -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/runner/micromamba/envs/pygmt/include -fPIC -O2 -isystem /home/runner/micromamba/envs/pygmt/include -fPIC -I/usr/local/include -I/home/runner/micromamba/envs/pygmt/include/python3.12 -c src/blob.c -o build/temp.linux-x86_64-cpython-312/src/blob.o In file included from src/blob.c:30: src/diff.h:33:10: fatal error: git2.h: No such file or directory 33 | #include <git2.h> | ^~~~~~~~ compilation terminated. error: command '/usr/bin/gcc' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for pygit2 Building wheel for frozenlist (pyproject.toml): started Building wheel for frozenlist (pyproject.toml): finished with status 'done' Created wheel for frozenlist: filename=frozenlist-1.4.0-cp312-cp312-linux_x86_64.whl size=53997 sha256=39bb8d4c56a724d4cd9ed87de83b7de69e78a79763d0d9f5ae5e95f8497b5b83 Stored in directory: /home/runner/.cache/pip/wheels/f1/9c/94/9386cb0ea511a93226456388d41d35f1c24ba15a62ffd7b1ef Building wheel for multidict (pyproject.toml): started Building wheel for multidict (pyproject.toml): finished with status 'done' Created wheel for multidict: filename=multidict-6.0.4-cp312-cp312-linux_x86_64.whl size=34743 sha256=b99632e55f0994365432703c422b45c7b80332be4690bfa943602cb30e69d013 Stored in directory: /home/runner/.cache/pip/wheels/f6/d8/ff/3c14a64b8f2ab1aa94ba2888f5a988be6ab446ec5c8d1a82da Building wheel for yarl (pyproject.toml): started Building wheel for yarl (pyproject.toml): finished with status 'done' Created wheel for yarl: filename=yarl-1.9.2-cp312-cp312-linux_x86_64.whl size=70034 sha256=d286d55089432e38d0441fa83a86e4a994b9537a62cec71e97187e4d2d41a9ec Stored in directory: /home/runner/.cache/pip/wheels/84/e3/6a/7d0fa1abee8e4aa39922b5bd54689b4b5e4269b2821f482a32 Successfully built antlr4-python3-runtime dulwich frozenlist multidict yarl Failed to build pygit2 ERROR: Could not build wheels for pygit2, which is required to install pyproject.toml-based projects

Might need to wait for libgit2/pygit2#1240.

Rather than waiting for pygit2 to release Python 3.12 wheels, we could also keep the GMT Dev Tests on Python 3.11 for now?

Suggested change

python=3.12

python=3.11

Looks like pygit2 has Python 3.12 wheels now - https://pypi.org/project/pygit2/1.13.2/#files

seisman · 2023-10-26T05:27:15Z

I think the idea was to try and raise GMTCLibNotFoundError by pretending to be on Windows (so that the script looks for gmt.dll), but have subprocess.check_output(["gmt", "--show-library"], ...) return libfakegmt.so (that doesn't actually exist).

But since libfakegmt.so doesn't exist, the GMTCLibNotFoundError will be raised.

I think your understanding of this test is inaccurate. We look for the GMT shared library in the following directories:

GMT_LIBARRY_PATH (in this test, it's skipped because the evn variable is not defined)
The path returned by gmt --show-library (in this test, it returns libfakegmt.so, which doesn't exist. So it's skipped. See

pygmt/pygmt/clib/loading.py

Line 124 in fddf0a3

assert libfullpath.exists()
On Windows, search in PATH by calling find_library (in the old test, we pretend to be on Windows, so we're looking for gmt.dll on Linux/macOS, which doesn't exist)
The system default path (again, in the old test, we pretend to be on Windows, so we're looking for gmt.dll on Linux/macOS)

Your workaround a048c86 fails on macOS CI, because on macOS, PyGMT will find the gmt.dylib successfully in the system default path (I think conda's lib path is in the default search path)

Two possible workarounds:

Monkeypatch sys.platform so it always returns the wrong platform, e.g., return linux on macOS/Windows and darwin on Linux
Instead of monkeypatching sys.platform, we can monkeypath clib_names to return invalid library names.

Hack test_load_libgmt_fails, so that if on Linux, sys.platform returns 'darwin', and if on macOS, sys.platform returns 'linux'.

weiji14 · 2023-11-07T01:56:30Z

Ok, ready for review again! The ci_tests.yml now pass on all platforms, and the install step works in GMT Dev Tests (though tests are failing due to baseline image differences).

Add support for Python 3.12

fc5c445

Python 3.12 has been released on 2 Oct 2023, changelog is at https://docs.python.org/3.12/whatsnew/3.12.html

weiji14 added the maintenance Boring but important stuff for the core devs label Oct 2, 2023

weiji14 self-assigned this Oct 2, 2023

Add conda-forge/label/python_rc channel

6bded07

weiji14 commented Oct 2, 2023

View reviewed changes

.github/workflows/ci_docs.yml Outdated Show resolved Hide resolved

seisman added this to the 0.11.0 milestone Oct 4, 2023

Merge branch 'main' into python-3.12

caa65d3

weiji14 mentioned this pull request Oct 15, 2023

Replace flakeheaven and isort with ruff #2747

Merged

12 tasks

weiji14 added 2 commits October 16, 2023 12:35

Merge branch 'main' into python-3.12

534bc30

Revert "Add conda-forge/label/python_rc channel"

654e8be

This reverts commit 6bded07.

weiji14 added 2 commits October 24, 2023 15:24

Merge branch 'main' into python-3.12

5c0d900

Merge branch 'main' into python-3.12

a22dfd5

Fix DeprecationWarning with use of datetime.utcnow() on Python 3.12

d1b01c3

Replace deprecated datetime.datetime.utcnow() with datetime.datetime.now(tz=datetime.UTC).

Pretend to be on macOS instead of Windows

a048c86

Workaround for AttributeError: 'NoneType' object has no attribute 'NeedCurrentDirectoryForExePath' when calling`_winapi.NeedCurrentDirectoryForExePath`, where `_winapi` is None on UNIX. See #2711 (comment)

weiji14 commented Oct 26, 2023

View reviewed changes

pygmt/tests/test_clib_put_vector.py Outdated Show resolved Hide resolved

Use datetime.timezone.utc instead of Python 3.11+ only datetime.UTC

90a3a49

Make the test compatible with Python<3.11

weiji14 marked this pull request as ready for review October 26, 2023 01:20

weiji14 commented Oct 26, 2023

View reviewed changes

weiji14 added 2 commits November 7, 2023 14:29

Merge branch 'main' into python-3.12

486c37a

Pretend to be on macOS if running on Linux, and vice versa

99fdbf8

Hack test_load_libgmt_fails, so that if on Linux, sys.platform returns 'darwin', and if on macOS, sys.platform returns 'linux'.

weiji14 marked this pull request as draft November 7, 2023 01:37

weiji14 marked this pull request as ready for review November 7, 2023 01:38

weiji14 added the needs review This PR has higher priority and needs review. label Nov 7, 2023

seisman approved these changes Nov 7, 2023

View reviewed changes

seisman removed the needs review This PR has higher priority and needs review. label Nov 7, 2023

seisman merged commit a320c3f into main Nov 7, 2023
14 of 20 checks passed

seisman deleted the python-3.12 branch November 7, 2023 13:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Python 3.12 #2711

Add support for Python 3.12 #2711

weiji14 commented Oct 2, 2023

weiji14 commented Oct 3, 2023

weiji14 commented Oct 11, 2023

weiji14 commented Oct 11, 2023

seisman commented Oct 12, 2023

weiji14 commented Oct 15, 2023 •

edited

Loading

weiji14 commented Oct 25, 2023

seisman commented Oct 25, 2023 •

edited

Loading

seisman commented Oct 25, 2023 •

edited

Loading

weiji14 commented Oct 25, 2023

seisman commented Oct 25, 2023

weiji14 commented Oct 26, 2023

weiji14 Oct 26, 2023

weiji14 Nov 7, 2023 •

edited

Loading

weiji14 Oct 26, 2023

weiji14 Oct 31, 2023

seisman Oct 31, 2023

weiji14 Nov 7, 2023

seisman commented Oct 26, 2023 •

edited

Loading

weiji14 commented Nov 7, 2023

Add support for Python 3.12 #2711

Add support for Python 3.12 #2711

Conversation

weiji14 commented Oct 2, 2023

weiji14 commented Oct 3, 2023

weiji14 commented Oct 11, 2023

weiji14 commented Oct 11, 2023

seisman commented Oct 12, 2023

weiji14 commented Oct 15, 2023 • edited Loading

weiji14 commented Oct 25, 2023

seisman commented Oct 25, 2023 • edited Loading

seisman commented Oct 25, 2023 • edited Loading

weiji14 commented Oct 25, 2023

seisman commented Oct 25, 2023

weiji14 commented Oct 26, 2023

weiji14 Oct 26, 2023

Choose a reason for hiding this comment

weiji14 Nov 7, 2023 • edited Loading

Choose a reason for hiding this comment

weiji14 Oct 26, 2023

Choose a reason for hiding this comment

weiji14 Oct 31, 2023

Choose a reason for hiding this comment

seisman Oct 31, 2023

Choose a reason for hiding this comment

weiji14 Nov 7, 2023

Choose a reason for hiding this comment

seisman commented Oct 26, 2023 • edited Loading

weiji14 commented Nov 7, 2023

weiji14 commented Oct 15, 2023 •

edited

Loading

seisman commented Oct 25, 2023 •

edited

Loading

seisman commented Oct 25, 2023 •

edited

Loading

weiji14 Nov 7, 2023 •

edited

Loading

seisman commented Oct 26, 2023 •

edited

Loading