path.sep goes missing when importing in a debugger in Python 3.10

[TomTruck]

I've created a repository that contains a working example that demonstrates this problem.

Code Sample, a copy-pastable example if possible

The problem arises when importing pyproj in a debugger. A minimal example is as follows.

import os

import pyproj

os.path.sep

if __name__ == '__main__':
    pass

More details are available in the repository README.

Problem description

When importing pyproj in a debugger the os.path module gets modified so that os.path.sep raises an AttributeError

Expected Output

os.path.sep returns the same value as before importing os.path.sep

Environment Information

Run inside a docker container defined as follows:

FROM python:3.10.2-bullseye@sha256:083f7050ed966b5e430b5c0ebc439af58094ac59f37861573966121652d2d790  AS base

WORKDIR /app

RUN pip install --upgrade pip==21.3.1

RUN curl -sSL https://install.python-poetry.org | python - --yes --version 1.1.12
ENV PATH="/root/.local/bin:$PATH"

RUN poetry config virtualenvs.create false

COPY pyproject.toml .

RUN poetry lock

RUN poetry install --no-interaction

Dependencies are defined via Poetry in a pyproject.toml file:

[tool.poetry]
name = "Debug missing sep"
version = "0.1.0"
description = "Routing"
authors = ["Tom Finill"]


[tool.poetry.dependencies]
python = "==3.10.2"
pyproj = "==3.3.0"

Installation method

Poetry (see Dockerfile above)

[snowman2]

I am wondering if this problem should be addressed by the debugger you mentioned and not in pyproj itself?

[TomTruck]

That's definitely a possibility. Although I'm thinking that the issue may be with Cython not playing nice with the new debug changes in Python 3.10. I guess if you can't see anything that's obviously wrong I'll try opening an issue elsewhere.

[snowman2]

That makes sense. If you wouldn't mind opening up an issue with Cython, that would be helpful to further pin down the issue.

[jannismain]

I also stumbled upon this issue, while trying to debug a codebase that relies on pyproj. It doesn't work and trying to patch around the exception wasn't successful either.

I'm not seeing any issues over at cython/cython that address this issue. @scoder are you aware of this?

Over at pytest, somebody mentioned that this is fixed in cython==3.0.0a10. So I guess in the meantime I'm going to build pyproj using that pre-release version and hope for the best 🙏🏻 😅

/edit: for my concrete use case, this is the traceback that is produced when trying to debug any pytest test that imports from itur/pyproj/certifi:

ImportError while loading conftest '[./test/conftest.py]()'.
test[/conftest.py:3](): in <module>
src[/some_module.py:8](): in <module>
    import itur
.[./../.pyenv/versions/3.10.2/lib/python3.10/site-packages/itur/__init__.py:29](): in <module>
    import itur.utils
.[./../.pyenv/versions/3.10.2/lib/python3.10/site-packages/itur/utils.py:18](): in <module>
    from pyproj import Geod
.[./../.pyenv/versions/3.10.2/lib/python3.10/site-packages/pyproj/__init__.py:93](): in <module>
    pyproj.network.set_ca_bundle_path()
.[./../.pyenv/versions/3.10.2/lib/python3.10/site-packages/pyproj/network.py:51](): in set_ca_bundle_path
    ca_bundle_path = certifi.where()
.[./../.pyenv/versions/3.10.2/lib/python3.10/site-packages/certifi/core.py:36](): in where
    _CACERT_CTX = get_path("certifi", "cacert.pem")
E   AttributeError: module 'posixpath' has no attribute 'sep'

[RemiDesgrange]

I have the problem too, and when I look at the posxipath object it seems that either pyproj or cython polute this package:

[scoder]

Might be this: cython/cython#4735

[RemiDesgrange]

If I read the issue correctly and the release on pypi, we need to use cython from source from the branch 0.29, or cython 3.

I tried installing this way:

pip install cython==3.0.0a10
pip install --no-binary ':all:' pyproj
=> run my project with debugpy, and it fail.

Maybe I'm missing something. I also tried it in a docker container and a brand new venv.

[snowman2]

How about this?

python -m pip install cython==3.0.0a10
python -m pip install --no-use-pep517 --no-binary ':all:' pyproj

[RemiDesgrange]

Working thanks 👍

[snowman2]

Cython 0.29.29 has been released: https://github.com/cython/cython/releases/tag/0.29.29. This should be resolved in the wheels build with the next release of pyproj.

In the meantime, this should work:

python -m pip install cython>=0.29.29
python -m pip install --no-use-pep517 --no-binary ':all:' pyproj