Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gevent._greenlet.pxd no longer provided in wheel (1.5 regression) #1568

Closed
navytux opened this issue Apr 15, 2020 · 9 comments
Closed

gevent._greenlet.pxd no longer provided in wheel (1.5 regression) #1568

navytux opened this issue Apr 15, 2020 · 9 comments

Comments

@navytux
Copy link

navytux commented Apr 15, 2020

  • gevent version: gevent-1.5.0-cp37-cp37m-manylinux2010_x86_64.whl
  • Python version: CPython 3.7.3 (default, Dec 20 2019, 18:57:59) provided by Debian10
  • Operating System: Debian GNU/Linux 10, Linux deco 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux

Description:

Hello up there. I've tried to pip install pygolang and got compilation failure with error saying that 'gevent/_greenlet.pxd' not found. I clearly remember this used to work out of the box on a fresh virtualenv. Investigating further it indeed confirmed that pygolang installation succeeds with gevent-1.4 and fails with recently released gevent-1.5 (btw thanks for the release). It boiled down to the following: gevent-1.4 ships _greenlet.pxd in its wheel, while gevent-1.5 no longer ships it:

gevent-1.4.0-cp37-cp37m-manylinux1_x86_64.whl
_abstract_linkable.c
__abstract_linkable.cpython-37m-x86_64-linux-gnu.so
_abstract_linkable.html
__abstract_linkable.pxd
_abstract_linkable.py
ares.py
backdoor.py
baseserver.py
builtins.py
_compat.py
_config.py
core.py
event.c
_event.cpython-37m-x86_64-linux-gnu.so
event.html
_event.pxd
event.py
events.py
exceptions.py
_ffi
_fileobjectcommon.py
_fileobjectposix.py
fileobject.py
greenlet.c
_greenlet.cpython-37m-x86_64-linux-gnu.so
greenlet.html
_greenlet_primitives.c
__greenlet_primitives.cpython-37m-x86_64-linux-gnu.so
_greenlet_primitives.html
__greenlet_primitives.pxd
_greenlet_primitives.py
_greenlet.pxd                           <-- NOTE
greenlet.py
_hub_local.c
__hub_local.cpython-37m-x86_64-linux-gnu.so
_hub_local.html
__hub_local.pxd
_hub_local.py
_hub_primitives.c
__hub_primitives.cpython-37m-x86_64-linux-gnu.so
_hub_primitives.html
__hub_primitives.pxd
_hub_primitives.py
hub.py
_ident.c
__ident.cpython-37m-x86_64-linux-gnu.so
_ident.html
__ident.pxd
_ident.py
_imap.c
__imap.cpython-37m-x86_64-linux-gnu.so
_imap.html
__imap.pxd
_imap.py
__init__.py
_interfaces.py
libev
libuv
local.c
_local.cpython-37m-x86_64-linux-gnu.so
local.html
_local.pxd
local.py
lock.py
_monitor.py
monkey.py
os.py
_patcher.py
pool.py
pywsgi.py
queue.c
_queue.cpython-37m-x86_64-linux-gnu.so
queue.html
_queue.pxd
queue.py
resolver
resolver_ares.py
resolver_thread.py
select.py
_semaphore.c
__semaphore.cpython-37m-x86_64-linux-gnu.so
_semaphore.html
__semaphore.pxd
_semaphore.py
server.py
signal.py
_socket2.py
_socket3.py
_socketcommon.py
socket.py
_ssl2.py
_ssl3.py
_sslgte279.py
ssl.py
subprocess.py
_tblib.py
testing
tests
_threading.py
threading.py
threadpool.py
thread.py
timeout.py
time.py
_tracer.c
__tracer.cpython-37m-x86_64-linux-gnu.so
_tracer.html
__tracer.pxd
_tracer.py
_util.py
util.py
_util_py2.py
_waiter.c
__waiter.cpython-37m-x86_64-linux-gnu.so
_waiter.html
__waiter.pxd
_waiter.py
win32util.py
gevent-1.5.0-cp37-cp37m-manylinux2010_x86_64.whl
__abstract_linkable.cpython-37m-x86_64-linux-gnu.so
_abstract_linkable.py
ares.py
backdoor.py
baseserver.py
builtins.py
_compat.py
_config.py
contextvars.py
core.py
_event.cpython-37m-x86_64-linux-gnu.so
event.py
events.py
exceptions.py
_ffi
_fileobjectcommon.py
_fileobjectposix.py
fileobject.py
_greenlet.cpython-37m-x86_64-linux-gnu.so
__greenlet_primitives.cpython-37m-x86_64-linux-gnu.so
_greenlet_primitives.py
greenlet.py
__hub_local.cpython-37m-x86_64-linux-gnu.so
_hub_local.py
__hub_primitives.cpython-37m-x86_64-linux-gnu.so
_hub_primitives.py
hub.py
__ident.cpython-37m-x86_64-linux-gnu.so
_ident.py
__imap.cpython-37m-x86_64-linux-gnu.so
_imap.py
__init__.py
_interfaces.py
libev
libuv
_local.cpython-37m-x86_64-linux-gnu.so
local.py
lock.py
_monitor.py
monkey.py
os.py
_patcher.py
pool.py
pywsgi.py
_queue.cpython-37m-x86_64-linux-gnu.so
queue.py
resolver
resolver_ares.py
resolver_thread.py
select.py
__semaphore.cpython-37m-x86_64-linux-gnu.so
_semaphore.py
server.py
signal.py
_socket2.py
_socket3.py
_socketcommon.py
socket.py
_ssl2.py
_ssl3.py
_sslgte279.py
ssl.py
subprocess.py
_tblib.py
testing
tests
_threading.py
threading.py
threadpool.py
thread.py
timeout.py
time.py
__tracer.cpython-37m-x86_64-linux-gnu.so
_tracer.py
_util.py
util.py
_util_py2.py
__waiter.cpython-37m-x86_64-linux-gnu.so
_waiter.py
win32util.py

It still works fine if I pip install -e gevent-1.5 in development mode from a git checkout. It breaks only if gevent is installed from a released wheel. Unfortunately the "released gevent install" is the default mode for most of the users.

Here is the place in pygolang that depends on gevent._greenlet.pxd and other gevent cython bits:

https://lab.nexedi.com/kirr/pygolang/blob/874371e2/golang/runtime/_runtime_gevent.pyx#L28-30

Is it please possible to restore providing the pxds to users?

Thanks beforehand,
Kirill

P.S.

`pip install pygolang` output
kirr@deco:~/tmp/trashme/r$ virtualenv -p python3 3.venv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/kirr/tmp/trashme/r/3.venv/bin/python3
Also creating executable in /home/kirr/tmp/trashme/r/3.venv/bin/python
Installing setuptools, pkg_resources, pip, wheel....done.
kirr@deco:~/tmp/trashme/r$ . 3.venv/bin/activate
(3.venv) kirr@deco:~/tmp/trashme/r$ pip install pygolang
Collecting pygolang
  Downloading pygolang-0.0.6.post1.tar.gz (177 kB)
     |████████████████████████████████| 177 kB 711 kB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting decorator
  Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting gevent
  Using cached gevent-1.5.0-cp37-cp37m-manylinux2010_x86_64.whl (5.1 MB)
Collecting six
  Downloading six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting geventmp; python_version >= "3"
  Downloading geventmp-0.0.1.dev3-py2.py3-none-any.whl (15 kB)
Collecting greenlet>=0.4.14; platform_python_implementation == "CPython"
  Using cached greenlet-0.4.15-cp37-cp37m-manylinux1_x86_64.whl (42 kB)
Building wheels for collected packages: pygolang
  Building wheel for pygolang (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/kirr/tmp/trashme/r/3.venv/bin/python3 /home/kirr/tmp/trashme/r/3.venv/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmppm5owj3e
       cwd: /tmp/pip-install-aex872tx/pygolang
  Complete output (315 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/gpython
  copying gpython/gpython_test.py -> build/lib.linux-x86_64-3.7/gpython
  copying gpython/__init__.py -> build/lib.linux-x86_64-3.7/gpython
  creating build/lib.linux-x86_64-3.7/golang
  copying golang/time_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/time.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/testing.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/sync_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/sync.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/strings_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/strconv_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/strconv.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/io_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/io.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/golang_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/gcompat_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/gcompat.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/fmt_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/fmt.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/errors_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/errors.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/cxx_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/context_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/context.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/_gopath_test.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/_gopath.py -> build/lib.linux-x86_64-3.7/golang
  copying golang/__init__.py -> build/lib.linux-x86_64-3.7/golang
  creating build/lib.linux-x86_64-3.7/golang/x
  copying golang/x/__init__.py -> build/lib.linux-x86_64-3.7/golang/x
  creating build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/__init__.py -> build/lib.linux-x86_64-3.7/golang/runtime
  creating build/lib.linux-x86_64-3.7/golang/pyx
  copying golang/pyx/runtime_test.py -> build/lib.linux-x86_64-3.7/golang/pyx
  copying golang/pyx/build_test.py -> build/lib.linux-x86_64-3.7/golang/pyx
  copying golang/pyx/build.py -> build/lib.linux-x86_64-3.7/golang/pyx
  copying golang/pyx/__init__.py -> build/lib.linux-x86_64-3.7/golang/pyx
  creating build/lib.linux-x86_64-3.7/golang/cmd
  copying golang/cmd/pybench.py -> build/lib.linux-x86_64-3.7/golang/cmd
  copying golang/cmd/__init__.py -> build/lib.linux-x86_64-3.7/golang/cmd
  creating build/lib.linux-x86_64-3.7/golang/_patch
  copying golang/_patch/pytest_py2.py -> build/lib.linux-x86_64-3.7/golang/_patch
  copying golang/_patch/ipython_py2.py -> build/lib.linux-x86_64-3.7/golang/_patch
  copying golang/_patch/__init__.py -> build/lib.linux-x86_64-3.7/golang/_patch
  creating build/lib.linux-x86_64-3.7/golang/x/perf
  copying golang/x/perf/benchlib.py -> build/lib.linux-x86_64-3.7/golang/x/perf
  copying golang/x/perf/__init__.py -> build/lib.linux-x86_64-3.7/golang/x/perf
  running egg_info
  writing pygolang.egg-info/PKG-INFO
  writing dependency_links to pygolang.egg-info/dependency_links.txt
  writing entry points to pygolang.egg-info/entry_points.txt
  writing requirements to pygolang.egg-info/requires.txt
  writing top-level names to pygolang.egg-info/top_level.txt
  reading manifest file 'pygolang.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching 'golang/pyx/runtime.cpp'
  writing manifest file 'pygolang.egg-info/SOURCES.txt'
  creating build/lib.linux-x86_64-3.7/gpython/testdata
  copying gpython/testdata/hello.py -> build/lib.linux-x86_64-3.7/gpython/testdata
  copying gpython/testdata/world.py -> build/lib.linux-x86_64-3.7/gpython/testdata
  copying golang/__init__.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/_context.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/_context.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_cxx_test.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_errors.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/_errors.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_errors_test.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_fmt.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/_fmt.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_fmt_test.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_golang.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/_golang.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_golang_test.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_io.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/_io.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_strings_test.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_sync.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/_sync.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_sync_test.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/_testing.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/_time.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/_time.pyx -> build/lib.linux-x86_64-3.7/golang
  copying golang/context.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/context.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/context.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/cxx.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/cxx.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/cxx_test.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/errors.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/errors.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/errors.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/errors_test.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/fmt.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/fmt.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/fmt.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/fmt_test.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/io.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/io.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/io.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/libgolang.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/strings.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/strings.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/strings.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/strings_test.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/sync.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/sync.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/sync.pxd -> build/lib.linux-x86_64-3.7/golang
  copying golang/sync_test.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/time.cpp -> build/lib.linux-x86_64-3.7/golang
  copying golang/time.h -> build/lib.linux-x86_64-3.7/golang
  copying golang/time.pxd -> build/lib.linux-x86_64-3.7/golang
  creating build/lib.linux-x86_64-3.7/golang/testdata
  creating build/lib.linux-x86_64-3.7/golang/testdata/src
  creating build/lib.linux-x86_64-3.7/golang/testdata/src/lab.nexedi.com
  creating build/lib.linux-x86_64-3.7/golang/testdata/src/lab.nexedi.com/kirr
  copying golang/testdata/src/lab.nexedi.com/kirr/hello.py -> build/lib.linux-x86_64-3.7/golang/testdata/src/lab.nexedi.com/kirr
  creating build/lib.linux-x86_64-3.7/golang/testdata/src/lab.nexedi.com/kirr/world
  copying golang/testdata/src/lab.nexedi.com/kirr/world/__init__.py -> build/lib.linux-x86_64-3.7/golang/testdata/src/lab.nexedi.com/kirr/world
  creating build/lib.linux-x86_64-3.7/golang/testprog
  copying golang/testprog/golang_test_defer_excchain.py -> build/lib.linux-x86_64-3.7/golang/testprog
  copying golang/testprog/golang_test_defer_excchain.txt -> build/lib.linux-x86_64-3.7/golang/testprog
  copying golang/testprog/golang_test_goleaked.py -> build/lib.linux-x86_64-3.7/golang/testprog
  copying golang/runtime/_libgolang.pxd -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/_runtime_gevent.pyx -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/_runtime_pymisc.pxd -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/_runtime_thread.pxd -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/_runtime_thread.pyx -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/libgolang.cpp -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/libgolang_test.cpp -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/libgolang_test_c.c -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/runtime/libpyxruntime.cpp -> build/lib.linux-x86_64-3.7/golang/runtime
  copying golang/pyx/_runtime_test.pyx -> build/lib.linux-x86_64-3.7/golang/pyx
  copying golang/pyx/runtime.h -> build/lib.linux-x86_64-3.7/golang/pyx
  copying golang/pyx/runtime.pxd -> build/lib.linux-x86_64-3.7/golang/pyx
  copying golang/pyx/runtime.pyx -> build/lib.linux-x86_64-3.7/golang/pyx
  creating build/lib.linux-x86_64-3.7/golang/pyx/testprog
  copying golang/pyx/testprog/cmdclass_custom.py -> build/lib.linux-x86_64-3.7/golang/pyx/testprog
  creating build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_dso_user
  copying golang/pyx/testprog/golang_dso_user/pyproject.toml -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_dso_user
  copying golang/pyx/testprog/golang_dso_user/setup.py -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_dso_user
  creating build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_dso_user/dsouser
  copying golang/pyx/testprog/golang_dso_user/dsouser/__init__.py -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_dso_user/dsouser
  copying golang/pyx/testprog/golang_dso_user/dsouser/dso.cpp -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_dso_user/dsouser
  copying golang/pyx/testprog/golang_dso_user/dsouser/test.pyx -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_dso_user/dsouser
  creating build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_pyx_user
  copying golang/pyx/testprog/golang_pyx_user/pyproject.toml -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_pyx_user
  copying golang/pyx/testprog/golang_pyx_user/setup.py -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_pyx_user
  creating build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_pyx_user/pyxuser
  copying golang/pyx/testprog/golang_pyx_user/pyxuser/__init__.py -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_pyx_user/pyxuser
  copying golang/pyx/testprog/golang_pyx_user/pyxuser/test.pyx -> build/lib.linux-x86_64-3.7/golang/pyx/testprog/golang_pyx_user/pyxuser
  running build_dso
  Building DSOs
  building 'golang.runtime.libgolang' DSO as build/lib.linux-x86_64-3.7/golang/runtime/liblibgolang.so.0.1
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/golang
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/golang/runtime
  creating build/temp.linux-x86_64-3.7/golang
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBGOLANG -I. -I3rdparty/include -Ibuild/lib.linux-x86_64-3.7/. -c golang/runtime/libgolang.cpp -o build/temp.linux-x86_64-3.7/golang/runtime/libgolang.o -std=gnu++11
  creating build/temp.linux-x86_64-3.7/golang
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBGOLANG -I. -I3rdparty/include -Ibuild/lib.linux-x86_64-3.7/. -c golang/context.cpp -o build/temp.linux-x86_64-3.7/golang/context.o -std=gnu++11
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBGOLANG -I. -I3rdparty/include -Ibuild/lib.linux-x86_64-3.7/. -c golang/errors.cpp -o build/temp.linux-x86_64-3.7/golang/errors.o -std=gnu++11
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBGOLANG -I. -I3rdparty/include -Ibuild/lib.linux-x86_64-3.7/. -c golang/fmt.cpp -o build/temp.linux-x86_64-3.7/golang/fmt.o -std=gnu++11
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBGOLANG -I. -I3rdparty/include -Ibuild/lib.linux-x86_64-3.7/. -c golang/sync.cpp -o build/temp.linux-x86_64-3.7/golang/sync.o -std=gnu++11
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBGOLANG -I. -I3rdparty/include -Ibuild/lib.linux-x86_64-3.7/. -c golang/time.cpp -o build/temp.linux-x86_64-3.7/golang/time.o -std=gnu++11
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBGOLANG -I. -I3rdparty/include -Ibuild/lib.linux-x86_64-3.7/. -c golang/io.cpp -o build/temp.linux-x86_64-3.7/golang/io.o -std=gnu++11
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBGOLANG -I. -I3rdparty/include -Ibuild/lib.linux-x86_64-3.7/. -c golang/strings.cpp -o build/temp.linux-x86_64-3.7/golang/strings.o -std=gnu++11
  x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/golang/runtime/libgolang.o build/temp.linux-x86_64-3.7/golang/io.o build/temp.linux-x86_64-3.7/golang/context.o build/temp.linux-x86_64-3.7/golang/strings.o build/temp.linux-x86_64-3.7/golang/errors.o build/temp.linux-x86_64-3.7/golang/sync.o build/temp.linux-x86_64-3.7/golang/fmt.o build/temp.linux-x86_64-3.7/golang/time.o -o build/lib.linux-x86_64-3.7/golang/runtime/liblibgolang.so.0.1 -Wl,-h,liblibgolang.so.0.1
  symlink liblibgolang.so.0.1 <- build/lib.linux-x86_64-3.7/golang/runtime/liblibgolang.so
  building 'golang.runtime.libpyxruntime' DSO as build/lib.linux-x86_64-3.7/golang/runtime/liblibpyxruntime.so.0.1
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBUILDING_LIBPYXRUNTIME -I. -I/usr/include/python3.7m -Ibuild/temp.linux-x86_64-3.7/. -Ibuild/lib.linux-x86_64-3.7/. -c golang/runtime/libpyxruntime.cpp -o build/temp.linux-x86_64-3.7/golang/runtime/libpyxruntime.o -std=c++11
  x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/golang/runtime/libpyxruntime.o -Lbuild/lib.linux-x86_64-3.7/golang/runtime -llibgolang -o build/lib.linux-x86_64-3.7/golang/runtime/liblibpyxruntime.so.0.1 -Wl,-rpath,\$ORIGIN/. -Wl,-rpath,$ORIGIN/. -Wl,-h,liblibpyxruntime.so.0.1
  symlink liblibpyxruntime.so.0.1 <- build/lib.linux-x86_64-3.7/golang/runtime/liblibpyxruntime.so
  running build_ext
  cythoning golang/_golang.pyx to golang/_golang.cpp
  cythoning golang/runtime/_runtime_thread.pyx to golang/runtime/_runtime_thread.c
  cythoning golang/runtime/_runtime_gevent.pyx to golang/runtime/_runtime_gevent.c
  cythoning golang/pyx/runtime.pyx to golang/pyx/runtime.cpp
  cythoning golang/_golang_test.pyx to golang/_golang_test.cpp
  cythoning golang/pyx/_runtime_test.pyx to golang/pyx/_runtime_test.cpp
  cythoning golang/_context.pyx to golang/_context.cpp
  cythoning golang/_cxx_test.pyx to golang/_cxx_test.cpp
  cythoning golang/_errors.pyx to golang/_errors.cpp
  cythoning golang/_errors_test.pyx to golang/_errors_test.cpp
  cythoning golang/_fmt.pyx to golang/_fmt.cpp
  cythoning golang/_fmt_test.pyx to golang/_fmt_test.cpp
  cythoning golang/_io.pyx to golang/_io.cpp
  cythoning golang/_strings_test.pyx to golang/_strings_test.cpp
  cythoning golang/_sync.pyx to golang/_sync.cpp
  cythoning golang/_sync_test.pyx to golang/_sync_test.cpp
  cythoning golang/_time.pyx to golang/_time.cpp
  building 'golang._golang' extension
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I. -I/home/kirr/tmp/trashme/r/3.venv/include/site/python3.7 -Ibuild/temp.linux-x86_64-3.7/. -I/usr/include/python3.7m -I/home/kirr/tmp/trashme/r/3.venv/include/python3.7m -c golang/_golang.cpp -o build/temp.linux-x86_64-3.7/golang/_golang.o -std=c++11 -fno-strict-aliasing
  golang/_golang.cpp: In function ‘PyObject* __pyx_f_6golang_7_golang_verify_chanElemBuf()’:
  golang/_golang.cpp:8569:21: warning: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Wsign-compare]
       if (((__pyx_t_4 > __pyx_t_5) != 0)) {
             ~~~~~~~~~~^~~~~~~~~~~
  x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/golang/_golang.o -Lbuild/lib.linux-x86_64-3.7/golang/runtime -llibgolang -o build/lib.linux-x86_64-3.7/golang/_golang.cpython-37m-x86_64-linux-gnu.so -Wl,-rpath,\$ORIGIN/runtime -Wl,-rpath,$ORIGIN/runtime
  building 'golang.runtime._runtime_thread' extension
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I. -I/home/kirr/tmp/trashme/r/3.venv/include/site/python3.7 -Ibuild/temp.linux-x86_64-3.7/. -I/usr/include/python3.7m -I/home/kirr/tmp/trashme/r/3.venv/include/python3.7m -c golang/runtime/_runtime_thread.c -o build/temp.linux-x86_64-3.7/golang/runtime/_runtime_thread.o -fno-strict-aliasing
  golang/runtime/_runtime_thread.c:2144:18: warning: ‘__Pyx_CFunc_uint64__t________nogil_to_py’ defined but not used [-Wunused-function]
   static PyObject *__Pyx_CFunc_uint64__t________nogil_to_py(uint64_t (*__pyx_v_f)(void)) {
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  golang/runtime/_runtime_thread.c:2006:18: warning: ‘__Pyx_CFunc_void____uint64__t____nogil_to_py’ defined but not used [-Wunused-function]
   static PyObject *__Pyx_CFunc_void____uint64__t____nogil_to_py(void (*__pyx_v_f)(uint64_t)) {
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.7/golang/runtime/_runtime_thread.o -Lbuild/lib.linux-x86_64-3.7/golang/runtime -llibgolang -o build/lib.linux-x86_64-3.7/golang/runtime/_runtime_thread.cpython-37m-x86_64-linux-gnu.so -Wl,-rpath,\$ORIGIN/. -Wl,-rpath,$ORIGIN/.
  building 'golang.runtime._runtime_gevent' extension
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I. -I/home/kirr/tmp/trashme/r/3.venv/include/site/python3.7 -Ibuild/temp.linux-x86_64-3.7/. -I/usr/include/python3.7m -I/home/kirr/tmp/trashme/r/3.venv/include/python3.7m -c golang/runtime/_runtime_gevent.c -o build/temp.linux-x86_64-3.7/golang/runtime/_runtime_gevent.o -fno-strict-aliasing
  golang/runtime/_runtime_gevent.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
   #error Do not use this file, it is the result of a failed Cython compilation.
    ^~~~~
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  
  # Gevent runtime uses gevent's greenlets and semaphores.
  # When sema.acquire() blocks, gevent switches us from current to another greenlet.
  
  IF not PYPY:
      from gevent._greenlet cimport Greenlet
     ^
  ------------------------------------------------------------
  
  golang/runtime/_runtime_gevent.pyx:28:4: 'gevent/_greenlet.pxd' not found
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  
  # Gevent runtime uses gevent's greenlets and semaphores.
  # When sema.acquire() blocks, gevent switches us from current to another greenlet.
  
  IF not PYPY:
      from gevent._greenlet cimport Greenlet
     ^
  ------------------------------------------------------------
  
  golang/runtime/_runtime_gevent.pyx:28:4: 'gevent/_greenlet/Greenlet.pxd' not found
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  # Gevent runtime uses gevent's greenlets and semaphores.
  # When sema.acquire() blocks, gevent switches us from current to another greenlet.
  
  IF not PYPY:
      from gevent._greenlet cimport Greenlet
      from gevent.__semaphore cimport Semaphore
     ^
  ------------------------------------------------------------
  
  golang/runtime/_runtime_gevent.pyx:29:4: 'gevent/__semaphore.pxd' not found
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  # Gevent runtime uses gevent's greenlets and semaphores.
  # When sema.acquire() blocks, gevent switches us from current to another greenlet.
  
  IF not PYPY:
      from gevent._greenlet cimport Greenlet
      from gevent.__semaphore cimport Semaphore
     ^
  ------------------------------------------------------------
  
  golang/runtime/_runtime_gevent.pyx:29:4: 'gevent/__semaphore/Semaphore.pxd' not found
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  # When sema.acquire() blocks, gevent switches us from current to another greenlet.
  
  IF not PYPY:
      from gevent._greenlet cimport Greenlet
      from gevent.__semaphore cimport Semaphore
      ctypedef Semaphore PYGSema
              ^
  ------------------------------------------------------------
  
  golang/runtime/_runtime_gevent.pyx:30:13: 'Semaphore' is not a type identifier
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  cdef:
      # XXX better panic with pyexc object and detect that at recover side?
  
      bint _go(void (*f)(void *) nogil, void *arg):
          _ = _togo(); _.f = f; _.arg = arg
          g = Greenlet(_goviapy, _)
             ^
  ------------------------------------------------------------
  
  golang/runtime/_runtime_gevent.pyx:67:12: 'Greenlet' is not a constant, variable or function identifier
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          g = Greenlet(_goviapy, _)
          g.start()
          return True
  
      _libgolang_sema* _sema_alloc():
          pygsema = Semaphore()
                   ^
  ------------------------------------------------------------
  
  golang/runtime/_runtime_gevent.pyx:72:18: 'Semaphore' is not a constant, variable or function identifier
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for pygolang
Failed to build pygolang
ERROR: Could not build wheels for pygolang which use PEP 517 and cannot be installed directly
@jamadden
Copy link
Member

jamadden commented Apr 15, 2020

The pxd files were never a documented interface. They're an internal implementation detail, subject to change at any time, even between minor releases, to the point where they might even come and go over time. That's especially true of the ones that have been added recently such as greenlet, queue, etc. It was an accident that they were ever in a distribution to start with, and a packaging update in 1.5a4 removed them.

I'd really like to avoid having to support them. Can you just not use them, like you don't on PyPy?

@navytux
Copy link
Author

navytux commented Apr 15, 2020

@jamadden thanks for feedback. While I understand the desire to minimize the surface of provided public API, there are reasons on pygolang side to prefer Cython interface compared to pure Python: the ultimate goal of pygolang is to provide its service in completely nogil mode at Cython level. This is currenly already achieved for thread runtime, but is only partially achieved for gevent runtime. Reverting to use gevent/py instead of gevent/pyx would be a step in opposite direction with respect to pygolang goal.

Besides nogil rationale using gevent/py instead of gevent/pyx brings performance regression even at Python level(*):

name             old time/op  new time/op   delta
go               12.6µs ± 2%   13.5µs ± 1%   +7.34%  (p=0.000 n=10+10)
chan             6.00µs ± 1%   6.81µs ± 1%  +13.40%  (p=0.000 n=10+10)
select           8.82µs ± 2%  10.16µs ± 1%  +15.13%  (p=0.000 n=10+10)
def              55.0ns ± 0%   57.2ns ± 2%   +4.00%  (p=0.000 n=10+10)
func_def         43.7µs ± 0%   44.0µs ± 1%   +0.57%  (p=0.015 n=10+10)
call             66.5ns ± 1%   64.8ns ± 4%     ~     (p=0.324 n=10+10)
func_call        1.25µs ± 1%   1.24µs ± 3%     ~     (p=0.218 n=10+9)
try_finally       138ns ± 1%    137ns ± 1%   -1.01%  (p=0.003 n=10+10)
defer            2.62µs ± 2%   2.58µs ± 2%   -1.59%  (p=0.045 n=10+10)
workgroup_empty  42.7µs ± 2%   49.0µs ± 1%  +14.66%  (p=0.000 n=10+10)
workgroup_raise  54.1µs ± 3%   62.5µs ± 3%  +15.67%  (p=0.000 n=10+10)

Regarding benchmarking at Python vs Cython-level - it is just that the benchmarks currently exist only in Python form. When used at Cython level (pygolang can be already used like this even with gevent runtime) the slowdown would be much more compared to the above 15%, because most of Python-level ballast overhead would be gone.

Said that isn't gevent/_greenlet.pxd, that we are talking about, providing Cython-level annotations for public gevent/greenlet.py? If i understand correctly the pxd is put separate and used in augmenting mode to keep gevent/greenlet.py pure python and so whole gevent usable in pure-python environments where Cython or compilation is not available. Is it indeed the case?

At least when using .pyx and .pxd Cython allows to separate cdef bits into public and private. I'm not sure such mechanism currently exists for .py + .pxd but would it help if there would be a way to fully annotate .py in augmenting .pxd file while also having a "public" .pxd that gets installed and that describes the "minimal public API"?

Once again, I understand the desire to avoid exposing things as API to users, but given the above rationales and that it already used to work out of the box for at least ~ 1 year, maybe we could do something that it is ok for both gevent and pygolang?

Thanks beforehand,
Kirill

P.S. I'm considering to drop PyPy support every time I hit a bug in its runtime, but so far it got survived.

(*) performance tests were done with caring about measurement stability.

@jamadden
Copy link
Member

jamadden commented Apr 15, 2020

Let me try to be more clear. The definitions in the .pxd files are not public, documented or supported. They may change at any time. And when they do, any project that had consumed them must be recompiled or it is highly likely to be broken. This could be the easy thing where an extension module won't even load because of unresolved symbols, but it's at least as likely to simply crash. The way that Cython lays out a class's vtable is fragile from the ABI point of view, and methods invoked through it use hard-coded offsets; any change in that vtable invalidates all previously compiled code. Calling a method is likely to jump to code expecting different parameters or jump into garbage.

If you wish to take that risk for the sake of performance, and wish to restrict users to exactly the version of gevent that your extension was compiled with, that's totally fine. But I don't want to encourage that because I don't have the time to deal with the inevitable bug reports; I choose to discourage that by not distributing the pxd files.

Perhaps you can copy the definitions you wish to use ("vendor" them)?

@navytux
Copy link
Author

navytux commented Apr 15, 2020

Thanks @jamadden, your message is clear. Even it is a pity to hear, I can understand you.
I will try to think what to do on my side.

navytux added a commit to navytux/pygolang that referenced this issue Apr 16, 2020
Starting from gevent >= 1.5 '*.pxd' files for gevent API are no longer
provided, at least in released gevent wheels. This broke pygolang:

    Error compiling Cython file:
    ------------------------------------------------------------
    ...

    # Gevent runtime uses gevent's greenlets and semaphores.
    # When sema.acquire() blocks, gevent switches us from current to another greenlet.

    IF not PYPY:
        from gevent._greenlet cimport Greenlet
       ^
    ------------------------------------------------------------

    golang/runtime/_runtime_gevent.pyx:28:4: 'gevent/_greenlet.pxd' not found

Since gevent upstream refuses to restore Cython level access[1], let's fix the
build by using gevent bits via Python-level.

Even when used via py import gevent-1.5 brings speed improvement compared to
gevent-1.4 (used via cimport):

	(on i7@2.6GHz, gevent runtime)

                      gevent-1.4   gevent-1.5
                      (cimport)    (py import)

    name              old time/op  new time/op  delta
    pyx_select_nogil  9.47µs ± 0%  8.74µs ± 0%   -7.70%  (p=0.000 n=10+9)
    pyx_go_nogil      14.3µs ± 1%  12.0µs ± 1%  -16.52%  (p=0.000 n=10+10)
    pyx_chan_nogil    7.10µs ± 1%  6.32µs ± 1%  -10.89%  (p=0.000 n=10+10)
    go                16.0µs ± 2%  13.4µs ± 1%  -16.37%  (p=0.000 n=10+10)
    chan              7.50µs ± 0%  6.79µs ± 0%   -9.53%  (p=0.000 n=10+10)
    select            10.8µs ± 1%  10.0µs ± 1%   -6.78%  (p=0.000 n=10+10)

Using gevent-1.5 could have been even faster via cimport (it is still
possible to compile and test against gevent installed in development
mode via `pip install -e` because pxd files are there in gevent worktree
and tarball):

                      gevent-1.5   gevent-1.5
                      (py import)  (cimport)

    name              old time/op  new time/op  delta
    pyx_select_nogil  8.74µs ± 0%  7.90µs ± 1%  -9.60%  (p=0.000 n=9+10)
    pyx_go_nogil      12.0µs ± 1%  11.2µs ± 2%  -6.35%  (p=0.000 n=10+10)
    pyx_chan_nogil    6.32µs ± 1%  5.89µs ± 0%  -6.80%  (p=0.000 n=10+9)
    go                13.4µs ± 1%  12.4µs ± 1%  -7.54%  (p=0.000 n=10+9)
    chan              6.79µs ± 0%  6.42µs ± 0%  -5.47%  (p=0.000 n=10+10)
    select            10.0µs ± 1%   9.4µs ± 1%  -6.39%  (p=0.000 n=10+10)

but we cannot use cimport to access gevent-1.5 universally, since pxd are not
shipped in gevent wheel releases.

In the future we might want to change plain version check into compile time
check whether gevent/_greenlet.pxd is actually present or not and use faster
access if yes. Requesting gevent to be installed in non-binary form
might be also an option worth trying.

However plain version check should be ok for now.

[1] gevent/gevent#1568
@jamadden
Copy link
Member

jamadden commented Apr 21, 2020

Argh! I myself spent time struggling with the binary compatibility issue today. I made what I thought was a trivial change (for speed!) in one pxd file, rebuilt, and ran tests. They passed. Except one that hardcore crashed the interpreter in a Py_incref of a nonsense value. Turned out adjusting a data member (of course) broke the vtable in a way that wasn’t picked up by any dependency tracking, so the wrong code was called with the wrong parameters by another cython module.

I truly sympathize with the desire for maximum performance, but the safety just isn’t there. It took me longer than I want to admit to figure out what was happening, and that was in a fully controlled, single project environment with debug flags enabled.

As these internal optimizations are obviously still in flux and likely to stay that way I can’t in good conscience foist these types of issues onto users or other maintainers. Thank you for understanding.

@navytux
Copy link
Author

navytux commented Apr 22, 2020

@jamadden, thanks for feedback. I understand you. With the given state of lack of reliable Cython dependency tracking, and gevent pxd bits being still in flux, it probably indeed makes more sense not to expose pxd API to users in general, at least for now.

The problem of unreliable dependency tracking on Cython side is old and, sadly, well-known. I myself am used to do git clean -fdX before rebuilding my projects just not to get into any kind of miscompilation trap you just described. For small projects it is annoying, but kind of works. However for larger codebases it does not work at all due to slow compilation and having to wait for long. It would be very good to teach Cython to implement reliable dependency tracking, so that whenever modules are compiled or just pyximport'ed, there is 100% just no way to get an inconsistently-compiled code to run.

Even if we are disabling pxd interface for gevent for now, I have to share that in my company gevent is regularly criticized for being slow. It is good to see gevent increasingly improving from release to release (thanks for your hard work on this!), but loosing any part of potential speedup is pity, since the competition is in order of magnitude, not percents:

https://nexedi.com/NXD-Document.Blog.UVLoop.Python.Benchmark
https://nexedi.com/NXD-Blog.Multicore.Python.HTTP.Server
https://nexedi.com/NXD-Blog.Cython.Multithreaded.Coroutines
https://nexedi.com/blog/NXD-Document.Blog.Cypclass
https://pypi.org/project/pygolang/#pygolang-change-history (0.0.3 - 0.0.5)

In my view, compared to other ways, gevent/greenlet/greenstack/... provide big advantage by preserving backward compatibility for existing codebases. This gives ability to switch existing programs into lightweight-threads mode without major rewrite and to further enhance them incrementally instead of rewriting everything in one go.

Once again, I understand current situation and the rationale for decision to keep pxd in private.
However I still hope that one day gevent pxd bits could be reexposed, and for Cython compilation to become working reliably.

Kirill

/cc @scoder, @samaingw, @xavth, @gidzit

@scoder
Copy link

scoder commented Apr 22, 2020

The problem of unreliable dependency tracking on Cython side is old and, sadly, well-known.

Is it? It hasn't failed me in years, so could you describe your setup and what doesn't work with it in a Cython ticket?

@navytux
Copy link
Author

navytux commented Apr 22, 2020

@scoder, thanks for feedback. I think what @jamadden hit in #1568 (comment) is cython/cython#1428.

For non-pure-python mode there is also a lot of confusion of whether to use Cython.Distutils.build_ext or Cython.Build.cythonize. While cythonize is mostly ok wrt dependency tracking on the first glance, lots of projects are still used to Cython.Distutils.build_ext / Cython.Build.build_ext which does not implement proper dependency handling: cython/cython#3541.

See also cython/cython#3208.

@scoder
Copy link

scoder commented Apr 22, 2020

Ah, yes. old_build_ext is really annoying, but still used a lot. I commented on the ticket that you created, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants