Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark functions as noexcept to ensure cython 3 compatibility #534

Closed
wants to merge 3 commits into from

Conversation

matusvalo
Copy link

@matusvalo matusvalo commented Mar 18, 2023

Cython 3 will require noexcept keyword for functions not raising an exception: https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#exception-values-and-noexcept
Cython 0.29.32 introduces this keyword too for forward compatibility. This PR fixes multiple issues found in cython/cython#5305.

Note: Currently uvloop cannot be compiled in cython 3.0.0b1 because of issue cython/cython#5270. In the next Cython release the issue should be fixed.

@matusvalo
Copy link
Author

Note: Currently uvloop cannot be compiled in cython 3.0.0b1 because of issue cython/cython#5270. In the next Cython release the issue should be fixed.

Now the fix is released with Cython 3.0.0b2

@scoder
Copy link

scoder commented May 26, 2023

@1st1 this is a straight forward change that makes the code base compile with Cython 3.0 (and 0.29.32+). Would be nice to get this merged.

@dulmandakh
Copy link
Contributor

@matusvalo could you please rebase from master, and see if CI is still green

@matusvalo
Copy link
Author

@matusvalo could you please rebase from master, and see if CI is still green

done

@dulmandakh
Copy link
Contributor

@fantix could you please review and merge, because we need Cython 3.0 to support Python 3.12 #555

@dulmandakh
Copy link
Contributor

@matusvalo could you please bump Cython to 3.0 with change below in setup.py and see if your changes fix compatibility.

CYTHON_DEPENDENCY = 'Cython~=3.0.0'

@matusvalo
Copy link
Author

matusvalo commented Aug 13, 2023

New bug created in Cython (causing failure of tests): cython/cython#5615

Cython 3 stops supporting mangles parameters starting with `__`
to reflect python behavior - see: cython/cython@abeb082
@kloczek
Copy link

kloczek commented Aug 30, 2023

After opening RFE with upgrade cython 3.x support (#558) I found this PR.
I made quick test and looks like build fails with this PR (in build env with cython 3.0.2)
In build output is possible to see use relative imports which using in case of modules with DSOs always is nothing more asking for troubled because using that forces to build DSOs in the source tree and in trashes "test as installed" methodology (install generated .whl in </install/prefix> and point $PYTHONPATH python sitearch/sitelib paths to inside </install/prefix> paths)

IMO it would be good first remove relative imports.

+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
* Building wheel...
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-38
creating build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/__init__.py -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/_noop.py -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/_testbase.py -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/_version.py -> build/lib.linux-x86_64-cpython-38/uvloop
running egg_info
creating uvloop.egg-info
writing uvloop.egg-info/PKG-INFO
writing dependency_links to uvloop.egg-info/dependency_links.txt
writing requirements to uvloop.egg-info/requires.txt
writing top-level names to uvloop.egg-info/top_level.txt
writing manifest file 'uvloop.egg-info/SOURCES.txt'
<string>:144: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
warning: uvloop/includes/consts.pxi:1:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:3:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:4:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:5:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:7:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:8:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:10:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:13:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:16:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:21:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:24:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:25:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310

Error compiling Cython file:
------------------------------------------------------------
...
# cython: language_level=3


from .includes cimport uv
^
------------------------------------------------------------

uvloop/loop.pxd:4:0: 'uvloop/includes.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
# cython: language_level=3


from .includes cimport uv
from .includes cimport system
^
------------------------------------------------------------

uvloop/loop.pxd:5:0: 'uvloop/includes.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
ctypedef object (*method3_t)(object, object, object, object)


cdef class Loop:
    cdef:
        uv.uv_loop_t *uvloop
        ^
------------------------------------------------------------

uvloop/loop.pxd:34:8: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
    cdef _init_debug_fields(self)

    cdef _on_wake(self)
    cdef _on_idle(self)

    cdef __run(self, uv.uv_run_mode)
                     ^
------------------------------------------------------------

uvloop/loop.pxd:136:21: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...

    cdef _on_wake(self)
    cdef _on_idle(self)

    cdef __run(self, uv.uv_run_mode)
    cdef _run(self, uv.uv_run_mode)
                    ^
------------------------------------------------------------

uvloop/loop.pxd:137:20: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
    cdef _getaddrinfo(self, object host, object port,
                      int family, int type,
                      int proto, int flags,
                      int unpack)

    cdef _getnameinfo(self, system.sockaddr *addr, int flags)
                            ^
------------------------------------------------------------

uvloop/loop.pxd:167:28: 'system' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
cdef class UVHandle:
    cdef:
        uv.uv_handle_t *_handle
        ^
------------------------------------------------------------

uvloop/handles/handle.pxd:3:8: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
cdef class UVStream(UVBaseTransport):
    cdef:
        uv.uv_shutdown_t _shutdown_req
        ^
------------------------------------------------------------

uvloop/handles/stream.pxd:3:8: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
cdef class TCPServer(UVStreamServer):
    cdef bind(self, system.sockaddr* addr, unsigned int flags=*)
                    ^
------------------------------------------------------------

uvloop/handles/tcp.pxd:2:20: 'system' is not declared

Error compiling Cython file:
------------------------------------------------------------
...

cdef class TCPTransport(UVStream):
    cdef:
        bint __peername_set
        bint __sockname_set
        system.sockaddr_storage __peername
        ^
------------------------------------------------------------

uvloop/handles/tcp.pxd:17:8: 'system' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
cdef class TCPTransport(UVStream):
    cdef:
        bint __peername_set
        bint __sockname_set
        system.sockaddr_storage __peername
        system.sockaddr_storage __sockname
        ^
------------------------------------------------------------

uvloop/handles/tcp.pxd:18:8: 'system' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
        bint __peername_set
        bint __sockname_set
        system.sockaddr_storage __peername
        system.sockaddr_storage __sockname

    cdef bind(self, system.sockaddr* addr, unsigned int flags=*)
                    ^
------------------------------------------------------------

uvloop/handles/tcp.pxd:20:20: 'system' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
        bint __sockname_set
        system.sockaddr_storage __peername
        system.sockaddr_storage __sockname

    cdef bind(self, system.sockaddr* addr, unsigned int flags=*)
    cdef connect(self, system.sockaddr* addr)
                       ^
------------------------------------------------------------

uvloop/handles/tcp.pxd:21:23: 'system' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
        bint _restore_signals

        list _fds_to_close

        # Attributes used to compose uv_process_options_t:
        uv.uv_process_options_t options
        ^
------------------------------------------------------------

uvloop/handles/process.pxd:14:8: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...

        list _fds_to_close

        # Attributes used to compose uv_process_options_t:
        uv.uv_process_options_t options
        uv.uv_stdio_container_t[3] iocnt
        ^
------------------------------------------------------------

uvloop/handles/process.pxd:15:8: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...

        list _fds_to_close

        # Attributes used to compose uv_process_options_t:
        uv.uv_process_options_t options
        uv.uv_stdio_container_t[3] iocnt
        ^
------------------------------------------------------------

uvloop/handles/process.pxd:15:8: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
cdef class UVRequest:
    cdef:
        uv.uv_req_t *request
        ^
------------------------------------------------------------

uvloop/request.pxd:3:8: 'uv' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
        bint __receiving
        int _family
        object _address

    cdef _init(self, Loop loop, unsigned int family)
    cdef _set_address(self, system.addrinfo *addr)
                            ^
------------------------------------------------------------

uvloop/handles/udp.pxd:8:28: 'system' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
        object _address

    cdef _init(self, Loop loop, unsigned int family)
    cdef _set_address(self, system.addrinfo *addr)

    cdef _connect(self, system.sockaddr* addr, size_t addr_len)
                        ^
------------------------------------------------------------

uvloop/handles/udp.pxd:10:24: 'system' is not declared

Error compiling Cython file:
------------------------------------------------------------
...
    cdef _init(self, Loop loop, unsigned int family)
    cdef _set_address(self, system.addrinfo *addr)

    cdef _connect(self, system.sockaddr* addr, size_t addr_len)

    cdef _bind(self, system.sockaddr* addr)
                     ^
------------------------------------------------------------

uvloop/handles/udp.pxd:12:21: 'system' is not declared
warning: uvloop/includes/consts.pxi:1:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:3:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:4:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:5:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:7:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:8:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:10:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:13:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:16:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:21:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:24:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/includes/consts.pxi:25:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/handles/stream.pyx:1:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/handles/process.pyx:337:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/handles/process.pyx:338:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/handles/process.pyx:339:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/handles/process.pyx:340:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: uvloop/handles/udp.pyx:351:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310

Error compiling Cython file:
------------------------------------------------------------
...
        self._recv_buffer_in_use = 0

        err = uv.uv_loop_init(self.uvloop)
        if err < 0:
            raise convert_error(err)
        self.uvloop.data = <void*> self
                           ^
------------------------------------------------------------

uvloop/loop.pyx:167:27: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        if self._handle is NULL:
            return

        # -> When we're at this point, something is wrong <-

        if self._handle.loop is NULL:
                             ^
------------------------------------------------------------

uvloop/handles/handle.pyx:44:29: Invalid types for 'is' (Python object, void *)

Error compiling Cython file:
------------------------------------------------------------
...

        # The handle is dealloced while open.  Let's try to close it.
        # Situations when this is possible include unhandled exceptions,
        # errors during Handle.__cinit__/__init__ etc.
        if self._inited:
            self._handle.data = NULL
                                ^
------------------------------------------------------------

uvloop/handles/handle.pyx:62:32: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
            self._closed = 1

    cdef inline _finish_init(self):
        self._inited = 1
        if self._has_handle == 1:
            self._handle.data = <void*>self
                                ^
------------------------------------------------------------

uvloop/handles/handle.pyx:117:32: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
                        '{} is alive, but _loop is None'.format(name))
                if self._handle.loop is not self._loop.uvloop:
                    raise RuntimeError(
                        '{} is alive, but _handle.loop is not '
                        'initialized'.format(name))
                if self._handle.data is not <void*>self:
                                     ^
------------------------------------------------------------

uvloop/handles/handle.pyx:151:37: Invalid types for 'is_not' (Python object, void *)

Error compiling Cython file:
------------------------------------------------------------
...

        if self._handle is NULL:
            return

        if UVLOOP_DEBUG:
            if self._handle.data is NULL:
                                 ^
------------------------------------------------------------

uvloop/handles/handle.pyx:194:33: Invalid types for 'is' (Python object, void *)

Error compiling Cython file:
------------------------------------------------------------
...
            return
        self.__shutting_down = 1

        self._ensure_alive()

        self._shutdown_req.data = <void*> self
                                  ^
------------------------------------------------------------

uvloop/handles/stream.pyx:257:34: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        TCPTransport transport
        uv.uv_connect_t _req_data

    def __cinit__(self, loop, transport):
        self.request = <uv.uv_req_t*>&self._req_data
        self.request.data = <void*>self
                            ^
------------------------------------------------------------

uvloop/handles/tcp.pyx:192:28: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        UnixTransport transport
        uv.uv_connect_t _req_data

    def __cinit__(self, loop, transport):
        self.request = <uv.uv_req_t*> &self._req_data
        self.request.data = <void*>self
                            ^
------------------------------------------------------------

uvloop/handles/pipe.pyx:195:28: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        # XXX: This is a workaround for a libuv bug:
        # - https://github.com/libuv/libuv/issues/1933
        # - https://github.com/libuv/libuv/pull/551
        if self._handle is NULL:
            return
        self._handle.data = NULL
                            ^
------------------------------------------------------------

uvloop/handles/process.pyx:21:28: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
            raise MemoryError()

        # Too early to call _finish_init, but still a lot of work to do.
        # Let's set handle.data to NULL, so in case something goes wrong,
        # callbacks have a chance to avoid casting *something* into UVHandle.
        self._handle.data = NULL
                            ^
------------------------------------------------------------

uvloop/handles/process.pyx:47:28: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
                       _stdin, _stdout, _stderr):

        memset(&self.options, 0, sizeof(uv.uv_process_options_t))

        self._init_env(env)
        self.options.env = self.uv_opt_env
                               ^
------------------------------------------------------------

uvloop/handles/process.pyx:240:31: Cannot convert 'char **' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        self._init_env(env)
        self.options.env = self.uv_opt_env

        self._init_args(args)
        self.options.file = self.uv_opt_file
        self.options.args = self.uv_opt_args
                                ^
------------------------------------------------------------

uvloop/handles/process.pyx:244:32: Cannot convert 'char **' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
                raise ValueError('cwd must be a str or bytes object')

            self.__cwd = cwd
            self.options.cwd = PyBytes_AsString(self.__cwd)

        self.options.exit_cb = &__uvprocess_on_exit_callback
                               ^
------------------------------------------------------------

uvloop/handles/process.pyx:260:31: Cannot convert 'void (*)(uv_process_t *, int64_t, int) noexcept nogil' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
            if self.request is NULL:
                raise RuntimeError(
                    '{}.cancel: .request is NULL'.format(
                        self.__class__.__name__))

            if self.request.data is NULL:
                                 ^
------------------------------------------------------------

uvloop/request.pyx:38:33: Invalid types for 'is' (Python object, void *)

Error compiling Cython file:
------------------------------------------------------------
...
        self.hints.ai_socktype = type
        self.hints.ai_protocol = proto

        self.request = <uv.uv_req_t*> &self._req_data
        self.callback = callback
        self.request.data = <void*>self
                            ^
------------------------------------------------------------

uvloop/dns.pyx:374:28: Cannot convert 'void *' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        uv.uv_getnameinfo_t _req_data

    def __cinit__(self, Loop loop, callback):
        self.request = <uv.uv_req_t*> &self._req_data
        self.callback = callback
        self.request.data = <void*>self
                            ^
------------------------------------------------------------

uvloop/dns.pyx:396:28: Cannot convert 'void *' to Python object
Compiling uvloop/loop.pyx because it changed.
[1/1] Cythonizing uvloop/loop.pyx
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
    main()
  File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
    return _build_backend().build_wheel(wheel_directory, config_settings,
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 416, in build_wheel
    return self._build_with_temp_dir(['bdist_wheel'], '.whl',
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
    self.run_setup()
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 487, in run_setup
    super(_BuildMetaLegacyBackend,
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in run_setup
    exec(code, locals())
  File "<string>", line 270, in <module>
  File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 107, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 349, in run
    self.run_command("build")
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/setuptools/command/build_py.py", line 62, in run
    self.build_package_data()
  File "/usr/lib/python3.8/site-packages/setuptools/command/build_py.py", line 158, in build_package_data
    for target, srcfile in self._get_package_data_output_mapping():
  File "/usr/lib/python3.8/site-packages/setuptools/command/build_py.py", line 150, in _get_package_data_output_mapping
    for package, src_dir, build_dir, filenames in self.data_files:
  File "/usr/lib/python3.8/site-packages/setuptools/command/build_py.py", line 71, in __getattr__
    self.data_files = self._get_data_files()
  File "/usr/lib/python3.8/site-packages/setuptools/command/build_py.py", line 83, in _get_data_files
    self.analyze_manifest()
  File "/usr/lib/python3.8/site-packages/setuptools/command/build_py.py", line 180, in analyze_manifest
    self.run_command('egg_info')
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 314, in run
    self.find_sources()
  File "/usr/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
    mm.run()
  File "/usr/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 551, in run
    self.add_defaults()
  File "/usr/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
    sdist.add_defaults(self)
  File "/usr/lib/python3.8/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
    super().add_defaults()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
    self._add_defaults_ext()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 335, in _add_defaults_ext
    build_ext = self.get_finalized_command('build_ext')
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 305, in get_finalized_command
    cmd_obj.ensure_finalized()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
    self.finalize_options()
  File "<string>", line 177, in finalize_options
  File "/usr/lib64/python3.8/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
    cythonize_one(*args)
  File "/usr/lib64/python3.8/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: uvloop/loop.pyx

ERROR Backend subprocess exited when trying to invoke build_wheel

@matusvalo
Copy link
Author

After opening RFE with upgrade cython 3.x support (#558) I found this PR. I made quick test and looks like build fails with this PR (in build env with cython 3.0.2) In build output is possible to see use relative imports which using in case of modules with DSOs always is nothing more asking for troubled because using that forces to build DSOs in the source tree and in trashes "test as installed" methodology (install generated .whl in </install/prefix> and point $PYTHONPATH python sitearch/sitelib paths to inside </install/prefix> paths)

IMO it would be good first remove relative imports.

This seems to be a bug in cython. Issue was created.

@matusvalo
Copy link
Author

This seems to be a bug in cython. Issue was created.

The issue is fixed in master and will be part of Cython 3.0.3

@g-as
Copy link

g-as commented Oct 5, 2023

Cython 3.0.3 is out.

https://pypi.org/project/Cython/

@kloczek
Copy link

kloczek commented Oct 5, 2023

Just tested this PR with cython 3.0.3. Here is the result

+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
* Building wheel...
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-38
creating build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/__init__.py -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/_noop.py -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/_testbase.py -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/_version.py -> build/lib.linux-x86_64-cpython-38/uvloop
running egg_info
creating uvloop.egg-info
writing uvloop.egg-info/PKG-INFO
writing dependency_links to uvloop.egg-info/dependency_links.txt
writing requirements to uvloop.egg-info/requires.txt
writing top-level names to uvloop.egg-info/top_level.txt
writing manifest file 'uvloop.egg-info/SOURCES.txt'
Compiling uvloop/loop.pyx because it changed.
[1/1] Cythonizing uvloop/loop.pyx
reading manifest file 'uvloop.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
<string>:144: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
warning: uvloop/handles/udp.pyx:351:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: no files found matching '*' under directory 'vendor/libuv'
warning: no previously-included files matching '*' found under directory 'vendor/libuv/.git'
warning: no previously-included files matching '*' found under directory 'vendor/libuv/docs'
warning: no previously-included files matching '*' found under directory 'vendor/libuv/img'
adding license file 'LICENSE-APACHE'
adding license file 'LICENSE-MIT'
writing manifest file 'uvloop.egg-info/SOURCES.txt'
copying uvloop/cbhandles.pxd -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/cbhandles.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/dns.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/errors.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/loop.c -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/loop.pxd -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/loop.pyi -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/loop.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/lru.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/pseudosock.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/py.typed -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/request.pxd -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/request.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/server.pxd -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/server.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/sslproto.pxd -> build/lib.linux-x86_64-cpython-38/uvloop
copying uvloop/sslproto.pyx -> build/lib.linux-x86_64-cpython-38/uvloop
creating build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/async_.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/async_.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/basetransport.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/basetransport.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/check.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/check.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/fsevent.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/fsevent.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/handle.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/handle.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/idle.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/idle.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/pipe.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/pipe.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/poll.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/poll.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/process.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/process.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/stream.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/stream.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/streamserver.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/streamserver.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/tcp.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/tcp.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/timer.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/timer.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/udp.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/handles
copying uvloop/handles/udp.pyx -> build/lib.linux-x86_64-cpython-38/uvloop/handles
creating build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/__init__.py -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/compat.h -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/consts.pxi -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/debug.h -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/debug.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/flowcontrol.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/fork_handler.h -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/python.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/stdlib.pxi -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/system.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/includes
copying uvloop/includes/uv.pxd -> build/lib.linux-x86_64-cpython-38/uvloop/includes
running build_ext
building 'uvloop.loop' extension
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/uvloop
/usr/bin/gcc -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -fno-semantic-interposition -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -fno-semantic-interposition -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -fno-semantic-interposition -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -fPIC -I/usr/include/python3.8 -c uvloop/loop.c -o build/temp.linux-x86_64-cpython-38/uvloop/loop.o ""
gcc: warning: : linker input file unused because linking not done
gcc: error: : linker input file not found: No such file or directory
/usr/lib/python3.8/site-packages/setuptools/command/build_py.py:204: _Warning: Package 'uvloop.handles' is absent from the `packages` configuration.
!!

        ********************************************************************************
        ############################
        # Package would be ignored #
        ############################
        Python recognizes 'uvloop.handles' as an importable package[^1],
        but it is absent from setuptools' `packages` configuration.

        This leads to an ambiguous overall configuration. If you want to distribute this
        package, please make sure that 'uvloop.handles' is explicitly added
        to the `packages` configuration field.

        Alternatively, you can also rely on setuptools' discovery methods
        (for example by using `find_namespace_packages(...)`/`find_namespace:`
        instead of `find_packages(...)`/`find:`).

        You can read more about "package discovery" on setuptools documentation page:

        - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

        If you don't want 'uvloop.handles' to be distributed and are
        already explicitly excluding 'uvloop.handles' via
        `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
        you can try to use `exclude_package_data`, or `include-package-data=False` in
        combination with a more fine grained `package-data` configuration.

        You can read more about "package data files" on setuptools documentation page:

        - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


        [^1]: For Python, any directory (with suitable naming) can be imported,
              even if it does not contain any `.py` files.
              On the other hand, currently there is no concept of package data
              directory, all directories are treated like packages.
        ********************************************************************************

!!
  check.warn(importable)
/usr/lib/python3.8/site-packages/setuptools/command/build_py.py:204: _Warning: Package 'uvloop.includes' is absent from the `packages` configuration.
!!

        ********************************************************************************
        ############################
        # Package would be ignored #
        ############################
        Python recognizes 'uvloop.includes' as an importable package[^1],
        but it is absent from setuptools' `packages` configuration.

        This leads to an ambiguous overall configuration. If you want to distribute this
        package, please make sure that 'uvloop.includes' is explicitly added
        to the `packages` configuration field.

        Alternatively, you can also rely on setuptools' discovery methods
        (for example by using `find_namespace_packages(...)`/`find_namespace:`
        instead of `find_packages(...)`/`find:`).

        You can read more about "package discovery" on setuptools documentation page:

        - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

        If you don't want 'uvloop.includes' to be distributed and are
        already explicitly excluding 'uvloop.includes' via
        `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
        you can try to use `exclude_package_data`, or `include-package-data=False` in
        combination with a more fine grained `package-data` configuration.

        You can read more about "package data files" on setuptools documentation page:

        - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


        [^1]: For Python, any directory (with suitable naming) can be imported,
              even if it does not contain any `.py` files.
              On the other hand, currently there is no concept of package data
              directory, all directories are treated like packages.
        ********************************************************************************

!!
  check.warn(importable)
error: command '/usr/bin/gcc' failed with exit code 1

ERROR Backend subprocess exited when trying to invoke build_wheel

@robd003
Copy link

robd003 commented Oct 6, 2023

Any chance we could get some eyes on this? @1st1 @fantix

Would love to fix some downstream packages that depend on uvloop

@kloczek
Copy link

kloczek commented Oct 13, 2023

Looks like it is not possible to apply this PR on top of 0.18.0

@matusvalo
Copy link
Author

Looks like it is not possible to apply this PR on top of 0.18.0

Yes this PR was re-implemented in: #570

@matusvalo matusvalo closed this Oct 14, 2023
@matusvalo matusvalo deleted the cython3 branch October 14, 2023 06:01
@kloczek
Copy link

kloczek commented Oct 14, 2023

Are you sure that it is #570 ? 🤔
Looks like this PR already is part of last f 0.18.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants