New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cython class inheritance with python threading #2834

Closed
SarmadEdward opened this Issue Feb 10, 2019 · 3 comments

Comments

Projects
None yet
3 participants
@SarmadEdward
Copy link

SarmadEdward commented Feb 10, 2019

I have this code in python to change to Cython -> to C
PYTHON:
class Foo(threading.Thread):
def init(self, threadid, name, counter):
int x,y,z,v,b,n float some_float

As we know Python threads are threading.Thread (contrary to Cython threads)

*I cannot do cdef class Foo(): because it inherits from threading.Thread which is Python, not the Cython parallelism concept

Inside the init function, I have dozens of variables that I have to convert to cdef

Eventually I cannot just type cdef int x,y,z,...., because of the rule in the language basics ""When subclassing an extension type with a Python class, def methods can override cpdef methods but not cdef methods:""

Is this still under development?
Any approach to get this problem resolved please?
Thanks

@mobiusklein

This comment has been minimized.

Copy link

mobiusklein commented Feb 11, 2019

Instead of subclassing threading.Thread, you should instead define a cpdef function that takes in Python objects as input, converts them to C values, then releases the GIL and does its expensive computational work. Then you pass that function and its arguments as the target and arguments of a threading.Thread object and use it as-is.

import threading

cpdef my_task(int threadid, bytes name, int counter):
  cdef:
      char* cname
      float some_float
  # coerce to C values
  cname = name
  with nogil:
       # do the hard work here

task = threading.Thread(target=my_task, args=(1, b"foo", 0))
task.start()
print(1)

Alternatively, you subclass threading.Thread as a Python class but in it's run method, you have it pass its data to a C function that also releases the GIL.

Cython itself does not have its own threading implementation. It allows you to use OpenMP, a compiler feature, or an external C threading library, which is essentially a feature of your operating system. It exposes the Python C-API for threads as well, but these are not nearly as expressive an abstraction as the Python-level interface, and lack fundamental components like a GIL-less thread-safe collection (e.g. queue.Queue).

@SarmadEdward

This comment has been minimized.

Copy link
Author

SarmadEdward commented Feb 11, 2019

Thanks I will try this approach, it will take some time - heads up.

@scoder

This comment has been minimized.

Copy link
Contributor

scoder commented Feb 11, 2019

Also note that you do not even need a cdef class to compile methods. That works perfectly fine for Python classes as well.

Not a bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment