Skip to content

Latest commit

 

History

History
242 lines (163 loc) · 8.93 KB

py-recv-chunk.rst

File metadata and controls

242 lines (163 loc) · 8.93 KB

Chunking receiver

For an overview, refer to recv-chunk. This page is a reference for the Python API.

Writing a place callback

A callback is needed to determine which chunk each heap belongs to and where it fits into that chunk. The callback is made from a C++ thread, so it cannot be written in pure Python, or even use the Python C API. It needs to be compiled code. The callback code should also not attempt to acquire the Global Interpreter Lock (GIL), as it may lead to a deadlock.

Once you've written the function (see below), it needs to be passed to spead2. The easiest way to do this is with :pyscipy.LowLevelCallable. However, it's not strictly necessary to use scipy. The callback must be represented with a tuple whose first element is a :cPyCapsule. The other elements are not used, but a reference to the tuple is held so this can be used to keep things alive). The capsule's pointer must be a function pointer to the code, the name must be the function signature, and a user-defined pointer may be set as the capsule's context.

For the place callback, the signature must be one of the following (exactly, with no whitespace changes):

  • "void (void *, size_t)"
  • "void (void *, size_t, void *)"

In the latter case, the capsule's context is provided as the final argument. The first two arguments are a pointer to :cppspead2::recv::chunk_place_data and the size of that structure.

There are lots of ways to write compiled code and access the functions from Python: ctypes, cffi, cython, pybind11 are some of the options. One for which spead2 provided some extra support is numba:

spead2.recv.numba.chunk_place_data

spead2.numba.intp_to_voidptr

Reference

spead2.recv.asyncio.ChunkRingbuffer