Nogil #116
Nogil #116
Conversation
|
LGTM |
|
I'll use the zeromq approach and just merge. I had a brief (not too detailed look) and it looks good to me. Merging it means it's in master, if you find any issues or amendments, just make another pull-request and I'll merge that too! |
| { | ||
| int gilstate, old_gilstate; | ||
|
|
||
| if (!PyArg_ParseTuple(args, "i:gilstate", &gilstate)) |
There was a problem hiding this comment.
Question: is this all that is required to release the GIL?
| @@ -189,15 +209,38 @@ compress_helper(void * input, size_t nbytes, size_t typesize, | |||
| } | |||
|
|
|||
| /* Compress */ | |||
| cbytes = blosc_compress(clevel, shuffle, typesize, nbytes, | |||
| input, PyBytes_AS_STRING(output), | |||
| // RAM: I don't think this macro requires the Python interpreter but let's leave it outside | |||
There was a problem hiding this comment.
You could remove the RAM comments and persist your thoughts. We can always use git blame to figure out who wrote the comments. This is valid for all your comments.
|
Note: this reminds me a little of the bcolz commit that releases the GIL. |
|
Also, a pull-request that updates the docs would be nice. |
|
Ok I'll work on an update to the docs and then revert my initials in the comments sometime soon, thanks. |
|
Cool, just make more pull-requests and I'll happily merge them. |
|
Thanks for working on this @robbmcleod . There are many others who will benefit :) |
|
No problem @mrocklin |
Permits release of Python Global Interpreter Lock (GIL) inside of c-blosc calls. Tested with bench/threadpool.py using the multiprocesing.ThreadPool class to map the operations in compressing frames of an image stack (aka movie). Intended to be used in situations where other bounds (e.g. file I/O, network I/O) could be significant, such that asychronous read/write/compress operations could make the compression step essentially free.
Usage:
blosc.set_releasegil(True)
Operate multiple blosc streams at a time