expose libuv uv_fs_event functionality #474

jensbjorgensen · 2022-06-15T20:59:17Z

It's quite natural for filesystem monitoring to be part of the event loop, and libuv has done a cross-platform implementation. Actually asyncio should probably expose this as well--maybe they will draw from this example implementation to inform their own api?

I wasn't sure what the best way to expose the FS_EVENT_* constants was--I preferred to have them refer directly to the constants defined in libuv but I'm a cython newb and I couldn't get the right incantation. In the merge below they are hardcoded in uvloop.const. Maybe you can suggest a better way.

jensbjorgensen · 2022-06-15T21:45:15Z

ok, looks like some items for me to look at. the specific tests I added passed but I see there are formatting issues, sorry I hadn't run those, will update

jensbjorgensen · 2022-06-15T21:59:26Z

the tests I've added are passing, seems like the test suite just takes too long to run for some reason?

jensbjorgensen · 2022-06-15T22:02:02Z

the failures in 3.9 are apparently due to asyncio.events._TransProtPair not being defined in python 3.9

…ne the tuple type directly

jensbjorgensen · 2022-06-21T20:24:44Z

ok! I got test_sourcecode.py fixed up (which I didn't break but ... bonus!), that'd be awesome if one of you maintainers could take a look

fantix

Thanks for adding a new feature!

.gitignore

uvloop/loop.pyx

uvloop/handles/fsevent.pxd

uvloop/const.py

uvloop/loop.pyx

jensbjorgensen · 2022-07-18T17:17:26Z

please have a look at my latest push and thanks for your time reviewing the PR :-)

…rted as sharedlib c function

jensbjorgensen · 2022-07-19T22:47:22Z

I think everything is lined up now with your comments? Let me know if there's anything else.

jensbjorgensen · 2022-07-27T00:29:52Z

ping-ping?

fantix · 2022-08-13T18:01:02Z

Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the get_uv_loop_ptr() API accepted (#310) for easier integration with the libuv loop.

jensbjorgensen · 2022-08-15T14:41:28Z

Hey thanks for getting back to me. Understood, if we can get access to the underlying uv_loop_t we can use the functionality from there. I'll pull from git and take a look!

On 8/13/22 1:01 PM, Fantix King wrote: Hey sorry for the delay! I talked to Yury and I'm convinced that this new API could have arguably different designs, and it's preferred to be a user-space extension of uvloop, while uvloop is targeting to be a drop-in replacement for asyncio itself only. With that said, we could still have the |get_uv_loop_ptr()| API accepted for easier integration with the libuv loop. — Reply to this email directly, view it on GitHub <#474 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNVZ3EX5JVUMTKGRJVYGS3VY7PGRANCNFSM5Y4UF4GQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Jens B. Jorgensen ***@***.***

fantix · 2022-09-01T17:32:33Z

@jensbjorgensen We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implemented recursive flag, and merge the rest as it is.

Please feel free to let me know if you have any comments, or create a new PR if this is already merged.

jensbjorgensen · 2022-09-01T17:48:43Z

Awesome, great news! No remaining comments or anything from my side. Looking forward to it.

On 9/1/22 12:32 PM, Fantix King wrote: @jensbjorgensen <https://github.com/jensbjorgensen> We changed our mind - let's move forward with this PR! I'll probably just push a few commits to make the API private, drop the partially-implemented <libuv/libuv#1778> |recursive| flag, and merge the rest as it is. Please feel free to let me know if you have any comments, or create a new PR if this is already merged. — Reply to this email directly, view it on GitHub <#474 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNVZ3F5TPGWGALPWEMJCXDV4DSDZANCNFSM5Y4UF4GQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Jens B. Jorgensen ***@***.***

* Move const into fsevent.pyx and import with uvloop.loop * Use libuv const to replace the redeclaration * Hide the _monitor_fs API * Duck-typing the asyncio.Handle API * Dropped recursive flag which is not functional on Linux * Proper UVHandle start and stop for fs_event * Callback with context * Errors in callback won't stop the watch now * Dropped the libuv loop pointer (already implemented in other PR)

This happens when executing chmod on a subdir on Linux

tests/test_fs_event.py

uvloop/loop.pyx

@pranavtbhat

This release adds Python 3.11 support, updates bundled libuv to 1.43.0 and fixes a handful of issues. Changes ======= * Expose uv_loop_t pointer for integration with other C-extensions (#310) (by @pranavtbhat in b332eb8 for #310) * Support python 3.11+ (#473) (by @zeroday0619 in 8e42921 for #473) * Expose libuv uv_fs_event functionality (#474) (by @jensbjorgensen @fantix in 74d381e for #474) * Activate debug mode when `-X dev` is used (by @jack1142 in 637a77a) * Expose uv_version() for libuv API compatibility (#491) (by @fantix in 089f6cb for #491) * Fix loop.getaddrinfo() and tests (#495) (by @fantix in 598b16f for #495) * Bump to libuv 1.43.0 (by @fantix in 94e5e53) Fixes ===== * _TransProtPair is no longer defined in asyncio.events (by @jensbjorgensen in fae5f7f) * use a TypeVar for asyncio.BaseProtocol (#478) (by @graingert in 3aacb35 for #478) * Fix segfault in TimerHandle.when() after cleared (by @jensbjorgensen in c39afff for #469) * Avoid self._errpipe_write double close (#466) (by @graingert in 72140d7 for #466) * Fix typo in test (#456) (by @kianmeng in 033d52d for #456) * Fix potential infinite loop (#446) (by @kfur in ada43c0 for #446) * use a stack of self._fds_to_close to prevent double closes (#481) (by @graingert in 3214cf6 for #481) * Fix incorrect main thread id value forking from a thread (#453) (by @horpto @fantix in e7934c8 for #453) * create_subprocess_exec should treat env={} as empty environment (#439) (#454) (by @byllyfish in e04637e for #439) * Queue write only after processing all buffers (#445) (by @jakirkham @fantix in 9c6ecb6 for #445) * Drop Python 3.6 support for thread ident (by @fantix in 9c37930) * bugfix: write to another transport in resume_writing() fails (#498) (by @fantix in d2deffe for #498) Build ===== * Upgrade GitHub Actions (#477) (#480) (by @cclauss in fcbf422 for #477, 1008694 for #480) * typo `same as same` (by @YoSTEALTH in fedba80) * setup.py: allow to override extra_compile_args (#443) (by @giuliobenetti in a130375 for #443) * Drop hack in setup.py in finalize_options (492) (by @fantix in 2f1bc83 for #492) * Fix tests invocation on release CI worklow (#489) (by @ben9923 in d6a2b59 for #489) Documentation ============= * use asyncio.Runner loop_factory on 3.11+ (#472) (by @graingert in 31ba48c for #472) * Fix CI badge in docs, remove remaining Travis CI references from docs (by @Nothing4You in c6901a7) * Fix typo in README (by @monosans in 73d7253)

@pranavtbhat

This release adds Python 3.11 support, updates bundled libuv to 1.43.0 and fixes a handful of issues. Changes ======= * Expose uv_loop_t pointer for integration with other C-extensions (#310) (by @pranavtbhat in b332eb8 for #310) * Support python 3.11+ (#473) (by @zeroday0619 in 8e42921 for #473) * Expose libuv uv_fs_event functionality (#474) (by @jensbjorgensen @fantix in 74d381e for #474) * Activate debug mode when `-X dev` is used (by @jack1142 in 637a77a) * Expose uv_version() for libuv API compatibility (#491) (by @fantix in 089f6cb for #491) * Fix loop.getaddrinfo() and tests (#495) (by @fantix in 598b16f for #495) * Bump to libuv 1.43.0 (by @fantix in 94e5e53) Fixes ===== * _TransProtPair is no longer defined in asyncio.events (by @jensbjorgensen in fae5f7f) * use a TypeVar for asyncio.BaseProtocol (#478) (by @graingert in 3aacb35 for #478) * Fix segfault in TimerHandle.when() after cleared (by @jensbjorgensen in c39afff for #469) * Avoid self._errpipe_write double close (#466) (by @graingert in 72140d7 for #466) * Fix typo in test (#456) (by @kianmeng in 033d52d for #456) * Fix potential infinite loop (#446) (by @kfur in ada43c0 for #446) * use a stack of self._fds_to_close to prevent double closes (#481) (by @graingert in 3214cf6 for #481) * Fix incorrect main thread id value forking from a thread (#453) (by @horpto @fantix in e7934c8 for #453) * create_subprocess_exec should treat env={} as empty environment (#439) (#454) (by @byllyfish in e04637e for #439) * Queue write only after processing all buffers (#445) (by @jakirkham @fantix in 9c6ecb6 for #445) * Drop Python 3.6 support for thread ident (by @fantix in 9c37930) * bugfix: write to another transport in resume_writing() fails (#498) (by @fantix in d2deffe for #498) Build ===== * Upgrade GitHub Actions (#477) (#480) (by @cclauss in fcbf422 for #477, 1008694 for #480) * typo `same as same` (by @YoSTEALTH in fedba80) * setup.py: allow to override extra_compile_args (#443) (by @giuliobenetti in a130375 for #443) * Drop hack in setup.py in finalize_options (492) (by @fantix in 2f1bc83 for #492) * Fix tests invocation on release CI worklow (#489) (by @ben9923 in d6a2b59 for #489) Documentation ============= * use asyncio.Runner loop_factory on 3.11+ (#472) (by @graingert in 31ba48c for #472) * Fix CI badge in docs, remove remaining Travis CI references from docs (by @Nothing4You in c6901a7) * Fix typo in README (by @monosans in 73d7253)

Jens Jorgensen added 3 commits June 15, 2022 10:51

first set of changes

0d514b5

finished testing functionality of fs_events

44b9d3e

remove file I was using for testing

93fa1b3

correct formatting etc

9fb994c

newer Python versions do not have asyncio.events._TransProtPair, defi…

39b2fed

…ne the tuple type directly

fantix requested changes Jul 16, 2022

View reviewed changes

jensbjorgensen added 2 commits July 18, 2022 10:43

Merge remote-tracking branch 'magicstack/master'

2aaa9a6

various changes per maintainer following review

950d8be

jensbjorgensen added 5 commits July 18, 2022 12:23

dang, forgot test_sourcecode.py... fixed

97aabcf

recursive param in test should be boolean, not 0

de85d80

move function to get underlying loop pointer outside the object, expo…

48f3a26

…rted as sharedlib c function

since type is assumed int, can certainly be negative

4c20dd9

fix indentation

55d5ec9

jensbjorgensen requested a review from fantix July 28, 2022 12:34

fantix mentioned this pull request Aug 13, 2022

Expose uv_loop_t pointer for integration with other C-extensions #310

Merged

fantix mentioned this pull request Aug 28, 2022

Reload TLS certificate files when they change edgedb/edgedb#4290

Closed

Merge branch 'master' into master

6807674

fantix added 2 commits September 5, 2022 12:24

Allow both RENAME and CHANGE

91def83

This happens when executing chmod on a subdir on Linux

Merge branch 'master' into master

9148731

1st1 reviewed Sep 6, 2022

View reviewed changes

tests/test_fs_event.py Outdated Show resolved Hide resolved

1st1 reviewed Sep 6, 2022

View reviewed changes

uvloop/loop.pyx Show resolved Hide resolved

1st1 approved these changes Sep 6, 2022

View reviewed changes

fantix added 2 commits September 9, 2022 09:29

CRF: optimize tests

fe77faa

Update pyOpenSSL to 22.x

5658110

fantix approved these changes Sep 9, 2022

View reviewed changes

fantix merged commit 74d381e into MagicStack:master Sep 9, 2022

fantix mentioned this pull request Sep 13, 2022

uvloop v0.17.0 #501

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

expose libuv uv_fs_event functionality #474

expose libuv uv_fs_event functionality #474

jensbjorgensen commented Jun 15, 2022

jensbjorgensen commented Jun 15, 2022

jensbjorgensen commented Jun 15, 2022

jensbjorgensen commented Jun 15, 2022

jensbjorgensen commented Jun 21, 2022

fantix left a comment

jensbjorgensen commented Jul 18, 2022

jensbjorgensen commented Jul 19, 2022

jensbjorgensen commented Jul 27, 2022 •

edited

fantix commented Aug 13, 2022 •

edited

jensbjorgensen commented Aug 15, 2022 via email

fantix commented Sep 1, 2022

jensbjorgensen commented Sep 1, 2022 via email

expose libuv uv_fs_event functionality #474

expose libuv uv_fs_event functionality #474

Conversation

jensbjorgensen commented Jun 15, 2022

jensbjorgensen commented Jun 15, 2022

jensbjorgensen commented Jun 15, 2022

jensbjorgensen commented Jun 15, 2022

jensbjorgensen commented Jun 21, 2022

fantix left a comment

Choose a reason for hiding this comment

jensbjorgensen commented Jul 18, 2022

jensbjorgensen commented Jul 19, 2022

jensbjorgensen commented Jul 27, 2022 • edited

fantix commented Aug 13, 2022 • edited

jensbjorgensen commented Aug 15, 2022 via email

fantix commented Sep 1, 2022

jensbjorgensen commented Sep 1, 2022 via email

jensbjorgensen commented Jul 27, 2022 •

edited

fantix commented Aug 13, 2022 •

edited