Windows: improve usability for low-level IOCP operations

[njsmith]

• Is wait_overlapped the best way to handle "call a function that takes an OVERLAPPED"?
• Make wait_overlapped return the information in OVERLAPPED_ENTRY, like dwNumberOfBytesTransferred
• Document everything

---

Haven't really been thought through, need a good look over

[njsmith]

One definite change is that instead of returning a new UnboundedQueue, these should have you pass in an existing UnboundedQueue. This allows multiple event streams to be multiplexed onto a single queue. (In the Windows case this also means our completion event objects should include the completion key as a field.) See #40.

[auvipy]

is there anything trio could be benefitted from this thread?

[njsmith]

@auvipy Sorry, I don't quite understand. Are you asking whether there's anything you can do to help with this, or...?

[auvipy]

yes, and if there is anything important we get from the problem described here? I am considering trio as an option beside asyncio to be used as the foundational event loop library for celery 5 :)

[auvipy]

so knowing the internals of trio, and its potential API breakage is very important to me for considering trio very seriously :)

[njsmith]

@auvipy ah, I see!

This particular issue is about some very low-level APIs, that are used to get direct access to exotic features of the underlying OS – like if you need to access Windows's IOCP interfaces directly for some reason, or the raw BSD kqueue APIs. They're useful for some kinds of projects, but most people will never touch them. (Asyncio doesn't give you any way to use these APIs at all ;-).) For some more details on what this might look like for kqueue, see #578 and #579. For more background on Windows, see #52. If this still sounds like something you're interested in, I can say more :-). I doubt this will be very relevant to celery though.

About the more general questions about Trio's internals, potential API breakage, and whether Trio would be a good fit for celery: I would love to talk about this! (And the short answer is: it depends on your time frame.) But, I would rather not have that discussion here on this issue, because it will be confusing later :-). I have two ideas, take your pick:

• We just (like, yesterday) created a new discourse forum to talk about Trio, that's available at: https://trio.discourse.group. It's very empty right now, but the plan is to move this kind of discussion over there. Would you be up for posting in the Help and advice area? (You can use your github login to post. I think.)

• Or if you'd rather stay on Github, that'd be fine too... but can you open a new issue?

Thanks!

[auvipy]

thanks a lot for your detailed response :) I will do what seems more appropriate and right place to do :)

[njsmith]

I wrote another brain dump about these here: #937 (comment)

[oremanj]

The kqueue case seems adequately covered by #578 so I'm going to repurpose this issue for just the Windows side.