Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows: improve usability for low-level IOCP operations #26

Open
njsmith opened this issue Jan 22, 2017 · 9 comments
Open

Windows: improve usability for low-level IOCP operations #26

njsmith opened this issue Jan 22, 2017 · 9 comments

Comments

@njsmith
Copy link
Member

@njsmith njsmith commented Jan 22, 2017

  • Is wait_overlapped the best way to handle "call a function that takes an OVERLAPPED"?
  • Make wait_overlapped return the information in OVERLAPPED_ENTRY, like dwNumberOfBytesTransferred
  • Document everything

Haven't really been thought through, need a good look over

@njsmith njsmith changed the title Design: kqueue "power user" interface Design: kqueue and windows "power user" interfaces Feb 13, 2017
@njsmith
Copy link
Member Author

@njsmith njsmith commented Feb 13, 2017

One definite change is that instead of returning a new UnboundedQueue, these should have you pass in an existing UnboundedQueue. This allows multiple event streams to be multiplexed onto a single queue. (In the Windows case this also means our completion event objects should include the completion key as a field.) See #40.

@auvipy
Copy link

@auvipy auvipy commented Feb 6, 2019

is there anything trio could be benefitted from this thread?

@njsmith
Copy link
Member Author

@njsmith njsmith commented Feb 6, 2019

@auvipy Sorry, I don't quite understand. Are you asking whether there's anything you can do to help with this, or...?

@auvipy
Copy link

@auvipy auvipy commented Feb 6, 2019

yes, and if there is anything important we get from the problem described here? I am considering trio as an option beside asyncio to be used as the foundational event loop library for celery 5 :)

@auvipy
Copy link

@auvipy auvipy commented Feb 6, 2019

so knowing the internals of trio, and its potential API breakage is very important to me for considering trio very seriously :)

@njsmith
Copy link
Member Author

@njsmith njsmith commented Feb 6, 2019

@auvipy ah, I see!

This particular issue is about some very low-level APIs, that are used to get direct access to exotic features of the underlying OS – like if you need to access Windows's IOCP interfaces directly for some reason, or the raw BSD kqueue APIs. They're useful for some kinds of projects, but most people will never touch them. (Asyncio doesn't give you any way to use these APIs at all ;-).) For some more details on what this might look like for kqueue, see #578 and #579. For more background on Windows, see #52. If this still sounds like something you're interested in, I can say more :-). I doubt this will be very relevant to celery though.

About the more general questions about Trio's internals, potential API breakage, and whether Trio would be a good fit for celery: I would love to talk about this! (And the short answer is: it depends on your time frame.) But, I would rather not have that discussion here on this issue, because it will be confusing later :-). I have two ideas, take your pick:

  • We just (like, yesterday) created a new discourse forum to talk about Trio, that's available at: https://trio.discourse.group. It's very empty right now, but the plan is to move this kind of discussion over there. Would you be up for posting in the Help and advice area? (You can use your github login to post. I think.)

  • Or if you'd rather stay on Github, that'd be fine too... but can you open a new issue?

Thanks!

@auvipy
Copy link

@auvipy auvipy commented Feb 6, 2019

thanks a lot for your detailed response :) I will do what seems more appropriate and right place to do :)

@njsmith
Copy link
Member Author

@njsmith njsmith commented Feb 20, 2019

I wrote another brain dump about these here: #937 (comment)

@oremanj oremanj changed the title Design: kqueue and windows "power user" interfaces Windows: should we improve monitor_completion_queue()? Mar 12, 2019
@oremanj
Copy link
Member

@oremanj oremanj commented Mar 12, 2019

The kqueue case seems adequately covered by #578 so I'm going to repurpose this issue for just the Windows side.

@oremanj oremanj added the Windows label Mar 12, 2019
@oremanj oremanj changed the title Windows: should we improve monitor_completion_queue()? Windows: think about usability for IOCP operations Mar 12, 2019
@oremanj oremanj changed the title Windows: think about usability for IOCP operations Windows: improve usability for low-level IOCP operations Mar 12, 2019
@oremanj oremanj added the low-level label May 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants