New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
selectors: provide a helper to choose a selector using constraints #63664
Comments
multiprocess, telnetlib (and subprocess in a near future, see bpo-18923) use the following code to select the best selector: # poll/select have the advantage of not requiring any extra file descriptor,
# contrarily to epoll/kqueue (also, they require a single syscall).
if hasattr(selectors, 'PollSelector'):
_TelnetSelector = selectors.PollSelector
else:
_TelnetSelector = selectors.SelectSelector I don't like the principle of "a default selector", selectors.DefaultSelector should be removed in my opinion. I would prefer a function returning the best selector using constraints. Example: def get_selector(use_fd=True) -> BaseSelector:
... By default, it would return the same than the current DefaultSelector. But if you set use_fd=False, the choice would be restricted to select() or poll(). I don't want to duplicate code like telnetlib uses in each module, it's harder to maintain. The selectors module may get new selectors in the future, see for example bpo-18931. Except use_fd, I don't have other ideas of constraints. I read somewhere that differenet selectors may have different limits on the number of file descriptors. I don't know if it's useful to use such constraint? |
What's the use case for not wanting to use an extra FD? Nevertheless I'm fine with using a function to pick the default selector (but it requires some changes to asyncio too, which currently uses DefaultSelector). Something I would find useful would be a way to override the selector choice on the command line. I currently have to build this into the app's arg parser and main(), e.g. http://code.google.com/p/tulip/source/browse/examples/sink.py#64 |
A selector may be used a few millisecond just to check if a socket is ready, and then destroyed. For such use case, select() is maybe enough (1 syscall). Epoll requires more system calls: create the epoll FD, register the socket, poll, destroy the epoll FD (4 syscalls). |
Hm... I'm trying to understand how you're using the selector in I wonder if you could just create the selector when the Telnet class is It still seems to me that this is pretty atypical use of selectors; the If you insist on having a function that prefers poll and select over kqueue |
1 similar comment
Hm... I'm trying to understand how you're using the selector in I wonder if you could just create the selector when the Telnet class is It still seems to me that this is pretty atypical use of selectors; the If you insist on having a function that prefers poll and select over kqueue |
I already implemented something similar to subprocess.Popen.communicate() when I was working on old Python versions without the timeout parameter of communicate(). IMO calling select with a few file descriptors (between 1 and 3) and destroying quickly the "selector" is no a rare use case. If I would port my code to selectors, I don't want to rewrite it to keep the selector alive longer, just because selectors force me to use the super-powerful fast epoll/kqueue selector. (To be honest, I will probably not notice any performance impact. But I like reducing the number of syscalls, not the opposite :-)) |
OK. Let's have a function to select a default selector. Can you think of a |
I prefer to leave the question to the author of the module, Charles-François :-) |
There are actually two reasons to choosing poll over epoll/kqueue
So I agree it would be nice to have a better way to get a selector not The reason I didn't add such a method in the first place is that I
Apart from select(), all other selectors don't have an upper limit. As for the performance profiles, depending on the application usage, To sum up, get_selector(use_fd=True) looks fine to me. |
Hm. If you really are going to create 300 instances, you should probably |
Of course, when I have 300 connections to remote nodes, I use poll() But there are times when you can have a large number of threads |
I think this is more of a documentation issue. People who don't want a new fd can hardcode PollSelector (poll has been POSIX for a long time). |
That's also what I now think. |
It looks you rejected my idea, so I'm in favor of just closing the issue. Do you agree? |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: