-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Windows support #524
Comments
wip. planned for R18. |
Some update on that:
TODO: handle service start/stop/restart to use gunicorn as a full windows service. It is replacing somehow the use of signals under unix. |
the support will finally happen in R20. i will create the new PR to track the development of this feature next week. Things to fix are:
|
Slow progress. I want to release R19 with the threaded work now. I will push a version on monday. the windows support will arrive right after. |
Is this windows supported version available somewhere ? |
Windows support will be in R20, so I guess it will be a while till guincorn will have windows support... I created a vm with Ubuntu, and this works for me..... |
Just curious as to any progress in this. As a first pass it doesn't even need to be fast; a big advantage will be just being able to develop with Gunicorn on Windows and push to a Linux server (e.g. Heroku) without changing config. |
@robertlagrant it's on my todo. I need to find a windows license first. R20 will be released in october, |
How about the VM images at modern.ie ? Those are free and legal
|
I used modern.ie images to test a web app in IE8 recently. Easy to get going and worked great. |
@benoitc is there a WIP branch for this? I'm not a Windows user, but I can take a look at this. |
@berkerpeksag no not yet :/ we should really start one asap. |
like @cyberdelia said we could ask a license of windows at the PSF that the gunicorn project could use for such development. Thoughts? |
It's a good idea. https://www.python.org/psf/grants/ has all information about grant proposals. |
Any updates on this? |
I am also looking for updates to this. |
@berkerpeksag let's have a talk asap on IRC, just ping me sometimes before since i'm away from any instant thing these days :) |
Is there a rough idea how it could be done on Windows to support proper multiprocessing? |
Could the Win10 dev VM's be a solution for you to run Windows and check gunicorn compatibility? |
Perhaps we can use |
I've written #2636 which fixes about half of the problems porting gunicorn to Windows. The other half requires finding a Windows-compatible alternative to using
|
I would admit that windows is a really low priority for me. Gunicorn is
done first to be deployed on POSIX/UNIX platforms. That’s a desirable goal
anyway :)
But in any case we need a more native way to handle it close to windows
libraries/system rather than using anything from multiprocessing. So we
can understand how it works and optimise when needed without depending on a
third party. This should work until Python stops to expose low level
libraries.
What we need is a way to share a socket between threads on windows and
basic signaling. I think it would be good to list common patterns in
windows and pick one for our implementation. Let’s start by doing this and
we will see how it can be handled our way?
Le sam. 28 août 2021 à 09:34, Bryan A. Jones ***@***.***> a
écrit :
I've written #2636 <#2636> which
fixes about half of the problems porting gunicorn to Windows. The other
half requires finding a Windows-compatible alternative to using os.pipe.
Could we simply use a multiprocessing listener and client
<https://docs.python.org/3/library/multiprocessing.html#module-multiprocessing.connection>
instead of the existing os.pipe?
- It doesn't require using any other part of multiprocessing, so it
avoid that baggage/overhead.
- It produces a Connection
<https://docs.python.org/3/library/multiprocessing.html#multiprocessing.connection.Connection>
object, which allows sending raw bytes for efficiency.
- It has a select-like function
<https://docs.python.org/3/library/multiprocessing.html#multiprocessing.connection.wait>
.
- It's (of course) multi-platform.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#524 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAADRIUEFRFIS3VU6ZQDWBDT7CGPNANCNFSM4AE7ECYA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
Sent from my Mobile
|
@benoitc, thanks for your feedback. Do we need only inter-thread synchronization, or is this actually inter-process synchronization? I assume inter-process, since you mention sockets and inter-thread is well supported already by the Python library. |
You’re right I meant inter-process synchronisation (or system threads) .I
am pretty sure there is a signaling API that could do it.
Le sam. 28 août 2021 à 10:18, Bryan A. Jones ***@***.***> a
écrit :
@benoitc <https://github.com/benoitc>, thanks for your feedback. Do we
need only inter-thread synchronization, or is this actually inter-process
synchronization? I assume inter-process, since you mention sockets and
inter-thread is well supported already by the Python library.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#524 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAADRIWL3L6OPDDFAYA2RMDT7CLW5ANCNFSM4AE7ECYA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
Sent from my Mobile
|
Thanks for taking the time to look at this. I really appreciate it. I know your time is very limited.
I'm a bit confused here. multiprocessing is from the Python standard library, so it's not a third-party library.
That's what I thought, but I wanted to check... |
Le sam. 28 août 2021 à 10:31, Bryan A. Jones ***@***.***> a
écrit :
I would admit that windows is a really low priority for me. Gunicorn is
done first to be deployed on POSIX/UNIX platforms. That’s a desirable goal
anyway :)
Thanks for taking the time to look at this. I really appreciate it. I know
your time is very limited.
But in any case we need a more native way to handle it close to windows
libraries/system rather than using anything from multiprocessing. So we
can understand how it works and optimise when needed without depending on a
third party. This should work until Python stops to expose low level
libraries.
I'm a bit confused here. multiprocessing
<https://docs.python.org/3/library/multiprocessing.html#module-multiprocessing.connection>
is from the Python standard library, so it's not a third-party library.
multiprocessing is an abstraction over systems calls. I would really prefer
we handle directly native calls so we can change any time or use only the
part we need. If really needed we can also create a binding for the one not
exposed.
You’re right I meant inter-process synchronisation (or system threads) .I
am pretty sure there is a signaling API that could do it.
That's what I thought, but I wanted to check...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#524 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAADRIRN6UJAJ45RO43O74DT7CNHJANCNFSM4AE7ECYA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
Sent from my Mobile
|
OK, sounds good. I'll look for signaling / socket IPC mechanisms and let you know what I find. |
I would love to use multiprocessing, if it's appropriate. These standard library abstractions are a great thing about the language, especially because the low level implementation details for CPython are often well specified in the documentation. If multiprocessing, or any other standard module, implements the right windows platform features to do what we need, we should consider using it. If it doesn't, we won't. |
In the past, then I've read multiprocessing docs, I haven't been convinced! But I'd be open to being educated if it does seem appropriate. I'm also open to measuring the impact if it turns out slower but vastly simpler. |
@tilgovi, thanks for your feedback. Unless @benoitc disagrees, I'll pursue an initial implementation using |
well, i hate to repeat myself. I would like to have a clean implementation not based on multiprocessing. It's not about optimisation but also in the process learning and make gunicorn a better citizen in windows. Multiprocessing and gunicorn doesn't share the same philosophy in handling multiprocessing. It's OK however to use low level python bindings, some coming with the multiprocessing. But definitly not higher level. We have to be cautious. And definitly removing anything using shared memory. |
I do not see how something not based on a standard python module but some native library should be the "cleaner" solution? |
I believe good code argues for itself. Would the gunicorn team be willing to look at an initial implementation based only on the inter-process communication components of If not, I can look at a |
there are many reason to not use multiprocessing. One of them is to be able to work with other possible workers and provide a facility that work for all workers. It is also to ensure our maintenance window. Something really hard with multiprocessing. And I also like to not have depends on optinion of others about what is interprocess-communication. My preference is the following:
@bjones1 if you believe the code can be used for example to spawn workiers that include gevent and check their liveness we can go with it. Better to speak over the code anyway. |
This is great. I see a lot of alignment. I'll give feedback on the issue you opened. Nice to see movement on this. I agree with everyone so far. It is good to be in control, avoid high level abstractions that don't do exactly what we need, but also not reinvent the wheel. So we will evaluate deeper, together, as we look at real code possibilities. |
I've started work on this in #2643. Comments welcome. |
Any news here? : ) |
This is related to the things we discussed at PyCon
The text was updated successfully, but these errors were encountered: