-
Notifications
You must be signed in to change notification settings - Fork 762
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
win32 support #2
Comments
Multiprocessing module docs: http://docs.python.org/library/multiprocessing.html Shows converting from os.fork to multiprocessing: http://www.ibm.com/developerworks/aix/library/au-multiprocessing/ |
Also, note that I would like to maintain os.fork() as a fallback for the python 2.4 case on Linux/UNIX. |
Ticket #2: #2 - win32 support. The 'resource' module is not available under Windows. We only use it for daemonizing so make it optional and disable daemonizing support on Windows for now. Also, refactor how the optional imports to turn them into data instead of code and iterate through them. Add early warnings to indicate when modules are missing and functionality is disabled.
I made the 'resource' module optional (and refactored how optional modules are imported): commit c659bcb. @ysangkok, please test again. I don't have a Windows system easily accessible at the moment. I'm sure you will run into more issues, so just post them here and we'll work through them iteratively. |
I tried running, but I can't get it to work. I tried running it with the PDB. See the last 10 statements (I added comments).
|
I appreciate the amount of debugging output you have captured. I may have to try and get access to a Windows system to test this since the behavior you are describing is quite odd. I'm not sure pdb is describing everything that is happening because the "if startsock" statement should always be false until the "startsock, address = lsock.accept()" line has been called which doesn't show up in your output. What behavior do you get when you run it without pdb but with the --verbose option? |
BTW I had to symlink websockify to websockify.py or I would get an ImportError:
|
That's strange, apparently the multiprocessing module is doing a new import of the module when spawning the process. Maybe that's normal on Windows, but I wouldn't expect it to throw an exception like that if things are working correctly. Apparently the socket we get from socket.accept is not valid or is being invalidated by the Process invocation. I've not done much socket programming on Windows so this might be a windows specific issue. Let's try eliminating the multiprocessing complexities from the mix and see what happens. This change will only be able to handle a single request at a time, but it should show us whether the problem is related to multiprocessing or not:
BTW, the change above is just calling the top_new_client method directly. |
Yup, works now: http://i.imgur.com/1r7tt.png :D Thanks for making it work. I appreciate it a lot. |
@ysangkok, I'll be on vacation for the next 10 days, but I wonder if you might be willing to track down why the handoff of the socket from parent process to child process is breaking. I had a thought that perhaps the startsock.close() in the parent might be the culprit but I don't have an opportunity to test before leaving. After the client is started, the socket should probably only be closed in the client (even on Linux but for some reason Linux tolerates this). |
@ysangkok, have you had a chance to try the multiprocessing support with the |
I'm working on refactoring websockify to use python's SocketServer module. This module is designed to work with the ForkingMixin and ThreadingMixin. For Linux I will continue to use the forking model (which should be faster because it will avoid the python GIL) and use threading on Windows (since sharing of listener sockets isn't supported across processes in Windows). I have basic proxying already working, and I'm working on all the other functionality that websockify supports. |
Forking on windows doesn't work and Python try to Pickling socket (that can't be pickled). I have the same problem and i resolve with pickling handler instead socket, that work in multiprocess mode (tested on Windows 7 32bit and Python 3.1). This is my quick fix: diff a/websocket.py b/websocket.py: 22a23,24
> from multiprocessing.reduction import reduce_handle
> from multiprocessing.reduction import rebuild_handle
732c734
< def top_new_client(self, startsock, address):
---
> def top_new_client(self, sockfd, address):
740a743,747
> fd = rebuild_handle(sockfd)
> startsock = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP)
>
> self.vmsg('%s: new handler Process' % address[0])
>
786c793
<
---
>
802a810
> client_sock = None
813c821
<
---
>
816c824,826
< startsock, address = lsock.accept()
---
> client_sock, address = lsock.accept()
> startsock = reduce_handle(client_sock.fileno())
>
818a829
>
873,874c884,885
< if startsock:
< startsock.close()
---
> if client_sock:
> client_sock.close()
|
Fascinating. multiprocessing.reduction is not particularly well publicized. Looking at the code in multiprocessing.reduction, I think the module is designed to automatically update the pickler to handle sockets/connections. The reduce_handle and rebuild_handle functions are not exported so they aren't really designed to be called externally. They are really routines internal to the module that are used for overloading the pickler. So, I think all that is necessary is the following patch:
It's curious to me that multiprocessing doesn't do that import automatically on Windows. Anyways, can those listening on this bug please test my simple patch on Windows and see if it does in fact work correctly? If you can, please test multiple simultaneous clients. Hopefully we have both python 2.6 and python 3.0 Windows users listening to this bug. Thanks |
@tdski82, thanks for the great catch BTW. I was starting to rewrite websockify to use python SocketServer with threading on Windows and forking on Linux. If this multiprocessing.reduction idea you found works, that will be much cleaner (and should be faster on Windows) and save me a bunch of work. So thanks! |
Also for testers, please test this with wss/SSL connections. |
It's enough only the import patch that you wrote, BTW i think that MultiThreading is best choise for the future. I read that the fix for handler pickling has been implemented from Python 3.1. This morning i try it behind noVNC and with 4 concurrent and active connection and work fine (sometime i see an Exception in output but i don't have dump of this error) Python version tested with new pickling fix (clean Python installation):
Fix for Python 3.01 or grater without socket pickling (less version doesn't have fromfd method on socket on Windows environment): # pass descriptor to top_new_client (without socket pickling)
self.top_new_client(startsock.fileno(), address)
# change parameter on method
def top_new_client(self, startsockfd, address):
# recreate socket by passed descriptor ( in top_new_client(....) )
startsock = socket.fromfd(startsockfd, socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP) |
I'm doing some testing to see if I can get websockify to replace another product we are using on windows, and after I did the symlink websockify.py to websockify trick, it looked like everything was going to run well. The browser is making the connection to the socket, and sending its data, however, the message never gets to the intended server. Here are the messages I get in verbose mode:
And that is all that I get. The warning that there is no base64 followed by base64: 'True' seems odd, but I don't know that t is actually an issue. I am not using the js file that is shipped with websockify at this time, I'm testing with our existing websocket code. |
Currently, if the client doesn't report that it supports raw binary (via the 'binary' in the websocket sub-protocol) then websockify assumes that it must base64 encode/decode any traffic to/from the browser. It's warning you that the client didn't report anything in the sub-protocol but that it is going ahead and using base64 encoding anyway. I might suggest that you try running the echo test. Run this: Then browse to |
I had to create another symbolic link from websockify/test/include to websockify/include and then the echo test worked. I've modified my script to send/receive base64 encoded data, but I'm still not seeing the packet get to the final server. Is there a better logging method than verbose where I could see that data that is passed across? |
I take it back. The message is getting through, and has the correct number of bytes, but they are all 0. |
Try this for really verbose logging:
With the echo test you would see something like this (if using a browser using HyBi-76 such as Chrome 13):
If you still are seeing a problem after adding that, post the debug you get here and I'll take a look. |
Thanks for the help code. I pretty quickly realized I was being dumb and missed the part in echo where it packed the string into an integer array before running the base64 encode. So, to summarize, I was able to get websockify running on Windows by using Python 3.2 with a symlink from websockify.py to websockify. Also, the tests required an additional symlink from websockify/test/include to websockify/include. |
Hi, I've tried to put echo.py to work but I'm getting the error: WARNING: no 'resource' module, daemonizing is slower or disabled Do you have any thoughts on what might be going on? |
I think @snorkeyg has gotten websockify to work with python 2.7 on Windows. I'll ask him to weigh in. In the meantime, if you scroll back a few comments to the test results from @tdski82, it indicates that python 3.1 and 3.2 should work fine and that python 2.X and python 3.0 do not work due to the lack of some Windows related fixes in the multiprocessing module. |
Hi @hugoslv. I have websockify working well on Windows. I have applied @kanaka's patch as per comment by @kanaka on May 20, 2011 in this thread to fix the error you are having above. I'm not actually sure if windows has a patch tool and I'm more familiar with this kind of thing in Unix so I actually applied the patch in linux, copied it all to windows box and it ran sweet. If you want I can just send you patched copy of this file? I had almost forgotten about this issue, I have been meaning to get back to this and see if I can have crack at proper multiprocessing on Windows. |
Also to clarify I was using Python 2.7. Not sure if Python version makes to much difference once the patch is applied but, that's a possibility. |
Thank's Guys, I'll try the suggested fix and get back to you in case the problems persist. Best regards, On Nov 7, 2011, at 11:30 PM, Chris Gordon wrote:
|
@hugoslv, just to be clear, the python 2.7 "fix" disables multi-processing in websockify. What this means is that you lose the ability to connect more than one client to the server through the same websockify instance. If you run websockify with python 3.1 or 3.2 then it should work on windows AND have multiprocessing support. |
Okay, that's probably sufficient. I'll catch modifications to websockify.py in any pull requests and have the devs using Windows re-submit pulls using websockify instead. |
hi guys, i'm still having issues.. I created a shortcut in the same folder named websockify.py (checked properties, it's named right) still get same issue as above from this:
Traceback (most recent call last): Next I tried just removing the shortcut and copying the websockify file directly to websockify.py file. Then i get a different error when i attempt to connect:
WARNING: no 'resource' module, daemonizing is slower or disabled it allows me to connect (from raw telnet to port 8080 on a different host) and then exits. |
also to get the above (very short) connection, i also had to remove the call to python in the websockify.py file (#!/usr/bin/env python ) |
"Ignoring socket not ready" means that when the server went to process the handshake, there was no data on the socket channel. You can try increasing the timeout from 3 to 100 (ms) in do_handshake:
There really shouldn't be any delay I don't think, but the way multiprocessing works on Windows could be introducing some delay somehow. |
with that change to increasing the timeout, i get this response now : 10: handler exception: file descriptor cannot be a negative integer (-1) when i telnet to localhost on 8080 while running and it closes the connection again after a few seconds. |
If it's working for someone else, can you please post the exact version of novnc and the exact version of python you are running? I'm testing with kanaka-noVNC-v0.1-64-g2fa565b i'm running windows xp sp3 in a virtualbox vm. |
Hey eph214, i guess i faced the same problem before, i am using python 2.7 with patched version of websockify as i described above. Its working fine for me. |
Hi @vicky555 , I am also using Windows 7 with python 2.7. Trying very hard to get this work. I read the thread about differences between python 2.7 and 3.1, I incorporated the changes for the import multiprocessing.reduction and also the Process call in start_server() function. But still I am getting the same error PS E:\Misc projects\noVNC> python .\utils\websockify.py --web ./ 8787 localhost:5901 --cert .\utils\self.pem
WARNING: no 'numpy' module, HyBi protocol is slower or disabled Actually I am too confused as to exactly what changes to do in my files to get it running for python 2.7. Can you help me out in this regard ? It would be very helpful if you could show me the modified websocket.py and websockify.py. |
@agnivade i am using following files which you have mentioned.. |
@vicky555 many thanks. Its working now. |
@agnivade you are welcome.. :) |
@vicky555 can u please share those files again. the links seem to be broken now. thanks in advance :) |
@pushakargaikwad please download files from following ur |
To fix |
Has anyone managed to get it working on Windows? I've tried on Vista and Win7 with python 3.4 and portable python, but I always get this error: ImportError: No module named 'websocket'. Any suggestions? It works fine on linux. |
Same problem here, after compiling websockify as windows executable I can't manage to use it without getting errors (I'm not even sure that the compiled (using portable python 2.7 version will support multiple client connection at a time)) , I also posted a Q' on stackoverflow , Help will be appreciated... |
The solution is to use the Node.js version on Windows, instead of the php version. It works like a charm, out of the box. |
Did anyone managed to run websockify (python) on windows with multiprocessing support (with the ability to connect more than one client to the server through the same websockify instance), if so, please tell me how, Thanks! (can't use Node.js version) |
C:\Users\majian-xy>python E:\noVNC\utils\websockify 7777 10.18.27.57:5900 WARNING: no 'numpy' module, HyBi protocol will be slower
During handling of the above exception, another exception occurred: Traceback (most recent call last): C:\Users\majian-xy>WARNING: no 'numpy' module, HyBi protocol will be slower |
Python 2 was end-of-lifed on January 1. Python 3.8 has complete Windows support for non-blocking TCP & UDP asyncio. Convert websockify to use asyncio and it'll work natively on Windows. |
Windows support was fixed back in #388 so this should have been closed then. |
Add support for win32.
One of the main changes will require using the multiprocessing module rather than os.fork().
The text was updated successfully, but these errors were encountered: