Spyder hangs while debugging #991
Comments
|
From jed.lud...@gmail.com on 2013-02-09T15:35:35Z Labels: MS-v2.3 |
|
From jed.lud...@gmail.com on 2013-06-01T22:51:34Z Started looking at this today. Removing the calls to notify_spyder() here: https://code.google.com/p/spyderlib/source/browse/spyderlib/widgets/externalshell/sitecustomize.py#411 and here: https://code.google.com/p/spyderlib/source/browse/spyderlib/widgets/externalshell/sitecustomize.py#427 solves the problem. That isn't really a solution since it breaks line number highlighting in the editor, but it's at least a clue. This testing was done on the 2.2 branch at revision 6460a1d7126f . Status: Started |
|
From ccordoba12 on 2013-07-12T18:14:58Z I would like this to be solved by 2.3. Jed, could you take a look at it again? Labels: -Priority-High Priority-Critical |
|
From jed.lud...@gmail.com on 2013-07-13T09:29:48Z I have looked at it in some detail already, but I'm not sure how to solve it. I can't promise anything right now as my availability over the coming weeks is questionable. |
|
From ccordoba12 on 2013-07-13T14:26:11Z Should we leave it for 2.4 or 2.3.1? |
|
From jed.lud...@gmail.com on 2013-07-13T19:02:33Z I think we leave it marked for 2.3 and try our best to get it solved. However, if it doesn't happen it's not a big enough issue to hold back the release. We bump it to the next release at that point. |
|
From jed.lud...@gmail.com on 2013-08-27T23:35:45Z After investing a fair amount of time into this issue, I can describe in detail what is happening here. Unfortunately, I don't have a simple solution. It has certainly been an educational experience studying the behavior of threading and the GIL (global interpreter lock). There are four important execution threads to consider, two in the external Python shell and then two in Spyder's interpreter. We'll call those: Pdb-Thread (the main thread in the external shell where the pdb session is running) Every pdb step generates a series of transactions on the communication sockets between the external shell process and the Spyder process. Those transactions contain blocking calls to socket.recv() which cause the calling thread to block completely until the socket receives data. So here's the scenario that that is supposed to happen at each pdb step:
The lockup problem associated with this issue occurs at step 3. At some point during the generation of the remote_view data, the Monitor-Thread loses the GIL before it completes, and the thread gets starved for execution time so no acknowledgement is ever sent to the Spyder-Thread which is blocking waiting for that acknowledgement. GIL releases occur at import statements, I/O activity, and at other periodic intervals. Any time the GIL is released, the internal scheduler decides which thread to activate next, and there is no reliable way to guarantee that any specific thread will be activated. Locks or semaphores won't work here (I tried!). It's not a problem of corrupting data on a shared socket like issue #857 . Instead, it's a thread scheduler issue, and there is no direct way to force Monitor-Thread to receive execution time. To recap, at step 3 above, Spyder-Thread is counting on Monitor-Thread (running in a completely different process) to write back, but there is no way to guarantee that Monitor-Thread keeps the GIL long enough to send the acknowledgement. You can watch it lose the GIL by adding more logging.debug() statements at various places inside of Monitor-Thread's processing functions, particularly make_remote_view and get_remote_data. The thread eventually looses the GIL to the Pdb-Thread, usually right at the import statements inside those functions. You can get it to execute a little further by moving the imports to the top of the module, but it still loses it a little bit later. The problem is well summarized by this Stackoverflow post, particularly in the comments to the higher rated answers: http://stackoverflow.com/questions/3795119/how-to-force-execution-of-a-different-thread In an effort to try and make the Pdb-Thread yield back the GIL I tried seeding time.sleep(0.25) statements at various locations in our monkey patched versions of pdb commands in sitecustomize.py. That seems to make things a little better. I can get past the s command in step 6 of the original post. But a few n commands later Spyder locks up in the same fashion as before. It's just not a reliable solution. Possible solutions:
Cc: pierre.raybaut ccordoba12 |
|
From contrebasse on 2013-11-14T05:00:01Z I have almost the same problem but less critical: after a pdb command, the code is run and spyder hangs for around 10-15s (I don't have to restart anything, just wait). |
|
From bwanama...@gmail.com on 2013-12-05T14:53:35Z Windows-7 (x64) Bummer, I am having this exact problem, during an import, just as Jed says. Is another option to consider is something like celery or twisted? Maybe that's the same as possible soln 2 in comment How can I help? |
|
From bwanama...@gmail.com on 2013-12-05T15:28:08Z @-Jed, Can you be more specific about what modules "make_remote_view" and "get_remote_data" are located in? I can't find them. Thanks! |
|
From bwanama...@gmail.com on 2013-12-05T18:01:25Z what about using our RPDB instead of PDB |
|
From jed.lud...@gmail.com on 2013-12-06T07:21:55Z @-Mark: make_remote_view and get_remote_data are both in in spyderlib/widgets/externalshell/monitor.py. The problem is not with pdb itself. I'm sure if you try to accomplish your debugging from the command line with pdb it will be successful at jumping into your import statement. The problem lies in the details of the inter-process communication between the Python process that is running the Spyder GUI itself and spawned external Python interpreters where the debugging is actually happening. Debugging IPC is not trivial, so be prepared to invest a fair amount of time if you decide to tread there. If you are looking for a quick visual debugging experience when Spyder stumbles, I would suggest winpdb. I have had very good success with it over the years. You can actually launch winpdb on the current editor script from within Spyder using F7. You'll have to define your breakpoints within the winpdb environment directly, but it remembers them from session to session. |
|
From ccordoba12 on 2013-12-11T12:22:01Z Labels: MS-v2.3.1 |
|
From ccordoba12 on 2013-12-11T12:23:47Z Labels: -MS-v2.3 |
|
From ccordoba12 on 2014-08-17T18:36:36Z Labels: -MS-v2.3.1 MS-v2.4 |
ccordoba12
changed the title
|
Closing because the Python console is going to be removed in Spyder 3.2. Please see issue #4524 for our reasons to do this. |
From dominic....@gmail.com on 2012-04-13T13:17:50Z
What steps will reproduce the problem?
Instead I step through some way and spyder hangs and I have to kill app and restart. What version of the product are you using? On what operating system? spyder 2.1.9
win 7 ulimate 32bit
python 2.6.6
side observation - if you want this stuff ( details about spyder and os) why not make it really easy for me to grab them from the about spyder dialog - as it stand I can't copy the data from the dialog as its either presenting as an image or as non copyable text.
Please provide any additional information below
. I have debugged the same files without problem using
python pdb error_check1_debug_problem.py at the command line
and in side wingide
Also I have had to recently reset spyder (using --reset command line parameter) as the console for debug and python interpreter disappeared from the gui at one point and regardless of what I did I could not recover them. I would put in a bug report on this but since I have no idea how to reproduce the problem it is not easy to say how you can work on it. That said I would suggest that you put some more tests on the logic associated with reattaching the panels when the have been undocked as I think this may have been the root cause.
Concerning my experience with spyder - looks very nice and has lots of cool features - just wish it did not stumble so blatantly debugging stuff.
Attachment: spyder_debug_problem.zip
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=991
The text was updated successfully, but these errors were encountered: