Skip to content
This repository

Qtconsole fix racecondition #1065

Merged
merged 4 commits into from over 2 years ago

3 participants

Matthias Bussonnier Fernando Perez Min RK
Matthias Bussonnier
Collaborator

See deee340f for descripition, 4573b16 just revert @fperez 65546bf which was temporary.

fix race condition in qtconsole due to a race condition beween the population
of the magic menu and the first prompt request. Resolved by upgrading the
logic of the console to handle several executions request in parallel.

Some namedTuple might still carry redundent information but the fix is the
first priority

Fixes #1057 , introduced by #956.

Note that I can't reproduce the bug, so please test carefully as it changes the logic of exec reply handeling in qtconsole

Matthias Bussonnier Revert "Temporary fix to work around #1057."
    This reverts commit 65546bf, done to
    temporaly fixed a race condition introduced by #956, next commits should
    fixe this race condition
4573b16
Matthias Bussonnier
Collaborator

forgot one case... close to reopen later... sorry

Matthias Bussonnier Carreau closed this November 29, 2011
Matthias Bussonnier qtconsole: fix race-cond, handle multiple exec
	fix race condition in qtconsole due to a race condition beween the population
	of the magic menu and the first prompt request. Resolved by upgrading the
	logic of the console to handle several executions request in parallel.

	Some namedTuple might still carry redundent information but the fix is the
	first priority

	fixes #1057 , introduced by #956
8553345
Matthias Bussonnier Carreau reopened this November 29, 2011
Matthias Bussonnier
Collaborator

Fixed and rebased... not my day...
Again, this fixes qtconsole race condition but changes the logic in qtconsole. Please test carefully before merging.
New Head is 8553345, github seems to not see it (yet ?)

Fernando Perez
Owner

I've tested it interactively and it looks good so far, I'll have another round of testing at home tonight on the computer where I saw the problem most reproducibly. I also want to wait for feedback from @minrk and @takluyver who were already participating on #1057, but it looks like we're close to a solution. Thanks for the work!

Fernando Perez
Owner

Code looks clean and logic OK to me. Unless the others have any objection, once we test it a bit more we can merge.

IPython/frontend/qt/console/frontend_widget.py
@@ -373,12 +374,14 @@ class FrontendWidget(HistoryConsoleWidget, BaseFrontendMixin):
373 374
         """ Handles replies for code execution.
374 375
         """
375 376
         self.log.debug("execute: %s", msg.get('content', ''))
376  
-        info = self._request_info.get('execute')
  377
+        info_list = self._request_info.get('execute')
  378
+        msg_id = msg['parent_header']['msg_id']
  379
+        if msg_id in info_list:
  380
+            info = info_list[msg_id]
1
Min RK Owner
minrk added a note November 29, 2011

You don't have to be protective with _request_info.get('execute'), because that should be guaranteed to exist. This can be reduced to:

msg_id = msg['parent_header'].get('msg_id')
info = self._request_info['execute'].get(msg_id)

which will also fix the bug in the following line, where info would not be defined if the msg_id was not found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
IPython/frontend/qt/console/history_console_widget.py
((6 lines not shown))
215 215
 
216 216
     def _handle_execute_reply(self, msg):
217 217
         """ Handles replies for code execution, here only session history length
218 218
         """
219  
-        info = self._request_info.get('execute')
220  
-        if info and info.id == msg['parent_header']['msg_id'] and \
221  
-                info.kind == 'save_magic' and not self._hidden:
222  
-            content = msg['content']
223  
-            status = content['status']
224  
-            if status == 'ok':
225  
-                self._max_session_history=(int(content['user_expressions']['hlen']))
  219
+        info_list = self._request_info.get('execute')
1
Min RK Owner
minrk added a note November 29, 2011

probably change info_list to info_dict here and elsewhere, since it's not a list anymore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
IPython/frontend/qt/console/history_console_widget.py
((6 lines not shown))
215 215
 
216 216
     def _handle_execute_reply(self, msg):
217 217
         """ Handles replies for code execution, here only session history length
218 218
         """
219  
-        info = self._request_info.get('execute')
220  
-        if info and info.id == msg['parent_header']['msg_id'] and \
221  
-                info.kind == 'save_magic' and not self._hidden:
222  
-            content = msg['content']
223  
-            status = content['status']
224  
-            if status == 'ok':
225  
-                self._max_session_history=(int(content['user_expressions']['hlen']))
  219
+        info_list = self._request_info.get('execute')
  220
+        msg_id = msg['parent_header']['msg_id']
1
Min RK Owner
minrk added a note November 29, 2011

same as before:

info = self._request_info['execute'].pop(msg_id, None)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Min RK
Owner

Looks good, with the few small changes I noted inline. Thanks!

Matthias Bussonnier
Collaborator

@minrk
I guess you're right, thanks, fixed.

Fernando Perez
Owner

Great, thanks. I'll do a final round of testing tonight at home where the problem was more clearly visible, and will merge if it looks OK.

Fernando Perez
Owner

OK, I've verified here that it works fine. So we're good to go, thanks! Merging now.

Fernando Perez fperez merged commit a5fd0a3 into from November 29, 2011
Fernando Perez fperez closed this November 29, 2011
Fernando Perez fperez referenced this pull request from a commit January 10, 2012
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 4 unique commits by 1 author.

Nov 29, 2011
Matthias Bussonnier Revert "Temporary fix to work around #1057."
    This reverts commit 65546bf, done to
    temporaly fixed a race condition introduced by #956, next commits should
    fixe this race condition
4573b16
Matthias Bussonnier qtconsole: fix race-cond, handle multiple exec
	fix race condition in qtconsole due to a race condition beween the population
	of the magic menu and the first prompt request. Resolved by upgrading the
	logic of the console to handle several executions request in parallel.

	Some namedTuple might still carry redundent information but the fix is the
	first priority

	fixes #1057 , introduced by #956
8553345
Matthias Bussonnier fix to minrk suggestion b43e054
Matthias Bussonnier fix forgot one pop in execute callback 12feaab
This page is out of date. Refresh to see the latest.
18  IPython/frontend/qt/console/frontend_widget.py
@@ -138,6 +138,7 @@ def __init__(self, *args, **kw):
138 138
         self._input_splitter = self._input_splitter_class(input_mode='cell')
139 139
         self._kernel_manager = None
140 140
         self._request_info = {}
  141
+        self._request_info['execute'] = {};
141 142
         self._callback_dict = {}
142 143
 
143 144
         # Configure the ConsoleWidget.
@@ -199,7 +200,7 @@ def _execute(self, source, hidden):
199 200
         See parent class :meth:`execute` docstring for full details.
200 201
         """
201 202
         msg_id = self.kernel_manager.shell_channel.execute(source, hidden)
202  
-        self._request_info['execute'] = self._ExecutionRequest(msg_id, 'user')
  203
+        self._request_info['execute'][msg_id] = self._ExecutionRequest(msg_id, 'user')
203 204
         self._hidden = hidden
204 205
         if not hidden:
205 206
             self.executing.emit(source)
@@ -343,7 +344,7 @@ def _silent_exec_callback(self, expr, callback):
343 344
         msg_id = self.kernel_manager.shell_channel.execute('',
344 345
             silent=True, user_expressions={ local_uuid:expr })
345 346
         self._callback_dict[local_uuid] = callback
346  
-        self._request_info['execute'] = self._ExecutionRequest(msg_id, 'silent_exec_callback')
  347
+        self._request_info['execute'][msg_id] = self._ExecutionRequest(msg_id, 'silent_exec_callback')
347 348
 
348 349
     def _handle_exec_callback(self, msg):
349 350
         """Execute `callback` corresonding to `msg` reply, after ``_silent_exec_callback``
@@ -373,12 +374,12 @@ def _handle_execute_reply(self, msg):
373 374
         """ Handles replies for code execution.
374 375
         """
375 376
         self.log.debug("execute: %s", msg.get('content', ''))
376  
-        info = self._request_info.get('execute')
  377
+        msg_id = msg['parent_header']['msg_id']
  378
+        info = self._request_info['execute'].get(msg_id)
377 379
         # unset reading flag, because if execute finished, raw_input can't
378 380
         # still be pending.
379 381
         self._reading = False
380  
-        if info and info.id == msg['parent_header']['msg_id'] and \
381  
-                info.kind == 'user' and not self._hidden:
  382
+        if info and info.kind == 'user' and not self._hidden:
382 383
             # Make sure that all output from the SUB channel has been processed
383 384
             # before writing a new prompt.
384 385
             self.kernel_manager.sub_channel.flush()
@@ -400,9 +401,10 @@ def _handle_execute_reply(self, msg):
400 401
 
401 402
             self._show_interpreter_prompt_for_reply(msg)
402 403
             self.executed.emit(msg)
403  
-        elif info and info.id == msg['parent_header']['msg_id'] and \
404  
-                info.kind == 'silent_exec_callback' and not self._hidden:
  404
+            self._request_info['execute'].pop(msg_id)
  405
+        elif info and info.kind == 'silent_exec_callback' and not self._hidden:
405 406
             self._handle_exec_callback(msg)
  407
+            self._request_info['execute'].pop(msg_id)
406 408
         else:
407 409
             super(FrontendWidget, self)._handle_execute_reply(msg)
408 410
 
@@ -559,7 +561,7 @@ def reset(self):
559 561
         """
560 562
         if self._executing:
561 563
             self._executing = False
562  
-            self._request_info['execute'] = None
  564
+            self._request_info['execute'] = {}
563 565
         self._reading = False
564 566
         self._highlighter.highlighting_on = False
565 567
 
8  IPython/frontend/qt/console/history_console_widget.py
@@ -211,14 +211,14 @@ def _request_update_session_history_length(self):
211 211
                 'hlen':'len(get_ipython().history_manager.input_hist_raw)',
212 212
                 }
213 213
             )
214  
-        self._request_info['execute'] = self._ExecutionRequest(msg_id, 'save_magic')
  214
+        self._request_info['execute'][msg_id] = self._ExecutionRequest(msg_id, 'save_magic')
215 215
 
216 216
     def _handle_execute_reply(self, msg):
217 217
         """ Handles replies for code execution, here only session history length
218 218
         """
219  
-        info = self._request_info.get('execute')
220  
-        if info and info.id == msg['parent_header']['msg_id'] and \
221  
-                info.kind == 'save_magic' and not self._hidden:
  219
+        msg_id = msg['parent_header']['msg_id']
  220
+        info = self._request_info.get['execute'].pop(msg_id,None)
  221
+        if info and info.kind == 'save_magic' and not self._hidden:
222 222
             content = msg['content']
223 223
             status = content['status']
224 224
             if status == 'ok':
17  IPython/frontend/qt/console/ipython_widget.py
@@ -165,13 +165,14 @@ def _handle_complete_reply(self, rep):
165 165
     def _handle_execute_reply(self, msg):
166 166
         """ Reimplemented to support prompt requests.
167 167
         """
168  
-        info = self._request_info.get('execute')
169  
-        if info and info.id == msg['parent_header']['msg_id']:
170  
-            if info.kind == 'prompt':
171  
-                number = msg['content']['execution_count'] + 1
172  
-                self._show_interpreter_prompt(number)
173  
-            else:
174  
-                super(IPythonWidget, self)._handle_execute_reply(msg)
  168
+        msg_id = msg['parent_header'].get('msg_id')
  169
+        info = self._request_info['execute'].get(msg_id)
  170
+        if info and info.kind == 'prompt':
  171
+           number = msg['content']['execution_count'] + 1
  172
+           self._show_interpreter_prompt(number)
  173
+           self._request_info['execute'].pop(msg_id)
  174
+        else:
  175
+           super(IPythonWidget, self)._handle_execute_reply(msg)
175 176
 
176 177
     def _handle_history_reply(self, msg):
177 178
         """ Implemented to handle history tail replies, which are only supported
@@ -358,7 +359,7 @@ def _show_interpreter_prompt(self, number=None):
358 359
         if number is None:
359 360
             msg_id = self.kernel_manager.shell_channel.execute('', silent=True)
360 361
             info = self._ExecutionRequest(msg_id, 'prompt')
361  
-            self._request_info['execute'] = info
  362
+            self._request_info['execute'][msg_id] = info
362 363
             return
363 364
 
364 365
         # Show a new prompt and save information about it so that it can be
60  IPython/frontend/qt/console/mainwindow.py
@@ -626,17 +626,12 @@ def init_magic_menu(self):
626 626
         # is updated at first kernel response. Though, it is necessary when
627 627
         # connecting through X-forwarding, as in this case, the menu is not
628 628
         # auto updated, SO DO NOT DELETE.
629  
-
630  
-        ########################################################################
631  
-        ## TEMPORARILY DISABLED - see #1057 for details.  Uncomment this
632  
-        ## section when a proper fix is found
633  
-        
634  
-        ## self.pop = QtGui.QAction("&Update All Magic Menu ",
635  
-        ##     self, triggered=self.update_all_magic_menu)
636  
-        ## self.add_menu_action(self.all_magic_menu, self.pop)
637  
-
638  
-        ## END TEMPORARY FIX
639  
-        ########################################################################
  629
+        self.pop = QtGui.QAction("&Update All Magic Menu ",
  630
+            self, triggered=self.update_all_magic_menu)
  631
+        self.add_menu_action(self.all_magic_menu, self.pop)
  632
+        # we need to populate the 'Magic Menu' once the kernel has answer at
  633
+        # least once let's do it immedialy, but it's assured to works
  634
+        self.pop.trigger()
640 635
 
641 636
         self.reset_action = QtGui.QAction("&Reset",
642 637
             self,
@@ -674,49 +669,6 @@ def init_magic_menu(self):
674 669
             triggered=self.whos_magic_active_frontend)
675 670
         self.add_menu_action(self.magic_menu, self.whos_action)
676 671
 
677  
-
678  
-        ########################################################################
679  
-        ## TEMPORARILY ADDED BACK - see #1057 for details.  The magic menu is
680  
-        ## supposed to be dynamic, but the mechanism merged in #1057 has a race
681  
-        ## condition that locks up the console very often.  We're putting back
682  
-        ## the static list temporarily/ Remove this code when a proper fix is
683  
-        ## found.
684  
-        
685  
-        # allmagics submenu.
686  
-        
687  
-        # for now this is just a copy and paste, but we should get this
688  
-        # dynamically
689  
-        magiclist = ["%alias", "%autocall", "%automagic", "%bookmark", "%cd",
690  
-        "%clear", "%colors", "%debug", "%dhist", "%dirs", "%doctest_mode",
691  
-        "%ed", "%edit", "%env", "%gui", "%guiref", "%hist", "%history",
692  
-        "%install_default_config", "%install_profiles", "%less", "%load_ext",
693  
-        "%loadpy", "%logoff", "%logon", "%logstart", "%logstate", "%logstop",
694  
-        "%lsmagic", "%macro", "%magic", "%man", "%more", "%notebook", "%page",
695  
-        "%pastebin", "%pdb", "%pdef", "%pdoc", "%pfile", "%pinfo", "%pinfo2",
696  
-        "%popd", "%pprint", "%precision", "%profile", "%prun", "%psearch",
697  
-        "%psource", "%pushd", "%pwd", "%pycat", "%pylab", "%quickref",
698  
-        "%recall", "%rehashx", "%reload_ext", "%rep", "%rerun", "%reset",
699  
-        "%reset_selective", "%run", "%save", "%sc", "%sx", "%tb", "%time",
700  
-        "%timeit", "%unalias", "%unload_ext", "%who", "%who_ls", "%whos",
701  
-        "%xdel", "%xmode"]
702  
-
703  
-        def make_dynamic_magic(i):
704  
-                def inner_dynamic_magic():
705  
-                    self.active_frontend.execute(i)
706  
-                inner_dynamic_magic.__name__ = "dynamics_magic_%s" % i
707  
-                return inner_dynamic_magic
708  
-
709  
-        for magic in magiclist:
710  
-            xaction = QtGui.QAction(magic,
711  
-                self,
712  
-                triggered=make_dynamic_magic(magic)
713  
-                )
714  
-            self.all_magic_menu.addAction(xaction)
715  
-
716  
-        ## END TEMPORARY FIX
717  
-        ########################################################################
718  
-
719  
-
720 672
     def init_window_menu(self):
721 673
         self.window_menu = self.menuBar().addMenu("&Window")
722 674
         if sys.platform == 'darwin':
10  IPython/frontend/qt/console/qtconsoleapp.py
@@ -452,16 +452,6 @@ def init_qt_elements(self):
452 452
         self.window.add_tab_with_frontend(self.widget)
453 453
         self.window.init_menu_bar()
454 454
 
455  
-        # we need to populate the 'Magic Menu' once the kernel has answer at
456  
-        # least once 
457  
-
458  
-        ########################################################################
459  
-        ## TEMPORARILY DISABLED - see #1057 for details, uncomment the next
460  
-        ## line when a proper fix is found:
461  
-        ## self.kernel_manager.shell_channel.first_reply.connect(self.window.pop.trigger)
462  
-        ## END TEMPORARY FIX
463  
-        ########################################################################
464  
-
465 455
         self.window.setWindowTitle('Python' if self.pure else 'IPython')
466 456
 
467 457
     def init_colors(self):
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.