Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Qtconsole fix racecondition #1065

Merged
merged 4 commits into from

3 participants

@Carreau
Owner

See deee340f for descripition, 4573b16 just revert @fperez 65546bf which was temporary.

fix race condition in qtconsole due to a race condition beween the population
of the magic menu and the first prompt request. Resolved by upgrading the
logic of the console to handle several executions request in parallel.

Some namedTuple might still carry redundent information but the fix is the
first priority

Fixes #1057 , introduced by #956.

Note that I can't reproduce the bug, so please test carefully as it changes the logic of exec reply handeling in qtconsole

@Carreau Carreau Revert "Temporary fix to work around #1057."
    This reverts commit 65546bf, done to
    temporaly fixed a race condition introduced by #956, next commits should
    fixe this race condition
4573b16
@Carreau
Owner

forgot one case... close to reopen later... sorry

@Carreau Carreau closed this
@Carreau Carreau qtconsole: fix race-cond, handle multiple exec
	fix race condition in qtconsole due to a race condition beween the population
	of the magic menu and the first prompt request. Resolved by upgrading the
	logic of the console to handle several executions request in parallel.

	Some namedTuple might still carry redundent information but the fix is the
	first priority

	fixes #1057 , introduced by #956
8553345
@Carreau Carreau reopened this
@Carreau
Owner

Fixed and rebased... not my day...
Again, this fixes qtconsole race condition but changes the logic in qtconsole. Please test carefully before merging.
New Head is 8553345, github seems to not see it (yet ?)

@fperez
Owner

I've tested it interactively and it looks good so far, I'll have another round of testing at home tonight on the computer where I saw the problem most reproducibly. I also want to wait for feedback from @minrk and @takluyver who were already participating on #1057, but it looks like we're close to a solution. Thanks for the work!

@fperez
Owner

Code looks clean and logic OK to me. Unless the others have any objection, once we test it a bit more we can merge.

IPython/frontend/qt/console/frontend_widget.py
@@ -373,12 +374,14 @@ class FrontendWidget(HistoryConsoleWidget, BaseFrontendMixin):
""" Handles replies for code execution.
"""
self.log.debug("execute: %s", msg.get('content', ''))
- info = self._request_info.get('execute')
+ info_list = self._request_info.get('execute')
+ msg_id = msg['parent_header']['msg_id']
+ if msg_id in info_list:
+ info = info_list[msg_id]
@minrk Owner
minrk added a note

You don't have to be protective with _request_info.get('execute'), because that should be guaranteed to exist. This can be reduced to:

msg_id = msg['parent_header'].get('msg_id')
info = self._request_info['execute'].get(msg_id)

which will also fix the bug in the following line, where info would not be defined if the msg_id was not found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
IPython/frontend/qt/console/history_console_widget.py
((6 lines not shown))
def _handle_execute_reply(self, msg):
""" Handles replies for code execution, here only session history length
"""
- info = self._request_info.get('execute')
- if info and info.id == msg['parent_header']['msg_id'] and \
- info.kind == 'save_magic' and not self._hidden:
- content = msg['content']
- status = content['status']
- if status == 'ok':
- self._max_session_history=(int(content['user_expressions']['hlen']))
+ info_list = self._request_info.get('execute')
@minrk Owner
minrk added a note

probably change info_list to info_dict here and elsewhere, since it's not a list anymore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
IPython/frontend/qt/console/history_console_widget.py
((6 lines not shown))
def _handle_execute_reply(self, msg):
""" Handles replies for code execution, here only session history length
"""
- info = self._request_info.get('execute')
- if info and info.id == msg['parent_header']['msg_id'] and \
- info.kind == 'save_magic' and not self._hidden:
- content = msg['content']
- status = content['status']
- if status == 'ok':
- self._max_session_history=(int(content['user_expressions']['hlen']))
+ info_list = self._request_info.get('execute')
+ msg_id = msg['parent_header']['msg_id']
@minrk Owner
minrk added a note

same as before:

info = self._request_info['execute'].pop(msg_id, None)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@minrk
Owner

Looks good, with the few small changes I noted inline. Thanks!

@Carreau
Owner

@minrk
I guess you're right, thanks, fixed.

@fperez
Owner

Great, thanks. I'll do a final round of testing tonight at home where the problem was more clearly visible, and will merge if it looks OK.

@fperez
Owner

OK, I've verified here that it works fine. So we're good to go, thanks! Merging now.

@fperez fperez merged commit a5fd0a3 into ipython:master
@fperez fperez referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 29, 2011
  1. @Carreau

    Revert "Temporary fix to work around #1057."

    Carreau authored
        This reverts commit 65546bf, done to
        temporaly fixed a race condition introduced by #956, next commits should
        fixe this race condition
  2. @Carreau

    qtconsole: fix race-cond, handle multiple exec

    Carreau authored
    	fix race condition in qtconsole due to a race condition beween the population
    	of the magic menu and the first prompt request. Resolved by upgrading the
    	logic of the console to handle several executions request in parallel.
    
    	Some namedTuple might still carry redundent information but the fix is the
    	first priority
    
    	fixes #1057 , introduced by #956
  3. @Carreau

    fix to minrk suggestion

    Carreau authored
  4. @Carreau
This page is out of date. Refresh to see the latest.
View
18 IPython/frontend/qt/console/frontend_widget.py
@@ -138,6 +138,7 @@ def __init__(self, *args, **kw):
self._input_splitter = self._input_splitter_class(input_mode='cell')
self._kernel_manager = None
self._request_info = {}
+ self._request_info['execute'] = {};
self._callback_dict = {}
# Configure the ConsoleWidget.
@@ -199,7 +200,7 @@ def _execute(self, source, hidden):
See parent class :meth:`execute` docstring for full details.
"""
msg_id = self.kernel_manager.shell_channel.execute(source, hidden)
- self._request_info['execute'] = self._ExecutionRequest(msg_id, 'user')
+ self._request_info['execute'][msg_id] = self._ExecutionRequest(msg_id, 'user')
self._hidden = hidden
if not hidden:
self.executing.emit(source)
@@ -343,7 +344,7 @@ def _silent_exec_callback(self, expr, callback):
msg_id = self.kernel_manager.shell_channel.execute('',
silent=True, user_expressions={ local_uuid:expr })
self._callback_dict[local_uuid] = callback
- self._request_info['execute'] = self._ExecutionRequest(msg_id, 'silent_exec_callback')
+ self._request_info['execute'][msg_id] = self._ExecutionRequest(msg_id, 'silent_exec_callback')
def _handle_exec_callback(self, msg):
"""Execute `callback` corresonding to `msg` reply, after ``_silent_exec_callback``
@@ -373,12 +374,12 @@ def _handle_execute_reply(self, msg):
""" Handles replies for code execution.
"""
self.log.debug("execute: %s", msg.get('content', ''))
- info = self._request_info.get('execute')
+ msg_id = msg['parent_header']['msg_id']
+ info = self._request_info['execute'].get(msg_id)
# unset reading flag, because if execute finished, raw_input can't
# still be pending.
self._reading = False
- if info and info.id == msg['parent_header']['msg_id'] and \
- info.kind == 'user' and not self._hidden:
+ if info and info.kind == 'user' and not self._hidden:
# Make sure that all output from the SUB channel has been processed
# before writing a new prompt.
self.kernel_manager.sub_channel.flush()
@@ -400,9 +401,10 @@ def _handle_execute_reply(self, msg):
self._show_interpreter_prompt_for_reply(msg)
self.executed.emit(msg)
- elif info and info.id == msg['parent_header']['msg_id'] and \
- info.kind == 'silent_exec_callback' and not self._hidden:
+ self._request_info['execute'].pop(msg_id)
+ elif info and info.kind == 'silent_exec_callback' and not self._hidden:
self._handle_exec_callback(msg)
+ self._request_info['execute'].pop(msg_id)
else:
super(FrontendWidget, self)._handle_execute_reply(msg)
@@ -559,7 +561,7 @@ def reset(self):
"""
if self._executing:
self._executing = False
- self._request_info['execute'] = None
+ self._request_info['execute'] = {}
self._reading = False
self._highlighter.highlighting_on = False
View
8 IPython/frontend/qt/console/history_console_widget.py
@@ -211,14 +211,14 @@ def _request_update_session_history_length(self):
'hlen':'len(get_ipython().history_manager.input_hist_raw)',
}
)
- self._request_info['execute'] = self._ExecutionRequest(msg_id, 'save_magic')
+ self._request_info['execute'][msg_id] = self._ExecutionRequest(msg_id, 'save_magic')
def _handle_execute_reply(self, msg):
""" Handles replies for code execution, here only session history length
"""
- info = self._request_info.get('execute')
- if info and info.id == msg['parent_header']['msg_id'] and \
- info.kind == 'save_magic' and not self._hidden:
+ msg_id = msg['parent_header']['msg_id']
+ info = self._request_info.get['execute'].pop(msg_id,None)
+ if info and info.kind == 'save_magic' and not self._hidden:
content = msg['content']
status = content['status']
if status == 'ok':
View
17 IPython/frontend/qt/console/ipython_widget.py
@@ -165,13 +165,14 @@ def _handle_complete_reply(self, rep):
def _handle_execute_reply(self, msg):
""" Reimplemented to support prompt requests.
"""
- info = self._request_info.get('execute')
- if info and info.id == msg['parent_header']['msg_id']:
- if info.kind == 'prompt':
- number = msg['content']['execution_count'] + 1
- self._show_interpreter_prompt(number)
- else:
- super(IPythonWidget, self)._handle_execute_reply(msg)
+ msg_id = msg['parent_header'].get('msg_id')
+ info = self._request_info['execute'].get(msg_id)
+ if info and info.kind == 'prompt':
+ number = msg['content']['execution_count'] + 1
+ self._show_interpreter_prompt(number)
+ self._request_info['execute'].pop(msg_id)
+ else:
+ super(IPythonWidget, self)._handle_execute_reply(msg)
def _handle_history_reply(self, msg):
""" Implemented to handle history tail replies, which are only supported
@@ -358,7 +359,7 @@ def _show_interpreter_prompt(self, number=None):
if number is None:
msg_id = self.kernel_manager.shell_channel.execute('', silent=True)
info = self._ExecutionRequest(msg_id, 'prompt')
- self._request_info['execute'] = info
+ self._request_info['execute'][msg_id] = info
return
# Show a new prompt and save information about it so that it can be
View
60 IPython/frontend/qt/console/mainwindow.py
@@ -626,17 +626,12 @@ def init_magic_menu(self):
# is updated at first kernel response. Though, it is necessary when
# connecting through X-forwarding, as in this case, the menu is not
# auto updated, SO DO NOT DELETE.
-
- ########################################################################
- ## TEMPORARILY DISABLED - see #1057 for details. Uncomment this
- ## section when a proper fix is found
-
- ## self.pop = QtGui.QAction("&Update All Magic Menu ",
- ## self, triggered=self.update_all_magic_menu)
- ## self.add_menu_action(self.all_magic_menu, self.pop)
-
- ## END TEMPORARY FIX
- ########################################################################
+ self.pop = QtGui.QAction("&Update All Magic Menu ",
+ self, triggered=self.update_all_magic_menu)
+ self.add_menu_action(self.all_magic_menu, self.pop)
+ # we need to populate the 'Magic Menu' once the kernel has answer at
+ # least once let's do it immedialy, but it's assured to works
+ self.pop.trigger()
self.reset_action = QtGui.QAction("&Reset",
self,
@@ -674,49 +669,6 @@ def init_magic_menu(self):
triggered=self.whos_magic_active_frontend)
self.add_menu_action(self.magic_menu, self.whos_action)
-
- ########################################################################
- ## TEMPORARILY ADDED BACK - see #1057 for details. The magic menu is
- ## supposed to be dynamic, but the mechanism merged in #1057 has a race
- ## condition that locks up the console very often. We're putting back
- ## the static list temporarily/ Remove this code when a proper fix is
- ## found.
-
- # allmagics submenu.
-
- # for now this is just a copy and paste, but we should get this
- # dynamically
- magiclist = ["%alias", "%autocall", "%automagic", "%bookmark", "%cd",
- "%clear", "%colors", "%debug", "%dhist", "%dirs", "%doctest_mode",
- "%ed", "%edit", "%env", "%gui", "%guiref", "%hist", "%history",
- "%install_default_config", "%install_profiles", "%less", "%load_ext",
- "%loadpy", "%logoff", "%logon", "%logstart", "%logstate", "%logstop",
- "%lsmagic", "%macro", "%magic", "%man", "%more", "%notebook", "%page",
- "%pastebin", "%pdb", "%pdef", "%pdoc", "%pfile", "%pinfo", "%pinfo2",
- "%popd", "%pprint", "%precision", "%profile", "%prun", "%psearch",
- "%psource", "%pushd", "%pwd", "%pycat", "%pylab", "%quickref",
- "%recall", "%rehashx", "%reload_ext", "%rep", "%rerun", "%reset",
- "%reset_selective", "%run", "%save", "%sc", "%sx", "%tb", "%time",
- "%timeit", "%unalias", "%unload_ext", "%who", "%who_ls", "%whos",
- "%xdel", "%xmode"]
-
- def make_dynamic_magic(i):
- def inner_dynamic_magic():
- self.active_frontend.execute(i)
- inner_dynamic_magic.__name__ = "dynamics_magic_%s" % i
- return inner_dynamic_magic
-
- for magic in magiclist:
- xaction = QtGui.QAction(magic,
- self,
- triggered=make_dynamic_magic(magic)
- )
- self.all_magic_menu.addAction(xaction)
-
- ## END TEMPORARY FIX
- ########################################################################
-
-
def init_window_menu(self):
self.window_menu = self.menuBar().addMenu("&Window")
if sys.platform == 'darwin':
View
10 IPython/frontend/qt/console/qtconsoleapp.py
@@ -452,16 +452,6 @@ def init_qt_elements(self):
self.window.add_tab_with_frontend(self.widget)
self.window.init_menu_bar()
- # we need to populate the 'Magic Menu' once the kernel has answer at
- # least once
-
- ########################################################################
- ## TEMPORARILY DISABLED - see #1057 for details, uncomment the next
- ## line when a proper fix is found:
- ## self.kernel_manager.shell_channel.first_reply.connect(self.window.pop.trigger)
- ## END TEMPORARY FIX
- ########################################################################
-
self.window.setWindowTitle('Python' if self.pure else 'IPython')
def init_colors(self):
Something went wrong with that request. Please try again.