Skip to content
This repository

Two-process terminal frontend (ipython core branch) #864

Merged
merged 50 commits into from over 2 years ago

5 participants

Paul Ivanov Fernando Perez Omar Andres Zapata Mesa Thomas Kluyver Min RK
Paul Ivanov
Collaborator

This PR supersedes @minrk's #708 (and in turn, @omazapa's #433)

It's not ready to merge yet, but I've fixed Min's complaint about ctrl-c crashing the kernel.

one thing i see currently wrong is when you invoke ipython zmq --existing ... - ctrl-c is broken and complains about not having a kernel (now fixed)

Fernando Perez
Owner

Thanks for pounding on this one! BTW, I'd like to change the name of the client from zmq (which is just an internal implementation detail) to console, which better follows qtconsole, and is a more user-informative name.

Omar Andres Zapata Mesa

thanks, great!!!

Thomas Kluyver
Collaborator

I think I'd given it the name zmqterminal at some point, to distinguish it from the standard single-process terminal interface. I don't know what's the best way to make that distinction clear.

Fernando Perez
Owner

My reasoning was: ipython is the simple, one-process call, by historic precedent in terminal. And ipython <X> refers to all two process clients, for <X> equal to:

  • console: regular text console.
  • qtconsole: gui
  • notebook: web notebook right now.
  • other future ones: if they develop, we'll see.

But at least that gives a clear pattern that should be easy for users to understand/remember.

Paul Ivanov
Collaborator

Progress report so far:

  1. refactored Qt-based Console and the terminal-based Console (which used to be called 'zmq') using one common mixin that both apps now utilize - both seem to work, though I need to do a more thorough job of flag and alias swallowing.
  2. renamed the ipython subcommand to 'console' - added some basic docs so it shows up in ipython --help and does the right thing for ipython console -h
  3. handle the interrupt handler for instances which started up with --existing flag in the same way as they're treated in qtconsole (print a message saying "Kernel process is either remote or unspecified. Cannot interrupt."
Min RK
Owner

awesome! I'll look through it later. For flags&aliases, I recommend looking at IPython.core.shellapp, and its two derivatives: IPKernelApp and TerminalIPythonApp. That mixin is actually less complete than what you put together, only defining the unique elements, and letting the descendants assemble the various components. I don't know which approach is better.

It would be great if you can split the swallow_args into just a function - it really should just be a single function with three args:

new_argv = strip_argv(argv, aliases=aliases, flags=flags)

since it's generically useful for any app that starts one or more others in a subprocess, but you want to be able to specify command-line args for both (e.g. I have designs on using it in ipcluster, so that some flags can be passed on to ipcontroller/ipengine).

I don't exactly know where that should go, though. Maybe utils.process?

IPython/frontend/kernelmixinapp.py
((186 lines not shown))
  186
+        # make this stuff after this a function, in case the super stuff goes
  187
+        # away. Also, Min notes that this functionality should be moved to a
  188
+        # generic library of kernel stuff
  189
+        self.swallow_args(app_aliases,app_flags,argv=argv)
  190
+
  191
+    def swallow_args(self, aliases,flags, argv=None):
  192
+        if argv is None:
  193
+            argv = sys.argv[1:]
  194
+        self.kernel_argv = list(argv) # copy
  195
+        # kernel should inherit default config file from frontend
  196
+        self.kernel_argv.append("--KernelApp.parent_appname='%s'"%self.name)
  197
+        # Scrub frontend-specific flags
  198
+        swallow_next = False
  199
+        was_flag = False
  200
+        # copy again, in case some aliases have the same name as a flag
  201
+        # argv = list(self.kernel_argv)
1
Min RK Owner
minrk added a note

This second copy is a leftover from when there were two passes. This isn't necessary anymore.

(I know, I'm reviewing my own messy code)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
IPython/frontend/kernelmixinapp.py
((96 lines not shown))
  96
+
  97
+
  98
+    ssh = 'IPythonMixinConsoleApp.sshserver',
  99
+)
  100
+aliases.update(app_aliases)
  101
+
  102
+#-----------------------------------------------------------------------------
  103
+# Classes
  104
+#-----------------------------------------------------------------------------
  105
+
  106
+#-----------------------------------------------------------------------------
  107
+# IPythonMixinConsole
  108
+#-----------------------------------------------------------------------------
  109
+
  110
+
  111
+class IPythonMixinConsoleApp(Configurable):
1
Min RK Owner
minrk added a note

minor thing - maybe call it ConsoleApp or KernelManagerApp? It's a Mixin for an application that starts/connects a KernelManager. For instance, InteractiveShellApp is relatively clear, because it affects apps that start an InteractiveShell (of any kind). Names are somewhat important, because they show up in config files. Our class/trait structure is our public configuration interface. This one is less important, because I don't think people will actually want to set any of these config values statically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Min RK
Owner

And if you could figure out how to cleanup connection files on sigint, that would be great. I think you are better at signal-handling than I am. The connection-file cleanup is handled in the KernelManager.__del__ method. The important thing is to not cleanup connection files that you didn't write.

Min RK
Owner

I'm pretty sure I've figured out the connection file cleanup, I'll push that shortly, unless you are working on it.

Min RK
Owner

okay, connection file cleanup is definitely better now. ctrl-C killed qtconsole will cleanup all of its files, and the terminal app will also get it done under normal circumstances. Obviously unclean shutdown (term) doesn't allow for cleanup.

Min RK
Owner

flags and aliases are working now, as well.

The main things that remain are:

  • a few renames
    • We need to make sure that the mixin's imports don't break tests or anything if zmq, etc. is not installed.
    • I'm not certain about the mixin class name. If it were to match the ShellApp, which is our mixin for applications that start an InteractiveShell, it would probably be KernelManagerApp, as it is a mixin for apps that start a KernelManager, but ConsoleApp may make more sense when it shows up in config.
    • frontend/terminal/zmqterminal -> frontend/terminal/console, if I understand our plans correctly
  • update docs, to take note of the new client
  • Figure out how to get the connection message from the kernel to not show up in the first prompt. Perhaps it should not be coming from the kernel, rather the frontend should print it, so that it can be managed properly. Otherwise, a time.sleep(1) is the only available answer.
  • add a way to kill the kernel. I don't know if you can bind keys to arbitrary python functions with readline, but if not, we need a frontend-only magic or something.
Min RK
Owner

Getting pretty close. I just fixed a few things regarding various message cases, signaling, startup, and unresponsive kernels, and moved the new code into frontend.terminal.console.

Another thing that needs to be done: port the multiline history changes in shell.interact, once #929 is merged.

I did have to fix two things outside the new code:

  • the heartbeat channel had various logical errors that come up when using the blockingKM, with its shorter poll (which I still had to lengthen to 1s)
  • zmqshell clobbered page.page at import, which meant that any frontends that import the class for config reference will be operating in a universe without a functional page command. This has been moved to shell instantiation.
Min RK
Owner

update on recent changes:

  • renames have been done (it is now IPython.frontend.consoleapp.IPythonConsoleApp)
  • connection message does not interfere with prompt

Still todo:

  • document new interface
  • kill kernel from frontend

after IRC discussion with @fperez, the idea is:

frontend magics can be accessed exclusively as %foo bar, and these should be checked by simple regexp in run_cell, to bypass relaying to the kernel.

For instance, this could be handled with %kernel kill / %kernel restart or similar.

Simple %kernel? should access a helpstring, but need not trigger full pinfo behavior.
Improvements affecting qtconsole or other code:

  • heartbeat logic fixed / improved
  • much more likely to cleanup connection files
  • swallow_argv is a standalone function
Fernando Perez
Owner

Here's a quick stab at the right regexp for this:

import re

names = ['kernel', 'edit']
pattern = '^%(?P<name>' + '|'.join(names) + ') (?P<args>.*\S)\s+'
magic_re = re.compile(pattern)

line = '%kernel a1 a2  '

m = magic_re.match(line)
print 'pattern:', pattern
if m:
    print 'name:', m.group('name')
    print 'args:', m.group('args')

We'll also need to add the logic to catch ? and ??, but that should be about it.

Fernando Perez
Owner

I fixed a couple of small problems arising from my renaming of lib.pylabtools, but I'm still getting an error in the test suite:

======================================================================
ERROR: Failure: ImportError (cannot import name ClientCompleter2p)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/console/frontend.py", line 33, in <module>
    from IPython.frontend.terminal.console.completer import ClientCompleter2p
ImportError: cannot import name ClientCompleter2p

----------------------------------------------------------------------

I looked at that file, and there's nothing named ClientCompleter2p in there, so I'm kind of puzzled as to what happened. A forgotten commit?

Fernando Perez
Owner

In actual testing, I noticed that %debug isn't working right, b/c of what appears to be a buffering issue. If I issue a few l commands, nothing gets printed (though prompts do come out fine). Once I quit the debugger, all the buffered listings show up at once.

Min RK
Owner

@fperez - I think terminal.console.frontend is no longer used at all - that was the first draft, that didn't use InteractiveShell or anything.

I see that you pushed some changes to ipython/origin-termzmq. Did you mean to push them here?

Fernando Perez
Owner
Min RK
Owner

I just did a rebase/push without issue. Also removed the unused frontend.py.

Fernando Perez
Owner

Thanks! But I'm really puzzled, I tried again on a different machine and got the same set of errors:

(termzmq)amirbar[ipython]> git rebase master
First, rewinding head to replay your work on top of it...
Applying: basic kernelmanager and frontend wrote, support indentation but it dont run code yet
Applying: working in handlers
Applying: working in tab completion
Applying: completer not working fine
Applying: little bug fixed in kernelmanager's queues
Applying: raw_input captured and working fine
Applying: -mworking in tab-completion
Applying: tab completion is not working yet, unknow error
Applying: Make readline tab-completion work in two-process terminal frontend.
/home/fperez/ipython/ipython/.git/rebase-apply/patch:18: trailing whitespace.

warning: 1 line adds whitespace errors.
Using index info to reconstruct a base tree...
<stdin>:18: trailing whitespace.

warning: 1 line applied after fixing whitespace errors.
Falling back to patching base and 3-way merge...
fatal: Unable to create '/home/fperez/ipython/ipython/.git/index.lock': File exists.

If no other git process is currently running, this probably means a
git process crashed in this repository earlier. Make sure no other git
process is running and remove the file manually to continue.
Failed to merge in the changes.
Patch failed at 0009 Make readline tab-completion work in two-process terminal frontend.

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

Furthermore, the above looks like a git bug: that index.lock file didn't exist before, I made sure to remove it... And even if I remove it again, it bombs even on --abort:

((d8c32bc...)|REBASE)amirbar[ipython]> git rebase --abort
fatal: Unable to create '/home/fperez/ipython/ipython/.git/index.lock': File exists.

If no other git process is currently running, this probably means a
git process crashed in this repository earlier. Make sure no other git
process is running and remove the file manually to continue.
(termzmq)amirbar[ipython]> rm .git/ind

Very strange...

In any case, thanks! I'll keep a close eye on this version of git, in case it turns out I have to file a git bug report (for reference, it's git 1.7.5.4 from ubuntu 11.10).

Back to ipython...

The test suite almost passes, but I'm getting these really strange errors:

======================================================================
ERROR: Failure: ImportError (No module named IPython)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 71, in importFromDir
    fh, filename, desc = find_module(part, path)
ImportError: No module named IPython

======================================================================
ERROR: Failure: ImportError (No module named IPython)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 71, in importFromDir
    fh, filename, desc = find_module(part, path)
ImportError: No module named IPython

======================================================================
ERROR: Failure: ImportError (No module named IPython)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 71, in importFromDir
    fh, filename, desc = find_module(part, path)
ImportError: No module named IPython

======================================================================
ERROR: Failure: ImportError (No module named IPython)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.7/nose/importer.py", line 71, in importFromDir
    fh, filename, desc = find_module(part, path)
ImportError: No module named IPython

----------------------------------------------------------------------

Do you see those too? They're in iptest IPython.frontend.

Min RK
Owner

Ah, I used to see those - I don't know why they happen, but it seems like it tries to do something multithreaded, and can hit a race condition when rebasing many commits. I always git rebase -i now, and for some reason that seems to solve the issue.

As for iptest errors - no, I do not see any errors, the test suite passes just fine.

Fernando Perez
Owner
Min RK
Owner

The %kernel frontend-magic stuff hasn't been implemented, but I've been thinking, and in my experience, most of the time that I want to kill the kernel, I want to do it when it's not responding, and a magic doesn't help.

Other than that, I can't think of anything else that jumped out.

Min RK
Owner

@fperez - raw_input/%debug should now be better behaved.

Do we want to block on the frontend-magics? Note that with the design we have proposed, the magics will only function if the frontend believes the kernel to be fully functional and waiting for input. That seems to completely eliminate the value of %kernel interrupt, and also means there's no difference between %kernel kill and os._exit(0) unless there is a bug in the frontend.

I don't think it's particularly useful to be able to signal the kernel only when it's perceived to be fully functional and responsive.

and others added some commits
Omar Andres Zapata Mesa basic kernelmanager and frontend wrote, support indentation but it do…
…nt run code yet
295d085
Omar Andres Zapata Mesa working in handlers 9a9fc06
Omar Andres Zapata Mesa working in tab completion e1e6feb
Omar Andres Zapata Mesa completer not working fine 447dc20
Omar Andres Zapata Mesa little bug fixed in kernelmanager's queues 9d266d4
Omar Andres Zapata Mesa raw_input captured and working fine 916cfd6
Omar Andres Zapata Mesa -mworking in tab-completion 4182704
Omar Andres Zapata Mesa tab completion is not working yet, unknow error 48461d3
Thomas Kluyver Make readline tab-completion work in two-process terminal frontend. 91df35a
Omar Andres Zapata Mesa little bug fixed in pyout message print. fdbcb3d
Omar Andres Zapata Mesa bug fixed prompt count using differents clients a7c91ab
Omar Andres Zapata Mesa traceback support added 25d06b0
Omar Andres Zapata Mesa bug fixed in prompt count after traceback 5f47482
Thomas Kluyver Replace tabs with spaces c9b58bd
Thomas Kluyver Separate out frontend.zmqterminal package. 2ebf9e3
Thomas Kluyver zmqterminal frontend now uses IPythonInputSplitter, and non-ascii cha…
…racters work.
b0e2b84
Thomas Kluyver Minor tidying up of zmqterminal.frontend 7b9b46e
Thomas Kluyver Minor tidy up of zmqterminal.completer 8d3b96f
Thomas Kluyver Refactor and simplification of zmqterminal. 1f9a029
Thomas Kluyver Simplify handling of messaging in zmqterminal. 2620844
Thomas Kluyver Nicer prompt formatting in zmqterminal, and use print_function. 6f1b48d
Min RK rebased and updated to master abe8bd2
Min RK zmqterminal subclasses TerminalInteractiveShell/IPApp
this does a lot of the boiler plate for us, so Omar's communication
code is used in place of run_cell / complete.

Now colors, prompts, indentation, cl-args, etc. are all inherited
from regular existing code.

keyboard interrupts don't work yet, the kernel just dies...
8360d3a
Min RK protect execution_count from longs returned by json 129880a
Paul Ivanov ignore KeyboardInterrupt until we can handle it fcf7cd5
Paul Ivanov refactoring common code out of qtconsole 24cd16e
Paul Ivanov zmqterminal now uses the zmq mixin app 2cecbf1
Paul Ivanov restore KeyboardInterrupt capability c887ab9
Paul Ivanov Out[] prompt now reflects updated execution_count 812a82f
Paul Ivanov documentation update da159e1
Paul Ivanov report inability to signal --existing kernels 31d0eac
Min RK improve cleanup of connection files
* allow clean shutdown of qtconsole on sigint
* init_kernel_manager registers cleanup_connection_file with atexit
* connection_file trait is guaranteed to be abspath at end of init_connection_file(), so find methods are no longer necessary after this call.
* move cf removal from KM.__del__ to KM.cleanup_connection_file
* _new_connection_file only uses last segment of uuid, not the whole thing, which is ugly and overkill 2**48 is enough kernels.
b8f41d5
Min RK update flags&aliases for two-process apps
swallow_args now appropriately removes only frontend-specific args,
and the apps parse front and backend args correctly
37a13a2
Min RK Move IPython.frontend.zmqterminal to IPython.frontend.terminal.console fd34263
Min RK ignore sigint during poll in kernel
Prevents KeyboardInterrupt caught in Kernel message under most
circumstances
796d78c
Min RK Don't install payload-pager at the module-level in zmqshell
The payload pager should be installed at shell instantiation, otherwise
any code that might import zmqshell that is not itself a kernel, will have broken page.page.
9277d76
Min RK Fixes to the heartbeat channel
* The heartbeat channel had some erroneous zeromq logic, and entirely False comments (as described in #967).  This has been fixed.

* KernelManager.is_alive() checks if the hb_channel is running if
the kernel is not owned, rather than always returning True.

* BlockingKM's hb_channel has been relaxed to 1s polling, because replies are not
reliably much faster than that.  There are occasional >0.5s outlier responses.
f7e44e6
Min RK Adjustment to console signal-handling
Do not explicitly interrupt the kernel, because the subprocess
already gets the interrupt.  Once we properly prevent this, then
the interrupt can be restored.

Also, the KeyboardInterrupt is still raised if not executing, in order
to restore the prompt.

In ipkernel, the interrupt is ignored during poll, and restored after
the loop exits, to prevent it being swallowed in future code that
expects normal sigint behavior.
9a0989a
Min RK two-process interactive shell progress
* execute payloads (e.g. page/pinfo output) are displayed and paged.
* aborted replies don't crash frontend.
* intermediate iopub is displayed as it comes
* wait for kernel to start before printing first prompt.
  This gets the right number on the first prompt, and also
  avoids the log output from being drawn after the first prompt.
* protect most of the interact block from keyboard interrupts,
  which could cause weird errors if ctrl-C was held for a while.
* separate restart/wait prompt when kernel dies.
7431cd6
Min RK bring multiline-hist fix from trunk into zmq console ca12bca
Min RK rename IPythonMixinConsoleApp to IPythonConsoleApp
this will make itself more friendly to cofig
a345811
Min RK don't use private TestCase._exc_info() method in parametric test
In Python 2.6, the method is one line, returning sys.exc_info(), and has been removed in Python 2.7.
d133113
Min RK Split swallow_argv into standalone function in lib.kernel
adds frontend_flags/aliases traits to ConsoleApp derivatives for
use in this function.
5e833cd
Min RK remove unused console/frontend.py ce04352
Min RK [termconsole] raw_input improvements
* allow EOF to be forwarded
* flush iopub in more appropriate places
* handle interrupting raw_input better
67a9e3a
Fernando Perez
Owner

Mmh, something broke here, I typed ipython console and I got a never-ending stream of these error messages:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/home/fperez/usr/lib/python2.7/site-packages/IPython/utils/ipstruct.py in __getattr__(self, key)
    149             result = self[key]
    150         except KeyError:
--> 151             raise AttributeError(key)
    152         else:
    153             return result

AttributeError: generate_prompt
---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
UnboundLocalError: local variable 'prompt' referenced before assignment
---------------------------------------------------------------------------

Does the console work for you?

This tells me that we should, at the very least, add a test that starts a console, runs something simple, and shuts it down. We could do it using pexpect, actually...

Min RK
Owner

Of course, this was the first rebase since the PromptManager. Should be a quick fix.

Min RK
Owner

PromptManager issue fixed, I'll add a simple test.

Min RK
Owner

simple pexpect test added as well

Fernando Perez
Owner

A few things:

  • prompt spacing is wrong, compare:
In [1]: import sys
In [2]: 'hello ipython'
Out[2]: 'hello ipython'
In [3]:

to

In [1]: import sys

In [2]: 'hello ipython'
Out[2]: 'hello ipython'

In [3]: 
  • plain exit doesn't cause an exit (which does work on the 2-process qt console).

  • Calling with --pylab causes crash:

Traceback (most recent call last):
  File "/home/fperez/usr/bin/ipython", line 7, in <module>
    launch_new_instance()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/ipapp.py", line 403, in launch_new_instance
    app.start()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/ipapp.py", line 373, in start
    return self.subapp.start()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/ipapp.py", line 377, in start
    self.shell.mainloop()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/console/interactiveshell.py", line 205, in mainloop
    self.interact(display_banner=display_banner)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/console/interactiveshell.py", line 250, in interact
    if not self.wait_for_kernel(3):
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/console/interactiveshell.py", line 220, in wait_for_kernel
    self.run_cell('1', False)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/console/interactiveshell.py", line 96, in run_cell
    self.handle_iopub()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/frontend/terminal/console/interactiveshell.py", line 145, in handle_iopub
    if self.session_id == sub_msg['parent_header']['session']:
KeyError: 'session'
Min RK
Owner

Interesting, because I see none of those issues. The KeyError I know is a bug that just doesn't happen to come up for me, but I'll fix it. However, it is strange that you have incorrect prompt spacing and exit doesn't work, when they behave perfectly for me. Do you have any related config?

Min RK
Owner

The KeyError should be fixed, but I don't know how to deal with the others since I can't reproduce them.

Fernando Perez
Owner

Ok, that's truly bizarre: no special config, just tried by making a clean, fresh profile and running with that one. Still see all the above problems.

Fernando Perez
Owner

Confirming that your last commit fixes the KeyError, now I can start with --pylab. I guess I'll have a crack at the others later on, gotta switch gears right now... You don't see them even on a linux box? I made sure to nuke all stale .pyc files, etc, so I'm surprised you're getting different behavior.

Min RK
Owner

I just tried it on Linux, and now I do see both. I wonder how they would be Linux-specific?

Fernando Perez
Owner

Beats me, that's really odd. But at least it's not just little green men on my box :)

Min RK
Owner

No, I am somehow completely crazy. It's new since yesterday after the rebase, so I was using the old version. The prompt separation should be fixed now.

I don't know why exit would stop working. Any idea?

Fernando Perez
Owner

Yup, prompts look fine now. No clue on the exit not working, b/c Control-D still does work, so our basic exiting machinery isn't really broken.

Min RK
Owner

I don't know how exit stopped being handled, but it should work now.

The main remaining issue is that the SIGINT catching machinery doesn't really work when a GUI is integrated (only affects owning frontend, not --existing).

Fernando Perez
Owner

Well, everything is looking good enough to merge. In your question above about the magics, did you mean whether we should block merging this until we sort that out, or whether the frontend should block on frontend magics?

Min RK
Owner

The former - should we block merge

Fernando Perez
Owner

I think we can merge now, and sort out what we want for %kernel later on. The earlier this goes in, the more chances we'll have of seeing any unintended ill side-effects.

Regarding %kernel, I think the main one would be %kernel restart, where one could get a fresh kernel without stopping the interactive session. Granted, it's a minimal difference to doing os._exit(); up-arrow-enter, but it may feel a little cleaner.

But precisely since it's small polish, let's worry about it post-merge. The rest of this is too useful and important not to have it in now.

I'm merging it; a huge thanks to you, @ivanov, @omazapa and everyone else who worked on this! @omazapa, you see how it took a long time but finally your work is in, and it will be part of the new 0.12 release :)

Fernando Perez fperez merged commit afa2b63 into from
Fernando Perez fperez closed this
Fernando Perez fperez referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 50 unique commits by 4 authors.

Dec 05, 2011
Omar Andres Zapata Mesa basic kernelmanager and frontend wrote, support indentation but it do…
…nt run code yet
295d085
Omar Andres Zapata Mesa working in handlers 9a9fc06
Omar Andres Zapata Mesa working in tab completion e1e6feb
Omar Andres Zapata Mesa completer not working fine 447dc20
Omar Andres Zapata Mesa little bug fixed in kernelmanager's queues 9d266d4
Omar Andres Zapata Mesa raw_input captured and working fine 916cfd6
Omar Andres Zapata Mesa -mworking in tab-completion 4182704
Omar Andres Zapata Mesa tab completion is not working yet, unknow error 48461d3
Thomas Kluyver Make readline tab-completion work in two-process terminal frontend. 91df35a
Omar Andres Zapata Mesa little bug fixed in pyout message print. fdbcb3d
Omar Andres Zapata Mesa bug fixed prompt count using differents clients a7c91ab
Omar Andres Zapata Mesa traceback support added 25d06b0
Omar Andres Zapata Mesa bug fixed in prompt count after traceback 5f47482
Thomas Kluyver Replace tabs with spaces c9b58bd
Thomas Kluyver Separate out frontend.zmqterminal package. 2ebf9e3
Thomas Kluyver zmqterminal frontend now uses IPythonInputSplitter, and non-ascii cha…
…racters work.
b0e2b84
Thomas Kluyver Minor tidying up of zmqterminal.frontend 7b9b46e
Thomas Kluyver Minor tidy up of zmqterminal.completer 8d3b96f
Thomas Kluyver Refactor and simplification of zmqterminal. 1f9a029
Thomas Kluyver Simplify handling of messaging in zmqterminal. 2620844
Thomas Kluyver Nicer prompt formatting in zmqterminal, and use print_function. 6f1b48d
Min RK rebased and updated to master abe8bd2
Min RK zmqterminal subclasses TerminalInteractiveShell/IPApp
this does a lot of the boiler plate for us, so Omar's communication
code is used in place of run_cell / complete.

Now colors, prompts, indentation, cl-args, etc. are all inherited
from regular existing code.

keyboard interrupts don't work yet, the kernel just dies...
8360d3a
Min RK protect execution_count from longs returned by json 129880a
Paul Ivanov ignore KeyboardInterrupt until we can handle it fcf7cd5
Paul Ivanov refactoring common code out of qtconsole 24cd16e
Paul Ivanov zmqterminal now uses the zmq mixin app 2cecbf1
Paul Ivanov restore KeyboardInterrupt capability c887ab9
Paul Ivanov Out[] prompt now reflects updated execution_count 812a82f
Paul Ivanov documentation update da159e1
Paul Ivanov report inability to signal --existing kernels 31d0eac
Min RK improve cleanup of connection files
* allow clean shutdown of qtconsole on sigint
* init_kernel_manager registers cleanup_connection_file with atexit
* connection_file trait is guaranteed to be abspath at end of init_connection_file(), so find methods are no longer necessary after this call.
* move cf removal from KM.__del__ to KM.cleanup_connection_file
* _new_connection_file only uses last segment of uuid, not the whole thing, which is ugly and overkill 2**48 is enough kernels.
b8f41d5
Min RK update flags&aliases for two-process apps
swallow_args now appropriately removes only frontend-specific args,
and the apps parse front and backend args correctly
37a13a2
Min RK Move IPython.frontend.zmqterminal to IPython.frontend.terminal.console fd34263
Min RK ignore sigint during poll in kernel
Prevents KeyboardInterrupt caught in Kernel message under most
circumstances
796d78c
Min RK Don't install payload-pager at the module-level in zmqshell
The payload pager should be installed at shell instantiation, otherwise
any code that might import zmqshell that is not itself a kernel, will have broken page.page.
9277d76
Min RK Fixes to the heartbeat channel
* The heartbeat channel had some erroneous zeromq logic, and entirely False comments (as described in #967).  This has been fixed.

* KernelManager.is_alive() checks if the hb_channel is running if
the kernel is not owned, rather than always returning True.

* BlockingKM's hb_channel has been relaxed to 1s polling, because replies are not
reliably much faster than that.  There are occasional >0.5s outlier responses.
f7e44e6
Min RK Adjustment to console signal-handling
Do not explicitly interrupt the kernel, because the subprocess
already gets the interrupt.  Once we properly prevent this, then
the interrupt can be restored.

Also, the KeyboardInterrupt is still raised if not executing, in order
to restore the prompt.

In ipkernel, the interrupt is ignored during poll, and restored after
the loop exits, to prevent it being swallowed in future code that
expects normal sigint behavior.
9a0989a
Min RK two-process interactive shell progress
* execute payloads (e.g. page/pinfo output) are displayed and paged.
* aborted replies don't crash frontend.
* intermediate iopub is displayed as it comes
* wait for kernel to start before printing first prompt.
  This gets the right number on the first prompt, and also
  avoids the log output from being drawn after the first prompt.
* protect most of the interact block from keyboard interrupts,
  which could cause weird errors if ctrl-C was held for a while.
* separate restart/wait prompt when kernel dies.
7431cd6
Min RK bring multiline-hist fix from trunk into zmq console ca12bca
Min RK rename IPythonMixinConsoleApp to IPythonConsoleApp
this will make itself more friendly to cofig
a345811
Min RK don't use private TestCase._exc_info() method in parametric test
In Python 2.6, the method is one line, returning sys.exc_info(), and has been removed in Python 2.7.
d133113
Min RK Split swallow_argv into standalone function in lib.kernel
adds frontend_flags/aliases traits to ConsoleApp derivatives for
use in this function.
5e833cd
Min RK remove unused console/frontend.py ce04352
Min RK [termconsole] raw_input improvements
* allow EOF to be forwarded
* flush iopub in more appropriate places
* handle interrupting raw_input better
67a9e3a
Dec 06, 2011
Min RK update two-process console with prompt_manager changes 59e0d5b
Min RK cleanup dangling kernel in test_kernelsession ba0b0db
Min RK add single two-process terminal test 7db0cae
Min RK protect term console from empty parent header (startup messages) 64d52d5
Min RK handle exit in `ipython console` 1205382
Something went wrong with that request. Please try again.