Skip to content
This repository

ZMQ terminal frontend #433

Closed
wants to merge 27 commits into from

3 participants

Omar Andres Zapata Mesa Thomas Kluyver Min RK
Omar Andres Zapata Mesa
omazapa commented

Frontend for two process model.
Done:
->Some KernelManager's Hendlers implemented
->Prompt Numeration for inputs and outputs
->Indentation
->All command line arguments for connection with a opened kernel eg: frontend -e --xreq 52797 --sub 38906 --rep 40895 --hb 4320
->Tab-completion bugs fixed (Thanks to Thomas)
->raw_input captured
->Print beautiful tracebacks
TODO
->Support colors in outputs
->Support Crtl+C to send the signal SIGINT
->Heartbeat's verification.

Thanks!

Thomas Kluyver
Collaborator

An organisational detail - would it make sense to put these files in a separate module under frontend? Something like IPython/frontend/zmq-terminal?

Omar Andres Zapata Mesa
omazapa commented

I think that is good idea.

Thomas Kluyver
Collaborator

We should also have a script that gets installed so we can start it easily. Something like ipython-zmqterm?

Thomas Kluyver
Collaborator

Oh, and unicode input! a = u"€" blows it up at the moment.

Thomas Kluyver
Collaborator

Would it make sense to subclass TerminalInteractiveShell, and overload the relevant methods (like run_cell?) A lot of the stuff like readline handling, inputsplitter, displaying prompts, and so on, is going to be the same.

Min RK
Owner
minrk commented

I've updated this to subclass the InteractiveShell and Application from the regular terminal frontend. This means the configuration, colors, prompts, etc. are all inherited, but the underlying execution and completion code you wrote is the same.

See here.

Still some work to be done, particularly wrt KeyboardInterrupts, but basic functionality is working.

Thomas Kluyver
Collaborator

Closing this one, as it's included in Min's PR #708.

Thomas Kluyver takluyver closed this
Damián Avila damianavila referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 27 unique commits by 2 authors.

Jan 29, 2011
Omar Andres Zapata Mesa basic kernelmanager and frontend wrote, support indentation but it do…
…nt run code yet
4ec0f65
Feb 05, 2011
Omar Andres Zapata Mesa working in handlers ec97b10
Mar 29, 2011
Omar Andres Zapata Mesa working in tab completion 401e8f2
Omar Andres Zapata Mesa completer not working fine 0fd201a
Omar Andres Zapata Mesa little bug fixed in kernelmanager's queues b933d68
Apr 03, 2011
Omar Andres Zapata Mesa raw_input captured and working fine dbc83d1
Omar Andres Zapata Mesa -mworking in tab-completion 743a66b
Omar Andres Zapata Mesa tab completion is not working yet, unknow error a86cb3d
May 05, 2011
Thomas Kluyver Make readline tab-completion work in two-process terminal frontend. 2873cab
Omar Andres Zapata Mesa Merge pull request #1 from takluyver/frontend-logging
Frontend logging
b4fdc33
May 08, 2011
Omar Andres Zapata Mesa Merge branch 'master' of git://github.com/ipython/ipython into fronte…
…nd-logging
de718cb
May 12, 2011
Omar Andres Zapata Mesa Merge branch 'master' of git://github.com/ipython/ipython into fronte…
…nd-logging
e8e7303
May 13, 2011
Omar Andres Zapata Mesa little bug fixed in pyout message print. 89f8ca1
Omar Andres Zapata Mesa bug fixed prompt count using differents clients f8c2dae
Omar Andres Zapata Mesa traceback support added a0b1bb0
Omar Andres Zapata Mesa bug fixed in prompt count after traceback f6eaa05
May 15, 2011
Thomas Kluyver Replace tabs with spaces dabac35
Thomas Kluyver Separate out frontend.zmqterminal package. d66d10e
Thomas Kluyver Add ipython-zmqterminal entry point. f226f4f
Thomas Kluyver zmqterminal frontend now uses IPythonInputSplitter, and non-ascii cha…
…racters work.
704cfbc
Thomas Kluyver Minor tidying up of zmqterminal.frontend 2428d9c
Thomas Kluyver Minor tidy up of zmqterminal.completer 77c005a
May 24, 2011
Thomas Kluyver Refactor and simplification of zmqterminal. c0be402
Omar Andres Zapata Mesa Merge pull request #3 from takluyver/refactor-zmqterminal
Refactor zmqterminal
85382a1
May 25, 2011
Thomas Kluyver Simplify handling of messaging in zmqterminal. d8df799
Thomas Kluyver Nicer prompt formatting in zmqterminal, and use print_function. 79c752d
May 26, 2011
Omar Andres Zapata Mesa Merge branch 'master' of git://github.com/ipython/ipython into fronte…
…nd-logging
5809ebf
This page is out of date. Refresh to see the latest.
0  IPython/frontend/zmqterminal/__init__.py
No changes.
41  IPython/frontend/zmqterminal/completer.py
... ...
@@ -0,0 +1,41 @@
  1
+# -*- coding: utf-8 -*-
  2
+import readline
  3
+from Queue import Empty
  4
+
  5
+class ClientCompleter2p(object):
  6
+    """Client-side completion machinery.
  7
+
  8
+    How it works: self.complete will be called multiple times, with
  9
+    state=0,1,2,... When state=0 it should compute ALL the completion matches,
  10
+    and then return them for each value of state."""
  11
+    
  12
+    def __init__(self,client, km):
  13
+        self.km =  km
  14
+        self.matches = []
  15
+        self.client = client
  16
+        
  17
+    def complete_request(self,text):
  18
+        line = readline.get_line_buffer()
  19
+        cursor_pos = readline.get_endidx()
  20
+        
  21
+        # send completion request to kernel
  22
+        # Give the kernel up to 0.5s to respond
  23
+        msg_id = self.km.xreq_channel.complete(text=text, line=line,
  24
+                                                        cursor_pos=cursor_pos)
  25
+        
  26
+        msg_xreq = self.km.xreq_channel.get_msg(timeout=0.5)
  27
+        if msg_xreq['parent_header']['msg_id'] == msg_id:
  28
+            return msg_xreq["content"]["matches"]
  29
+        return []
  30
+    
  31
+    def complete(self, text, state):
  32
+        if state == 0:
  33
+            try:
  34
+                self.matches = self.complete_request(text)
  35
+            except Empty:
  36
+                print('WARNING: Kernel timeout on tab completion.')
  37
+        
  38
+        try:
  39
+            return self.matches[state]
  40
+        except IndexError:
  41
+            return None
262  IPython/frontend/zmqterminal/frontend.py
... ...
@@ -0,0 +1,262 @@
  1
+# -*- coding: utf-8 -*-
  2
+"""Frontend of ipython working with python-zmq
  3
+
  4
+Ipython's frontend, is a ipython interface that send request to kernel and proccess the kernel's outputs.
  5
+
  6
+For more details, see the ipython-zmq design
  7
+"""
  8
+#-----------------------------------------------------------------------------
  9
+# Copyright (C) 2010 The IPython Development Team
  10
+#
  11
+# Distributed under the terms of the BSD License. The full license is in
  12
+# the file COPYING, distributed as part of this software.
  13
+#-----------------------------------------------------------------------------
  14
+
  15
+#-----------------------------------------------------------------------------
  16
+# Imports
  17
+#-----------------------------------------------------------------------------
  18
+from __future__ import print_function
  19
+
  20
+import __builtin__
  21
+import sys
  22
+import os
  23
+from Queue import Empty
  24
+import readline
  25
+import rlcompleter
  26
+
  27
+#-----------------------------------------------------------------------------
  28
+# Imports from ipython
  29
+#-----------------------------------------------------------------------------
  30
+from IPython.external.argparse import ArgumentParser
  31
+from IPython.core.inputsplitter import IPythonInputSplitter
  32
+from IPython.zmq.blockingkernelmanager import BlockingKernelManager as KernelManager
  33
+from IPython.frontend.zmqterminal.completer import ClientCompleter2p 
  34
+
  35
+#-----------------------------------------------------------------------------
  36
+# Network Constants
  37
+#-----------------------------------------------------------------------------
  38
+
  39
+from IPython.utils.localinterfaces import LOCALHOST, LOCAL_IPS
  40
+class Frontend(object):
  41
+    """This class is a simple frontend to ipython-zmq 
  42
+       
  43
+      NOTE: this class uses kernelmanager to manipulate sockets
  44
+      
  45
+      Parameters:
  46
+      -----------
  47
+      kernelmanager : object
  48
+        instantiated object from class KernelManager in module kernelmanager
  49
+        
  50
+    """
  51
+
  52
+    def __init__(self, kernelmanager):
  53
+       self.km = kernelmanager
  54
+       self.session_id = self.km.session.session
  55
+       self.completer = ClientCompleter2p(self, self.km)
  56
+       readline.parse_and_bind("tab: complete")
  57
+       readline.parse_and_bind('set show-all-if-ambiguous on')
  58
+       readline.set_completer(self.completer.complete)
  59
+
  60
+       history_path = os.path.expanduser('~/.ipython/history')
  61
+       if os.path.isfile(history_path):
  62
+           rlcompleter.readline.read_history_file(history_path)
  63
+       else:
  64
+           print("history file cannot be read.")   
  65
+
  66
+       self.messages = {}
  67
+
  68
+       self._splitter = IPythonInputSplitter()
  69
+       self.code = ""
  70
+ 
  71
+       self.prompt_count = 0
  72
+       self._get_initial_prompt()
  73
+ 
  74
+    def _get_initial_prompt(self):
  75
+       self._execute('', hidden=True)
  76
+   
  77
+    def interact(self):
  78
+       """Gets input from console using inputsplitter, then
  79
+       while you enter code it can indent and set index id to any input
  80
+       """    
  81
+       
  82
+       try:
  83
+           print()
  84
+           self._splitter.push(raw_input(' In[%i]: '%self.prompt_count+self.code))
  85
+           while self._splitter.push_accepts_more():
  86
+              self.code = raw_input(' .....: '+' '*self._splitter.indent_spaces)
  87
+              self._splitter.push(' '*self._splitter.indent_spaces+self.code)
  88
+           self._execute(self._splitter.source,False)
  89
+           self._splitter.reset()
  90
+       except KeyboardInterrupt:
  91
+           print('\nKeyboardInterrupt\n')
  92
+           pass
  93
+          
  94
+       
  95
+    def start(self):
  96
+       """Start the interaction loop, calling the .interact() method for each
  97
+       input cell.
  98
+       """
  99
+       while True:
  100
+           try:
  101
+               self.interact()
  102
+           except KeyboardInterrupt:
  103
+                print('\nKeyboardInterrupt\n')
  104
+                pass
  105
+           except EOFError:
  106
+               answer = ''    
  107
+               while True:
  108
+                   answer = raw_input('\nDo you really want to exit ([y]/n)?')
  109
+                   if answer == 'y' or answer == '' :
  110
+                       self.km.shutdown_kernel()
  111
+                       sys.exit()
  112
+                   elif answer == 'n':
  113
+                       break
  114
+
  115
+    def _execute(self, source, hidden = True):
  116
+        """ Execute 'source'. If 'hidden', do not show any output.
  117
+
  118
+        See parent class :meth:`execute` docstring for full details.
  119
+        """
  120
+        msg_id = self.km.xreq_channel.execute(source, hidden)
  121
+        while not self.km.xreq_channel.msg_ready():
  122
+            try:
  123
+                self.handle_rep_channel(timeout=0.1)
  124
+            except Empty:
  125
+                pass
  126
+        self.handle_execute_reply(msg_id)
  127
+
  128
+    def handle_execute_reply(self, msg_id):
  129
+        msg_xreq = self.km.xreq_channel.get_msg()
  130
+        if msg_xreq["parent_header"]["msg_id"] == msg_id:
  131
+            if msg_xreq["content"]["status"] == 'ok' :
  132
+                self.handle_sub_channel()
  133
+               
  134
+            elif msg_xreq["content"]["status"] == 'error':
  135
+                for frame in msg_xreq["content"]["traceback"]:
  136
+                    print(frame, file=sys.stderr)
  137
+            
  138
+            self.prompt_count = msg_xreq["content"]["execution_count"] + 1
  139
+
  140
+
  141
+    def handle_sub_channel(self):
  142
+       """ Method to procces subscribe channel's messages
  143
+
  144
+           This method reads a message and processes the content in different
  145
+           outputs like stdout, stderr, pyout and status
  146
+
  147
+           Arguments:
  148
+           sub_msg:  message receive from kernel in the sub socket channel
  149
+                     capture by kernel manager.
  150
+       """
  151
+       while self.km.sub_channel.msg_ready():
  152
+           sub_msg = self.km.sub_channel.get_msg()
  153
+           if self.session_id == sub_msg['parent_header']['session']:
  154
+               if sub_msg['msg_type'] == 'status' :
  155
+                    if sub_msg["content"]["execution_state"] == "busy" :
  156
+                        pass
  157
+
  158
+               elif sub_msg['msg_type'] == 'stream' :
  159
+                  if sub_msg["content"]["name"] == "stdout":
  160
+                    print(sub_msg["content"]["data"], file=sys.stdout, end="")
  161
+                    sys.stdout.flush()
  162
+                  elif sub_msg["content"]["name"] == "stderr" :
  163
+                    print(sub_msg["content"]["data"], file=sys.stderr, end="")
  164
+                    sys.stderr.flush()
  165
+                
  166
+               elif sub_msg['msg_type'] == 'pyout' :
  167
+                    print("Out[%i]:"%sub_msg["content"]["execution_count"],
  168
+                            sub_msg["content"]["data"]["text/plain"],
  169
+                            file=sys.stdout)
  170
+                    sys.stdout.flush()
  171
+
  172
+    def handle_rep_channel(self, timeout=0.1):
  173
+        """ Method to capture raw_input
  174
+        """
  175
+        msg_rep = self.km.rep_channel.get_msg(timeout=timeout)
  176
+        if self.session_id == msg_rep["parent_header"]["session"] :
  177
+            raw_data = raw_input(msg_rep["content"]["prompt"])
  178
+            self.km.rep_channel.input(raw_data)
  179
+             
  180
+       
  181
+
  182
+       
  183
+def start_frontend():
  184
+    """ Entry point for application.
  185
+    
  186
+    """
  187
+    # Parse command line arguments.
  188
+    parser = ArgumentParser()
  189
+    kgroup = parser.add_argument_group('kernel options')
  190
+    kgroup.add_argument('-e', '--existing', action='store_true',
  191
+                        help='connect to an existing kernel')
  192
+    kgroup.add_argument('--ip', type=str, default=LOCALHOST,
  193
+                        help=\
  194
+            "set the kernel\'s IP address [default localhost].\
  195
+            If the IP address is something other than localhost, then \
  196
+            Consoles on other machines will be able to connect\
  197
+            to the Kernel, so be careful!")
  198
+    kgroup.add_argument('--xreq', type=int, metavar='PORT', default=0,
  199
+                        help='set the XREQ channel port [default random]')
  200
+    kgroup.add_argument('--sub', type=int, metavar='PORT', default=0,
  201
+                        help='set the SUB channel port [default random]')
  202
+    kgroup.add_argument('--rep', type=int, metavar='PORT', default=0,
  203
+                        help='set the REP channel port [default random]')
  204
+    kgroup.add_argument('--hb', type=int, metavar='PORT', default=0,
  205
+                        help='set the heartbeat port [default random]')
  206
+
  207
+    egroup = kgroup.add_mutually_exclusive_group()
  208
+    egroup.add_argument('--pure', action='store_true', help = \
  209
+                        'use a pure Python kernel instead of an IPython kernel')
  210
+    egroup.add_argument('--pylab', type=str, metavar='GUI', nargs='?', 
  211
+                       const='auto', help = \
  212
+        "Pre-load matplotlib and numpy for interactive use. If GUI is not \
  213
+         given, the GUI backend is matplotlib's, otherwise use one of: \
  214
+         ['tk', 'gtk', 'qt', 'wx', 'inline'].")
  215
+    egroup.add_argument('--colors', type=str,
  216
+                        help="Set the color scheme (LightBG,Linux,NoColor). This is guessed\
  217
+                        based on the pygments style if not set.")
  218
+
  219
+    args = parser.parse_args()
  220
+
  221
+    # parse the colors arg down to current known labels
  222
+    if args.colors:
  223
+        colors=args.colors.lower()
  224
+        if colors in ('lightbg', 'light'):
  225
+            colors='lightbg'
  226
+        elif colors in ('dark', 'linux'):
  227
+            colors='linux'
  228
+        else:
  229
+            colors='nocolor'
  230
+    else:
  231
+        colors=None
  232
+
  233
+    # Create a KernelManager and start a kernel.
  234
+    kernel_manager = KernelManager(xreq_address=(args.ip, args.xreq),
  235
+                                     sub_address=(args.ip, args.sub),
  236
+                                     rep_address=(args.ip, args.rep),
  237
+                                     hb_address=(args.ip, args.hb))
  238
+    if not args.existing:
  239
+        # if not args.ip in LOCAL_IPS+ALL_ALIAS:
  240
+        #     raise ValueError("Must bind a local ip, such as: %s"%LOCAL_IPS)
  241
+
  242
+        kwargs = dict(ip=args.ip)
  243
+        if args.pure:
  244
+            kwargs['ipython']=False
  245
+        else:
  246
+            kwargs['colors']=colors
  247
+            if args.pylab:
  248
+                kwargs['pylab']=args.pylab
  249
+        kernel_manager.start_kernel(**kwargs)
  250
+
  251
+    
  252
+    kernel_manager.start_channels()
  253
+ 
  254
+    frontend=Frontend(kernel_manager)
  255
+    return frontend
  256
+
  257
+def main():
  258
+    frontend=start_frontend()
  259
+    frontend.start()
  260
+
  261
+if __name__ == "__main__" :
  262
+    main() 
6  IPython/scripts/ipython-zmqterminal
... ...
@@ -0,0 +1,6 @@
  1
+#!/usr/bin/env python
  2
+# -*- coding: utf-8 -*-
  3
+
  4
+from IPython.frontend.zmqterminal.frontend import main
  5
+
  6
+main()
26  IPython/zmq/blockingkernelmanager.py
@@ -16,6 +16,7 @@
16 16
 
17 17
 # Stdlib
18 18
 from Queue import Queue, Empty
  19
+from threading import Event
19 20
 
20 21
 # Our own
21 22
 from IPython.utils import io
@@ -95,10 +96,31 @@ def get_msgs(self):
95 96
     
96 97
 
97 98
 class BlockingRepSocketChannel(RepSocketChannel):
98  
-    
  99
+    def __init__(self, context, session, address=None):
  100
+        super(BlockingRepSocketChannel, self).__init__(context, session, address)
  101
+        self._in_queue = Queue()
  102
+        
99 103
     def call_handlers(self, msg):
100 104
         #io.rprint('[[Rep]]', msg) # dbg
101  
-        pass
  105
+        self._in_queue.put(msg)
  106
+        
  107
+    def get_msg(self, block=True, timeout=None):
  108
+        "Gets a message if there is one that is ready."
  109
+        return self._in_queue.get(block, timeout)
  110
+        
  111
+    def get_msgs(self):
  112
+        """Get all messages that are currently ready."""
  113
+        msgs = []
  114
+        while True:
  115
+            try:
  116
+                msgs.append(self.get_msg(block=False))
  117
+            except Empty:
  118
+                break
  119
+        return msgs
  120
+    
  121
+    def msg_ready(self):
  122
+        "Is there a message that has been received?"
  123
+        return not self._in_queue.empty()
102 124
 
103 125
 
104 126
 class BlockingHBSocketChannel(HBSocketChannel):
7  setupbase.py
@@ -125,7 +125,8 @@ def find_packages():
125 125
     add_package(packages, 'frontend')
126 126
     add_package(packages, 'frontend.qt')
127 127
     add_package(packages, 'frontend.qt.console', tests=True)
128  
-    add_package(packages, 'frontend.terminal', tests=True)    
  128
+    add_package(packages, 'frontend.terminal', tests=True)
  129
+    add_package(packages, 'frontend.zmqterminal')
129 130
     add_package(packages, 'lib', tests=True)
130 131
     add_package(packages, 'parallel', tests=True, scripts=True, 
131 132
                                     others=['apps','engine','client','controller'])
@@ -277,7 +278,8 @@ def find_scripts(entry_points=False):
277 278
             'iplogger = IPython.parallel.apps.iploggerapp:launch_new_instance',
278 279
             'ipcluster = IPython.parallel.apps.ipclusterapp:launch_new_instance',
279 280
             'iptest = IPython.testing.iptest:main',
280  
-            'irunner = IPython.lib.irunner:main'
  281
+            'irunner = IPython.lib.irunner:main',
  282
+            'ipython-zmqterminal = IPython.frontend.zmqterminal.frontend:main'
281 283
         ]
282 284
         gui_scripts = [
283 285
             'ipython-qtconsole = IPython.frontend.qt.console.ipythonqt:main',
@@ -293,6 +295,7 @@ def find_scripts(entry_points=False):
293 295
                    pjoin(parallel_scripts, 'iplogger'),
294 296
                    pjoin(main_scripts, 'ipython'),
295 297
                    pjoin(main_scripts, 'ipython-qtconsole'),
  298
+                   pjoin(main_scripts, 'ipython-zmqterminal'),
296 299
                    pjoin(main_scripts, 'pycolor'),
297 300
                    pjoin(main_scripts, 'irunner'),
298 301
                    pjoin(main_scripts, 'iptest')
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.