Skip to content
This repository

Prompt manager #507

Merged
merged 12 commits into from over 2 years ago

4 participants

Thomas Kluyver Min RK Fernando Perez Brian E. Granger
Thomas Kluyver
Collaborator

I'm not expecting this to land before 0.11, but I thought I'd get it on the radar.

Every time I've wondered how prompts were produced, I've ended up rather confused. So this is an attempt at cleaning it up, with a new PromptManager class responsible for handling everything to do with the prompts. The critical part is its render method, which assembles the necessary information, then uses the string formatting introduced in Python 2.6 to fill in the prompt template.

I've expanded the definition of 'prompts' to include the auto_rewrite prompt ("------> " by default). So there are now four prompts: input, continuation, output, and rewrite.

This definition of prompts does not include input/output separators. For now, I've left those as attributes of the main InteractiveShell object.

Min RK
Owner
minrk commented June 07, 2011

Thanks, this looks good.

This should be part of the general move removing all prompt/separator/ui-specific code from the InteractiveShell object to frontends, which definitely won't make it into 0.11.

Thomas Kluyver
Collaborator

Rebased to bring it up to date.

Min RK
Owner

Can you hook up the qtconsole prompts to your new PromptManager as part of this?

I think making that change will help us make the right decisions on the PromptManager, since it will require support for things like HTML prompts without ANSI colors, divorcing the prompt from the InteractiveShell object, and executing things like os.getcwdu() either locally or in the kernel.

Thomas Kluyver
Collaborator

Does it make sense to simply generate an HTML prompt in the kernel, and send it over? Or, more generally, have the frontend make a prompt_request with a template (which could be in any format), and the kernel replies with the filled in version?

Min RK
Owner

The kernel should probably know exactly nothing about prompts.

Thomas Kluyver
Collaborator

Pragmatically, though, I suspect it will, at least to the extent of filling in slots in a template.

Min RK
Owner

Possibly, but hopefully not. Any such information should be accessible via the user_expressions/user_variables fields of execute requests.

Thomas Kluyver
Collaborator

OK, that makes more sense, I didn't know about those fields. So the frontend needs to parse the format string, work out which variables it needs, and retrieve them from the backend. I think we still need to think a bit more about how this works, though, because a lot of the details that could go in the prompt aren't accessible from the user namespace without importing extra modules (like os or time).

Fernando Perez
Owner

Yes, the idea was to have frontends request whatever information/expressions they want along with each new prompt (possibly including calls to things like os.getcwd() or time.time(), etc). Then they can build whichever display they want with this. In principle, the same kernel could be monitored by two clients that offer different levels of customizability for the prompts.

So the only thing the kernel supplies always is the execution counter, plus whatever variables or expressions may be requested in this form.

Fernando Perez
Owner

Thomas, this overall looks pretty good, and even if it doesn't get us all the way in terms of restructuring the prompt system as we'd like to, it's probably worth merging now. The prompt rework will likely take a few passes until we have a full separation of duties between kernels (who just handle the namespace) and frontends (who present data from that namespace to the users in various ways, including possibly in a prompt).

But there's one thing we should think about: prompts were the one thing for which we kept using Itpl, because it allows the interpolation of arbitrary python expressions, while the new-style string formatting only allows simple variable name access, nothing else. And I know there are projects out there embedding ipython and making weird prompts that update dynamically with function calls, something like: "${some_function()[somearg]}> ".

So it seems your change removes that ability, right?

Now, I'm not totally against that, as long as we plan to complete the transition into a system that allows users to achieve their goal again, even if by different means (i.e. by asking the kernel for these things and using them to build the prompt).

Min RK
Owner

@fperez the EvalFormatter I used in the parallel templating code is a step towards exactly that. It doesn't support more than super-simple evals, but that shouldn't be hard. It's in utils.text.

For more arbitrary execution, you can do:

class EvalFormatter2(Formatter):
    def get_field(self, name, args, kwargs):
        v = eval(name, kwargs)
        return v, name

fmt = EvalFormatter2()
import os
ns = dict(os=os)
fmt.format(' we are in {os.getcwdu()}...', **ns)
# returns 'we are in /home/you/...'
Thomas Kluyver
Collaborator

Plain string formatting can handle attribute.access and item[access], but specifically doesn't handle function calls. It should be trivial to use Min's EvalFormatter here instead.

Min RK
Owner

Note that neither my EvalFormatter in utils.text nor the one I typed up in 5 minutes above are finished implementations (at least something breaks in both), but it shouldn't be hard to get something that works.

Fernando Perez
Owner

I guess one question is: do we really need to write a new one? I mean, Itpl may be old code, but it works and hasn't needed major updating. So what is the argument for effectively reimplementing itpl?

Thomas Kluyver
Collaborator

Well, just that we can do essentially the same thing with a fraction of the code. Also, Itpl parses the string character-by-character in Python code, so we should get better performance from string formatting. I had the idea of dropping some of our external dependencies, and there are only a couple of places that we actually use Itpl. But I'm not going to push the issue.

Min RK
Owner

I don't think Itpl handles unicode at all. So either we need to reimplement Itpl with real string formatting (I doubt such a formatter would be more than ~20 lines), or we need to fix Itpl for unicode support.

Fernando Perez
Owner

Fair enough, I'm ok with both of these arguments. I just wanted to see them :)

So Thomas, do you want to finish up a full EvalFormatter and we can merge this puppy?

Fernando Perez
Owner

ps - we're on IRC now if you want to have a look at any of this right away.

Thomas Kluyver
Collaborator

I could do with hitting the sack now. @minrk, what breaks in the current implementation of EvalFormatter? And is it a condition that we should care about? The only one I'm currently aware of is that you can't readily put a dict literal ({}) inside an expression in a string (it might work if you double both braces, although I've not checked).

Fernando Perez
Owner

A few quick tests of EvalFormatter2 show that it probably supports everything we need for this PR:

In [2]: from string import Formatter 

class EvalFormatter2(Formatter): 
    def get_field(self, name, args, kwargs): 
        v = eval(name, kwargs) 
        return v, name 

fmt = EvalFormatter2() 
import os 
ns = dict(os=os) 
print fmt.format(' we are in {os.getcwdu()}...', **ns)

we are in /home/fperez/ipython/notebooks...

In [3]: print fmt.format(' we are in {os.getcwdu()+"hi"}...', **ns)

we are in /home/fperez/ipython/notebookshi...

In [5]: print fmt.format(' we are in {os.getcwdu()}, a is {a} and a0 is {a[0]}...', 
                 **ns)

we are in /home/fperez/ipython/notebooks, a is [1, 2] and a0 is 1...

So @takluyver, it seems to me that if you update the code to use this (perhaps providing EvalFormatter2 as utils.text.FulEvalFormatter), we'd be good to go.

Min RK
Owner

Lots of stuff (function calls) don't work in the EvalFormatter in master. EvalFormatter2 seems to work for just about everything, but slicing definitely doesn't work: fmt.format("we are in ...{os.getcwdu()[-10:]}", os=os). It raises a SyntaxError, because the token is split at the colon for some reason, so os.getcwdu()[-10 is what gets eval'd.

Like I said above, I didn't spend more than 5-10 minutes on either of these, and I had never looked at any Formatter code before. So perhaps a more careful eye (and a test suite for various known Itpl use cases) would be prudent.

Fernando Perez
Owner

OK, the issue is that ':' is part of the format specification, see http://www.python.org/dev/peps/pep-3101/. Specifically:

Format Specifiers
    Each field can also specify an optional set of 'format
    specifiers' which can be used to adjust the format of that field.
    Format specifiers follow the field name, with a colon (':')
    character separating the two:

        "My name is {0:8}".format('Fred')

so the parser splits on ':' and returns the stuff after the colon as the format specifier.

Which means that this syntax simply is incompatible with arbitrary Python expressions.

That doesn't mean we can't use it, just that we'd be setting some restrictions on what kinds of things can go in there.

So Thomas, if you can do a little audit of our uses of Itpl and see if a more restrictive version based on EvalFormatter2 (hopefully with a nicer name) isn't a problem, then I have no issues with ditching itpl.

And I hope we'll find the time to complete the prompt machinery for frontends before 0.12.

Min RK
Owner

@fperez, I see, so possibly slicing is out, but it's still possible that even the colon-splitting is implemented in a method that we can easily override. If we can take over right at where the Formatter decides what to do with something inside '{...}', then we should be able to just eval that block, and be done.

I don't see any reason for the new one to not just replace EvalFormatter. I just called the example I tossed up EvalFormatter2 because it was my second 10-minute attempt at writing a formatter that evals code. There should be no need for more than one of these.

Min RK
Owner

Turns out it's actually pretty easy. I looked at the string.Formatter source, and the string docs, and all I have to do is override the _vformat method instead of get_field/get_value like I was doing. Now I can execute arbitrary code, and with a switch interpret ':' for slicing (disabling the format_spec part of fancy-formatting), or leave it as-is, preventing slicing (obviously you can't have both).

I'll do a PR soon, but I'm writing some tests first. For now a preview:

class EvalFormatter(Formatter):
    """A String Formatter that allows evaluation of simple expressions.

    Any time a format key is not found in the kwargs,
    it will be tried as an expression in the kwargs namespace.

    This is to be used in templating cases, such as the parallel batch
    script templates, where simple arithmetic on arguments is useful.

    Examples
    --------

    In [1]: f = EvalFormatter()
    In [2]: f.format('{n/4}', n=8)
    Out[2]: '2'

    In [3]: f.format('{range(3)}')
    Out[3]: '[0, 1, 2]'

    In [4]: f.format('{3*2}')
    Out[4]: '6'

    """

    # should we allow slicing by disabling the format_spec feature?
    allow_slicing = True

    # copied from Formatter._vformat with minor changes to allow eval
    # and replace the format_spec code with 
    def _vformat(self, format_string, args, kwargs, used_args, recursion_depth):
        if recursion_depth < 0:
            raise ValueError('Max string recursion exceeded')
        result = []
        for literal_text, field_name, format_spec, conversion in \
                self.parse(format_string):

            # output the literal text
            if literal_text:
                result.append(literal_text)

            # if there's a field, output it
            if field_name is not None:
                # this is some markup, find the object and do
                #  the formatting

                if self.allow_slicing:
                    # override format spec, to allow slicing:
                    field_name = ':'.join([field_name, format_spec])
                    format_spec = ''

                # eval the contents of the field for the object
                # to be formatted
                obj = eval(field_name, kwargs)

                # do any conversion on the resulting object
                obj = self.convert_field(obj, conversion)

                # expand the format spec, if needed
                format_spec = self._vformat(format_spec, args, kwargs,
                                            used_args, recursion_depth-1)

                # format the object and append to the result
                result.append(self.format_field(obj, format_spec))

        return ''.join(result)

And the difference between this and the base Formatter is tiny. If we disallow slicing, then the EvalFormatter2 above is cleaner and identical to this when allow_slicing=False.

[edit: obviously without the print statement I had initially]

Min RK
Owner

EvalFormatter updated: #716

Just pass the namespace in which you want the code to execute as the kwargs to format.

Fernando Perez
Owner

OK, I've merged #716, so that @takluyver can update this one cleanly, to use the new EvalFormatter that has full slicing support for these situations.

We do need to make one design decision: for these kinds of expressions, do we value more the ability to do slicing or the ability to specify formatting (i.e. using the ':N' syntax for format specification)? I'm leaning towards the latter, both to keep this syntax in sync with normal Python formatting syntax and because fine-tuning formatting seems to be more likely to be useful in producing things like prompts than slicing. And if slicing is needed, people can always hide it behind a little utility function they can write, so it's not like slicing is impossible.

But I'd like to get some consensus before we pull the trigger on the final design for this part.

Thomas Kluyver
Collaborator

I'd lean towards having format string enabled as well. If people do want slicing, they don't even need a utility function: x[slice(1,10)] is equivalent to x[1:10]. To give a particular use case, we could expose the timestamp as a datetime object, which can be formatted using the format string.

Min RK
Owner

It is a hard choice. The reason I discovered that slicing didn't work in the EvalFormatter2 above, is the first example I tried was a truncated version of cwd:

f.format("...{os.getcwd()[-10:]}", os=os)

however

you can actually get slicing by using slice objects, and never need a colon:

f.format("...{os.getcwd()[slice(-10,None,None)]}", os=os)

So you can get slice behavior without colon notation, if slightly inconvenient to type, but you can't get formatting-notation back in any reasonably way if you allow slicing. Further, if we don't want to support the slicing, the three-line EvalFormatter2 implementation above is much simpler, and probably preferable (and functionally identical when allow_slicing=False).

Fernando Perez
Owner

OK, it seems we're in agreement that give how slice(a,b,c) is an option (if slightly less convenient), that's the way to go. In that case, we should perhaps go with the really simple EvalFormatter2 (renamed) and be done with it.

It's still useful to have the full one @minrk wrote, as it gives us a replacement for itpl with the modern python syntax and unicode support, but for this we should probably stick to the easy, comprehensible 3-liner.

Thomas Kluyver
Collaborator

The only other place we're using Itpl is for expanding variables in %magic and !system calls. My gut feeling is that slicing is more useful than format strings there, but on the other hand we may want to keep closer to the standard Python format as people get more used to that.

@minrk: Should EvalFormatter2 replace the existing EvalFormatter that we shipped with 0.11? I think the main difference is that the existing EvalFormatter will treat {0} as referring to a positional argument, but EvalFormatter2 will treat it as a literal 0.

Min RK
Owner

I think the EvalFormatter has no need to support positional arguments. I would just replace the 0.11 EvalFormatter with EvalFormatter2 under the EvalFormatter name, and keep the big one that supports splitting around as a separate class (FullEvalFormatter, RichEvalFormatter, or something), just in case there is a desire to split args somewhere down the line.

I think of the EvalFormatter as evaluating code in a namespace. Positional args don't really make sense in that context.

The Itpl syntax is really much nicer for expanding variables (especially in the shell profile when it is restored), but it's not very Python, and as we have discovered doesn't play nice with unicode.

Fernando Perez
Owner

@takluyver, did you see my message on-list about the idea of a frontend_ns to handle this issue? The more I think about it, the more it seems it's the way to go, as it gives us a clean solution to the prompt namespace question as well as a place for other things frontends may legitimately want to do in isolation from the user's namespace.

Now, we could go ahead and finish merging this before tackling that to ensure this doesn't bitrot away... What's your take?

Thomas Kluyver
Collaborator

Hi Fernando, I did see your message - sorry, I've been a bit busy for the last week or so.

I think a frontend_ns makes sense, but a few questions about it:
1. Do we add frontend_variables & frontend_expressions fields to the execute request message? Or add another message type for accessing those?
2. What goes in the frontend_ns initially? I guess os and sys are obvious choices, along with the cwd-truncating functions currently defined for prompts. And something for timestamps. Anything else?
3. How can the frontend manipulate what's in the frontend_ns, e.g. for importing extra stuff? Can it send arbitrary Python code to be executed? Or a specific way to request an import? Or a way to "push" variables, perhaps from the user_ns (or even from the frontend, if it's in Python)?

As for merging this now - do you think this is the general structure of what we want to go forward with? It has the advantage that it makes the prompts code much more tractable for other people to work on. But if large parts are going to be redone, it might lead people down the wrong track. First, though, I need to sort out the various EvalFormatters. I'll get onto that now.

Thomas Kluyver
Collaborator

I've reintroduced the very simple EvalFormatter, and made the slicing-enabled one FullEvalFormatter.

Fernando Perez
Owner

No worries :)

  1. I'm thinking right now that the cleanest approach would be to use the standard execute request message, but simply add an optional field to it indicating the target namespace. Normal messages would go (as they do now) to the user_ns, but if frontend_ns is specified as the target, then they would get executed there.

  2. I'd say we don't put anything in there other than perhaps our prompt-manipulation machinery (since that's an obvious use case). With 1 above, frontends can simply execute import statements.

  3. Via 1, it's taken care of.

Currently our spec has the notion of user_variables and user_expressions. Those would be interpreted as variables and expressions in the execution namespace (perhaps we should change these names accordingly).

So basically, all we'd need is to add a new optional field, namespace, to the execute_request message. And the behavior would be such that if namespace != user_ns, then silent=True automatically, so that the user counter isn't auto-incremented behind the user's back.

How does this sound?

Thomas Kluyver
Collaborator

So each namespace would have a standard string name, e.g. "user_local"? That seems a reasonable way to do it. Although I should note that technically you evaluate or execute code in two namespaces - local and global - just to complicate matters a little.

On the other hand, @ellisonbg has a point (from his email) that this should probably be kept as simple as possible, because we don't want the burden of complex code for a fairly minor feature. Is there a simpler way to handle prompts for the frontends? The idea I had previously was that the frontend would send a prompt template, and the kernel would return a formatted prompt (although the frontend still needs some flexibility so it can update prompt numbers).

Fernando Perez
Owner
Thomas Kluyver
Collaborator
Fernando Perez
Owner
Thomas Kluyver
Collaborator
Brian E. Granger
Owner

I think there are some good points here. If multiple frontends want to watch the value of a variable, then this data needs to be published on the iopub channel. There would need to be a way of registering variables that get published in this way after each execution. Also, there would need to be a way for those variables to become modified. For that the simplest thing is to use the user_ns, but you might want to compute some expression each time a variable is published (like os.getcwd()).

Min RK
Owner

Brian,

What you are describing does indeed sound quite useful, and would suggest something along the lines of a 'state change' message. frontends could register names or expressions to check, and a check_state() call is made at the end of each execute request, publishing its results. Of course, this could add arbitrarily expensive computations to every execute request.

Brian E. Granger
Owner

I am also thinking about how we can design things to enable "Manipulate" style rich widgets to exist in the browser and receive updates as a computation progresses. This type of design might help in that context as well.

Min RK
Owner

Do we want to merge this PromptManager work into 0.12, since it seems like the user_expressions / zmq frontend prompts still has some discussion to resolve?

Fernando Perez
Owner

@takluyver, just noticed this one needs a rebase... I can work on it later if you want, we might be able to flush this and your other two before 0.12...

Thomas Kluyver
Collaborator

Rebased, run the tests on 2.7 and 3.2.

Fernando Perez
Owner

@takluyver, I was going to start testing this puppy but a conflict seems to have appeared between yesterday and today, sorry... The conflict is tiny and I can fix it here while testing, but you may want to rebase this guy.

Fernando Perez
Owner

Also note that the test suite doesn't pass anymore. Make sure to clear your .pyc files first to see the problem. It's the same reason as the conflict, simply that lib.pylabtools got moved to core.pylabtools.

Fernando Perez
Owner

Sorry about those, it was my emergency merge last night that caused these. Fortunately it's a very easy fix.

Fernando Perez
Owner

Other than the above small problems, here's a proposal for moving forward with this one. The code looks solid, and it's indeed already an improvement and cleanup of the prompt machinery, so I don't see a reason for holding it back further, risking more (and possibly non-trivial) merge conflicts popping up.

The only thing I see missing is a good description of the changes in the docs, since the format of the special expressions for prompts has now changed as we move away from itpl.

As far as the larger discussion on the whole frontend_ns ideas, we should probably not rush into it. @ellisonbg's caution is often wisely guided, and I find @minrk's argument about non-atomicity of a separate request and the possibility of it being 'backed up' in a busy kernel very compelling.

The whole question of how to let multiple clients interact with a kernel regarding 'sideband' information (i.e. stuff beyond the raw execution of pure code blocks for the user) is a very tricky design problem, and we're just feeling our way through it. So it's probably best to go in small steps and see what works. We can keep thinking about how to serve those needs slowly, and for now the user_var,exp system will probably be sufficient in most cases. We might add some functionality to register variables protected of deletion in %reset so that people with special prompts can safely use %reset.

How does that sound? We could move forward with this (pending rebase and docs) and then we'll see how things work out in practice with the prompt system, and whether we can continue to work just with user_vars/exp or not. Given we have no actual use of even those components yet, it may be premature to extend the system even further.

Thomas Kluyver
Collaborator

I've just rebased. I've done my best to clear .pyc files, and import IPython.lib.pylabtools fails, but I'm not seeing any test failure. I'll look into docs later.

Fernando Perez
Owner

Yes, pylabtools was moved to core. But that change was made in master already, so a rebase should pick it up for you; did it not?

Thomas Kluyver
Collaborator
Thomas Kluyver
Collaborator

Just to mention: I haven't got time to go through the docs tonight, but hopefully I can get to it tomorrow.

Fernando Perez
Owner

OK, yes: working correctly is what you should expect. No problem with the docs, thanks for taking care of them.!

Thomas Kluyver
Collaborator

I've updated the docs; let me know if you spot any other bits that need to be updated.

Still to do:

  • Update pysh profile to use new prompt config system.
  • Work out why %config doesn't seem to see PromptManager. Probably a trivial fix.
Thomas Kluyver
Collaborator

I've tidied those bits up - I think this is all OK now.

Fernando Perez
Owner

Is the test suite passing for you? I'm having some very bizarre problems with the test suite right now on my box at work, but they are odd enough that I think it's my problem and nothing with the codebase. What do you get?

Thomas Kluyver
Collaborator
Fernando Perez
Owner

OK, great. Let's merge this then as-is, and we'll continue mulling the deeper frontend namespace questions later. No point in holding this any further, lots of work has gone into it. Thanks!

Fernando Perez fperez merged commit 272152c into from November 29, 2011
Fernando Perez fperez closed this November 29, 2011
Thomas Kluyver
Collaborator

Thanks, Fernando. It's an odd feeling, not having an open pull request against IPython ;-).

Fernando Perez
Owner
Fernando Perez fperez referenced this pull request November 30, 2011
Closed

PromptManager fixes #1078

Fernando Perez fperez referenced this pull request from a commit January 10, 2012
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
8  IPython/config/profile/pysh/ipython_config.py
@@ -5,11 +5,11 @@
5 5
 # and merge it into the current one.
6 6
 load_subconfig('ipython_config.py', profile='default')
7 7
 
8  
-c.InteractiveShell.prompt_in1 = r'\C_LightGreen\u@\h\C_LightBlue[\C_LightCyan\Y1\C_LightBlue]\C_Green|\#> '
9  
-c.InteractiveShell.prompt_in2 = r'\C_Green|\C_LightGreen\D\C_Green> '
10  
-c.InteractiveShell.prompt_out = r'<\#> '
  8
+c.PromptManager.in_template = r'{color.LightGreen}\u@\h{color.LightBlue}[{color.LightCyan}\Y1{color.LightBlue}]{color.Green}|\#> '
  9
+c.PromptManager.in2_template = r'{color.Green}|{color.LightGreen}\D{color.Green}> '
  10
+c.PromptManager.out_template = r'<\#> '
11 11
 
12  
-c.InteractiveShell.prompts_pad_left = True
  12
+c.PromptManager.justify = True
13 13
 
14 14
 c.InteractiveShell.separate_in = ''
15 15
 c.InteractiveShell.separate_out = ''
86  IPython/core/displayhook.py
@@ -25,7 +25,6 @@
25 25
 import __builtin__
26 26
 
27 27
 from IPython.config.configurable import Configurable
28  
-from IPython.core import prompts
29 28
 from IPython.utils import io
30 29
 from IPython.utils.traitlets import Instance, List
31 30
 from IPython.utils.warn import warn
@@ -34,32 +33,20 @@
34 33
 # Main displayhook class
35 34
 #-----------------------------------------------------------------------------
36 35
 
37  
-# TODO: The DisplayHook class should be split into two classes, one that
38  
-# manages the prompts and their synchronization and another that just does the
39  
-# displayhook logic and calls into the prompt manager.
40  
-
41  
-# TODO: Move the various attributes (cache_size, colors, input_sep,
42  
-# output_sep, output_sep2, ps1, ps2, ps_out, pad_left). Some of these are also
43  
-# attributes of InteractiveShell. They should be on ONE object only and the
44  
-# other objects should ask that one object for their values.
  36
+# TODO: Move the various attributes (cache_size, [others now moved]). Some
  37
+# of these are also attributes of InteractiveShell. They should be on ONE object
  38
+# only and the other objects should ask that one object for their values.
45 39
 
46 40
 class DisplayHook(Configurable):
47 41
     """The custom IPython displayhook to replace sys.displayhook.
48 42
 
49 43
     This class does many things, but the basic idea is that it is a callable
50 44
     that gets called anytime user code returns a value.
51  
-
52  
-    Currently this class does more than just the displayhook logic and that
53  
-    extra logic should eventually be moved out of here.
54 45
     """
55 46
 
56 47
     shell = Instance('IPython.core.interactiveshell.InteractiveShellABC')
57 48
 
58  
-    def __init__(self, shell=None, cache_size=1000,
59  
-                 colors='NoColor', input_sep='\n',
60  
-                 output_sep='\n', output_sep2='',
61  
-                 ps1 = None, ps2 = None, ps_out = None, pad_left=True,
62  
-                 config=None):
  49
+    def __init__(self, shell=None, cache_size=1000, config=None):
63 50
         super(DisplayHook, self).__init__(shell=shell, config=config)
64 51
 
65 52
         cache_size_min = 3
@@ -75,36 +62,10 @@ def __init__(self, shell=None, cache_size=1000,
75 62
             self.do_full_cache = 1
76 63
 
77 64
         self.cache_size = cache_size
78  
-        self.input_sep = input_sep
79 65
 
80 66
         # we need a reference to the user-level namespace
81 67
         self.shell = shell
82  
-
83  
-        # Set input prompt strings and colors
84  
-        if cache_size == 0:
85  
-            if ps1.find('%n') > -1 or ps1.find(r'\#') > -1 \
86  
-                   or ps1.find(r'\N') > -1:
87  
-                ps1 = '>>> '
88  
-            if ps2.find('%n') > -1 or ps2.find(r'\#') > -1 \
89  
-                   or ps2.find(r'\N') > -1:
90  
-                ps2 = '... '
91  
-        self.ps1_str = self._set_prompt_str(ps1,'In [\\#]: ','>>> ')
92  
-        self.ps2_str = self._set_prompt_str(ps2,'   .\\D.: ','... ')
93  
-        self.ps_out_str = self._set_prompt_str(ps_out,'Out[\\#]: ','')
94  
-
95  
-        self.color_table = prompts.PromptColors
96  
-        self.prompt1 = prompts.Prompt1(self,sep=input_sep,prompt=self.ps1_str,
97  
-                               pad_left=pad_left)
98  
-        self.prompt2 = prompts.Prompt2(self,prompt=self.ps2_str,pad_left=pad_left)
99  
-        self.prompt_out = prompts.PromptOut(self,sep='',prompt=self.ps_out_str,
100  
-                                    pad_left=pad_left)
101  
-        self.set_colors(colors)
102  
-
103  
-        # Store the last prompt string each time, we need it for aligning
104  
-        # continuation and auto-rewrite prompts
105  
-        self.last_prompt = ''
106  
-        self.output_sep = output_sep
107  
-        self.output_sep2 = output_sep2
  68
+        
108 69
         self._,self.__,self.___ = '','',''
109 70
 
110 71
         # these are deliberately global:
@@ -115,32 +76,6 @@ def __init__(self, shell=None, cache_size=1000,
115 76
     def prompt_count(self):
116 77
         return self.shell.execution_count
117 78
 
118  
-    def _set_prompt_str(self,p_str,cache_def,no_cache_def):
119  
-        if p_str is None:
120  
-            if self.do_full_cache:
121  
-                return cache_def
122  
-            else:
123  
-                return no_cache_def
124  
-        else:
125  
-            return p_str
126  
-
127  
-    def set_colors(self, colors):
128  
-        """Set the active color scheme and configure colors for the three
129  
-        prompt subsystems."""
130  
-
131  
-        # FIXME: This modifying of the global prompts.prompt_specials needs
132  
-        # to be fixed. We need to refactor all of the prompts stuff to use
133  
-        # proper configuration and traits notifications.
134  
-        if colors.lower()=='nocolor':
135  
-            prompts.prompt_specials = prompts.prompt_specials_nocolor
136  
-        else:
137  
-            prompts.prompt_specials = prompts.prompt_specials_color
138  
-
139  
-        self.color_table.set_active_scheme(colors)
140  
-        self.prompt1.set_colors()
141  
-        self.prompt2.set_colors()
142  
-        self.prompt_out.set_colors()
143  
-
144 79
     #-------------------------------------------------------------------------
145 80
     # Methods used in __call__. Override these methods to modify the behavior
146 81
     # of the displayhook.
@@ -180,8 +115,8 @@ def write_output_prompt(self):
180 115
         ``io.stdout``.
181 116
         """
182 117
         # Use write, not print which adds an extra space.
183  
-        io.stdout.write(self.output_sep)
184  
-        outprompt = str(self.prompt_out)
  118
+        io.stdout.write(self.shell.separate_out)
  119
+        outprompt = self.shell.prompt_manager.render('out')
185 120
         if self.do_full_cache:
186 121
             io.stdout.write(outprompt)
187 122
 
@@ -235,11 +170,12 @@ def write_format_data(self, format_dict):
235 170
             # So that multi-line strings line up with the left column of
236 171
             # the screen, instead of having the output prompt mess up
237 172
             # their first line.
238  
-            # We use the ps_out_str template instead of the expanded prompt
  173
+            # We use the prompt template instead of the expanded prompt
239 174
             # because the expansion may add ANSI escapes that will interfere
240 175
             # with our ability to determine whether or not we should add
241 176
             # a newline.
242  
-            if self.ps_out_str and not self.ps_out_str.endswith('\n'):
  177
+            prompt_template = self.shell.prompt_manager.out_template
  178
+            if prompt_template and not prompt_template.endswith('\n'):
243 179
                 # But avoid extraneous empty lines.
244 180
                 result_repr = '\n' + result_repr
245 181
 
@@ -286,7 +222,7 @@ def log_output(self, format_dict):
286 222
 
287 223
     def finish_displayhook(self):
288 224
         """Finish up all displayhook activities."""
289  
-        io.stdout.write(self.output_sep2)
  225
+        io.stdout.write(self.shell.separate_out2)
290 226
         io.stdout.flush()
291 227
 
292 228
     def __call__(self, result=None):
9  IPython/core/hooks.py
@@ -51,7 +51,7 @@ def calljed(self,filename, linenum):
51 51
 
52 52
 __all__ = ['editor', 'fix_error_editor', 'synchronize_with_editor',
53 53
            'input_prefilter', 'shutdown_hook', 'late_startup_hook',
54  
-           'generate_prompt', 'show_in_pager','pre_prompt_hook',
  54
+           'show_in_pager','pre_prompt_hook',
55 55
            'pre_run_code_hook', 'clipboard_get']
56 56
 
57 57
 def editor(self,filename, linenum=None):
@@ -187,13 +187,6 @@ def late_startup_hook(self):
187 187
     #print "default startup hook ok" # dbg
188 188
 
189 189
 
190  
-def generate_prompt(self, is_continuation):
191  
-    """ calculate and return a string with the prompt to display """
192  
-    if is_continuation:
193  
-        return str(self.displayhook.prompt2)
194  
-    return str(self.displayhook.prompt1)
195  
-
196  
-
197 190
 def show_in_pager(self,s):
198 191
     """ Run a string through pager """
199 192
     # raising TryNext here will use the default paging functionality
16  IPython/core/interactiveshell.py
@@ -63,6 +63,7 @@
63 63
 from IPython.core.prefilter import PrefilterManager, ESC_MAGIC
64 64
 from IPython.core.profiledir import ProfileDir
65 65
 from IPython.core.pylabtools import pylab_activate
  66
+from IPython.core.prompts import PromptManager
66 67
 from IPython.external.Itpl import ItplNS
67 68
 from IPython.utils import PyColorize
68 69
 from IPython.utils import io
@@ -594,10 +595,8 @@ def init_io(self):
594 595
             io.stderr = io.IOStream(sys.stderr)
595 596
 
596 597
     def init_prompts(self):
597  
-        # TODO: This is a pass for now because the prompts are managed inside
598  
-        # the DisplayHook. Once there is a separate prompt manager, this
599  
-        # will initialize that object and all prompt related information.
600  
-        pass
  598
+        self.prompt_manager = PromptManager(shell=self, config=self.config)
  599
+        self.configurables.append(self.prompt_manager)
601 600
 
602 601
     def init_display_formatter(self):
603 602
         self.display_formatter = DisplayFormatter(config=self.config)
@@ -613,13 +612,6 @@ def init_displayhook(self):
613 612
             config=self.config,
614 613
             shell=self,
615 614
             cache_size=self.cache_size,
616  
-            input_sep = self.separate_in,
617  
-            output_sep = self.separate_out,
618  
-            output_sep2 = self.separate_out2,
619  
-            ps1 = self.prompt_in1,
620  
-            ps2 = self.prompt_in2,
621  
-            ps_out = self.prompt_out,
622  
-            pad_left = self.prompts_pad_left
623 615
         )
624 616
         self.configurables.append(self.displayhook)
625 617
         # This is a context manager that installs/revmoes the displayhook at
@@ -2149,7 +2141,7 @@ def auto_rewrite_input(self, cmd):
2149 2141
         after the user's input prompt.  This helps the user understand that the
2150 2142
         input line was transformed automatically by IPython.
2151 2143
         """
2152  
-        rw = self.displayhook.prompt1.auto_rewrite() + cmd
  2144
+        rw = self.prompt_manager.render('rewrite') + cmd
2153 2145
 
2154 2146
         try:
2155 2147
             # plain ascii works better w/ pyreadline, on some machines, so
38  IPython/core/magic.py
@@ -2554,12 +2554,12 @@ def color_switch_err(name):
2554 2554
 
2555 2555
         # Set prompt colors
2556 2556
         try:
2557  
-            shell.displayhook.set_colors(new_scheme)
  2557
+            shell.prompt_manager.color_scheme = new_scheme
2558 2558
         except:
2559 2559
             color_switch_err('prompt')
2560 2560
         else:
2561 2561
             shell.colors = \
2562  
-                       shell.displayhook.color_table.active_scheme_name
  2562
+                   shell.prompt_manager.color_scheme_table.active_scheme_name
2563 2563
         # Set exception colors
2564 2564
         try:
2565 2565
             shell.InteractiveTB.set_colors(scheme = new_scheme)
@@ -3237,7 +3237,7 @@ def magic_doctest_mode(self,parameter_s=''):
3237 3237
 
3238 3238
         # Shorthands
3239 3239
         shell = self.shell
3240  
-        oc = shell.displayhook
  3240
+        pm = shell.prompt_manager
3241 3241
         meta = shell.meta
3242 3242
         disp_formatter = self.shell.display_formatter
3243 3243
         ptformatter = disp_formatter.formatters['text/plain']
@@ -3252,23 +3252,23 @@ def magic_doctest_mode(self,parameter_s=''):
3252 3252
         save_dstore('xmode',shell.InteractiveTB.mode)
3253 3253
         save_dstore('rc_separate_out',shell.separate_out)
3254 3254
         save_dstore('rc_separate_out2',shell.separate_out2)
3255  
-        save_dstore('rc_prompts_pad_left',shell.prompts_pad_left)
  3255
+        save_dstore('rc_prompts_pad_left',pm.justify)
3256 3256
         save_dstore('rc_separate_in',shell.separate_in)
3257 3257
         save_dstore('rc_plain_text_only',disp_formatter.plain_text_only)
  3258
+        save_dstore('prompt_templates',(pm.in_template, pm.in2_template, pm.out_template))
3258 3259
 
3259 3260
         if mode == False:
3260 3261
             # turn on
3261  
-            oc.prompt1.p_template = '>>> '
3262  
-            oc.prompt2.p_template = '... '
3263  
-            oc.prompt_out.p_template = ''
  3262
+            pm.in_template = '>>> '
  3263
+            pm.in2_template = '... '
  3264
+            pm.out_template = ''
3264 3265
 
3265 3266
             # Prompt separators like plain python
3266  
-            oc.input_sep = oc.prompt1.sep = ''
3267  
-            oc.output_sep = ''
3268  
-            oc.output_sep2 = ''
  3267
+            shell.separate_in = ''
  3268
+            shell.separate_out = ''
  3269
+            shell.separate_out2 = ''
3269 3270
 
3270  
-            oc.prompt1.pad_left = oc.prompt2.pad_left = \
3271  
-                                  oc.prompt_out.pad_left = False
  3271
+            pm.justify = False
3272 3272
 
3273 3273
             ptformatter.pprint = False
3274 3274
             disp_formatter.plain_text_only = True
@@ -3276,17 +3276,14 @@ def magic_doctest_mode(self,parameter_s=''):
3276 3276
             shell.magic_xmode('Plain')
3277 3277
         else:
3278 3278
             # turn off
3279  
-            oc.prompt1.p_template = shell.prompt_in1
3280  
-            oc.prompt2.p_template = shell.prompt_in2
3281  
-            oc.prompt_out.p_template = shell.prompt_out
  3279
+            pm.in_template, pm.in2_template, pm.out_template = dstore.prompt_templates
3282 3280
 
3283  
-            oc.input_sep = oc.prompt1.sep = dstore.rc_separate_in
  3281
+            shell.separate_in = dstore.rc_separate_in
3284 3282
 
3285  
-            oc.output_sep = dstore.rc_separate_out
3286  
-            oc.output_sep2 = dstore.rc_separate_out2
  3283
+            shell.separate_out = dstore.rc_separate_out
  3284
+            shell.separate_out2 = dstore.rc_separate_out2
3287 3285
 
3288  
-            oc.prompt1.pad_left = oc.prompt2.pad_left = \
3289  
-                         oc.prompt_out.pad_left = dstore.rc_prompts_pad_left
  3286
+            pm.justify = dstore.rc_prompts_pad_left
3290 3287
 
3291 3288
             ptformatter.pprint = dstore.rc_pprint
3292 3289
             disp_formatter.plain_text_only = dstore.rc_plain_text_only
@@ -3603,6 +3600,7 @@ def magic_config(self, s):
3603 3600
                 PrefilterManager
3604 3601
                 AliasManager
3605 3602
                 IPCompleter
  3603
+                PromptManager
3606 3604
                 DisplayFormatter
3607 3605
 
3608 3606
         To view what is configurable on a given class, just pass the class name::
498  IPython/core/prompts.py
@@ -5,6 +5,7 @@
5 5
 
6 6
 * Fernando Perez
7 7
 * Brian Granger
  8
+* Thomas Kluyver
8 9
 """
9 10
 
10 11
 #-----------------------------------------------------------------------------
@@ -23,20 +24,23 @@
23 24
 import re
24 25
 import socket
25 26
 import sys
  27
+import time
26 28
 
  29
+from IPython.config.configurable import Configurable
27 30
 from IPython.core import release
28  
-from IPython.external.Itpl import ItplNS
29 31
 from IPython.utils import coloransi
  32
+from IPython.utils.traitlets import (Unicode, Instance, Dict, Bool, Int)
30 33
 
31 34
 #-----------------------------------------------------------------------------
32 35
 # Color schemes for prompts
33 36
 #-----------------------------------------------------------------------------
34 37
 
35  
-PromptColors = coloransi.ColorSchemeTable()
36 38
 InputColors = coloransi.InputTermColors  # just a shorthand
37 39
 Colors = coloransi.TermColors  # just a shorthand
38 40
 
39  
-PromptColors.add_scheme(coloransi.ColorScheme(
  41
+color_lists = dict(normal=Colors(), inp=InputColors(), nocolor=coloransi.NoColors())
  42
+
  43
+PColNoColors = coloransi.ColorScheme(
40 44
     'NoColor',
41 45
     in_prompt  = InputColors.NoColor,  # Input prompt
42 46
     in_number  = InputColors.NoColor,  # Input prompt number
@@ -47,10 +51,10 @@
47 51
     out_number = Colors.NoColor, # Output prompt number
48 52
 
49 53
     normal = Colors.NoColor  # color off (usu. Colors.Normal)
50  
-    ))
  54
+    )
51 55
 
52 56
 # make some schemes as instances so we can copy them for modification easily:
53  
-__PColLinux =  coloransi.ColorScheme(
  57
+PColLinux =  coloransi.ColorScheme(
54 58
     'Linux',
55 59
     in_prompt  = InputColors.Green,
56 60
     in_number  = InputColors.LightGreen,
@@ -62,25 +66,35 @@
62 66
 
63 67
     normal = Colors.Normal
64 68
     )
65  
-# Don't forget to enter it into the table!
66  
-PromptColors.add_scheme(__PColLinux)
67 69
 
68 70
 # Slightly modified Linux for light backgrounds
69  
-__PColLightBG  = __PColLinux.copy('LightBG')
  71
+PColLightBG  = PColLinux.copy('LightBG')
70 72
 
71  
-__PColLightBG.colors.update(
  73
+PColLightBG.colors.update(
72 74
     in_prompt  = InputColors.Blue,
73 75
     in_number  = InputColors.LightBlue,
74 76
     in_prompt2 = InputColors.Blue
75 77
 )
76  
-PromptColors.add_scheme(__PColLightBG)
77  
-
78  
-del Colors,InputColors
79 78
 
80 79
 #-----------------------------------------------------------------------------
81 80
 # Utilities
82 81
 #-----------------------------------------------------------------------------
83 82
 
  83
+class LazyEvaluate(object):
  84
+    """This is used for formatting strings with values that need to be updated
  85
+    at that time, such as the current time or working directory."""
  86
+    def __init__(self, func, *args, **kwargs):
  87
+        self.func = func
  88
+        self.args = args
  89
+        self.kwargs = kwargs
  90
+    
  91
+    def __call__(self, **kwargs):
  92
+        self.kwargs.update(kwargs)
  93
+        return self.func(*self.args, **self.kwargs)
  94
+    
  95
+    def __str__(self):
  96
+        return str(self())
  97
+
84 98
 def multiple_replace(dict, text):
85 99
     """ Replace in 'text' all occurences of any key in the given
86 100
     dictionary by its corresponding value.  Returns the new string."""
@@ -121,51 +135,43 @@ def multiple_replace(dict, text):
121 135
 USER           = os.environ.get("USER")
122 136
 HOSTNAME       = socket.gethostname()
123 137
 HOSTNAME_SHORT = HOSTNAME.split(".")[0]
124  
-ROOT_SYMBOL    = "$#"[os.name=='nt' or os.getuid()==0]
  138
+ROOT_SYMBOL    = "#" if (os.name=='nt' or os.getuid()==0) else "$"
125 139
 
126  
-prompt_specials_color = {
  140
+prompt_abbreviations = {
127 141
     # Prompt/history count
128  
-    '%n' : '${self.col_num}' '${self.cache.prompt_count}' '${self.col_p}',
129  
-    r'\#': '${self.col_num}' '${self.cache.prompt_count}' '${self.col_p}',
  142
+    '%n' : '{color.number}' '{count}' '{color.prompt}',
  143
+    r'\#': '{color.number}' '{count}' '{color.prompt}',
130 144
     # Just the prompt counter number, WITHOUT any coloring wrappers, so users
131 145
     # can get numbers displayed in whatever color they want.
132  
-    r'\N': '${self.cache.prompt_count}',
  146
+    r'\N': '{count}',
133 147
 
134 148
     # Prompt/history count, with the actual digits replaced by dots.  Used
135 149
     # mainly in continuation prompts (prompt_in2)
136  
-    #r'\D': '${"."*len(str(self.cache.prompt_count))}',
137  
-
138  
-    # More robust form of the above expression, that uses the __builtin__
139  
-    # module.  Note that we can NOT use __builtins__ (note the 's'), because
140  
-    # that can either be a dict or a module, and can even mutate at runtime,
141  
-    # depending on the context (Python makes no guarantees on it).  In
142  
-    # contrast, __builtin__ is always a module object, though it must be
143  
-    # explicitly imported.
144  
-    r'\D': '${"."*__builtin__.len(__builtin__.str(self.cache.prompt_count))}',
  150
+    r'\D': '{dots}',
145 151
 
146  
-    # Current working directory
147  
-    r'\w': '${os.getcwd()}',
148 152
     # Current time
149  
-    r'\t' : '${time.strftime("%H:%M:%S")}',
  153
+    r'\T' : '{time}',
  154
+    # Current working directory
  155
+    r'\w': '{cwd}',
150 156
     # Basename of current working directory.
151 157
     # (use os.sep to make this portable across OSes)
152  
-    r'\W' : '${os.getcwd().split("%s")[-1]}' % os.sep,
  158
+    r'\W' : '{cwd_last}',
153 159
     # These X<N> are an extension to the normal bash prompts.  They return
154 160
     # N terms of the path, after replacing $HOME with '~'
155  
-    r'\X0': '${os.getcwd().replace("%s","~")}' % HOME,
156  
-    r'\X1': '${self.cwd_filt(1)}',
157  
-    r'\X2': '${self.cwd_filt(2)}',
158  
-    r'\X3': '${self.cwd_filt(3)}',
159  
-    r'\X4': '${self.cwd_filt(4)}',
160  
-    r'\X5': '${self.cwd_filt(5)}',
  161
+    r'\X0': '{cwd_x[0]}',
  162
+    r'\X1': '{cwd_x[1]}',
  163
+    r'\X2': '{cwd_x[2]}',
  164
+    r'\X3': '{cwd_x[3]}',
  165
+    r'\X4': '{cwd_x[4]}',
  166
+    r'\X5': '{cwd_x[5]}',
161 167
     # Y<N> are similar to X<N>, but they show '~' if it's the directory
162 168
     # N+1 in the list.  Somewhat like %cN in tcsh.
163  
-    r'\Y0': '${self.cwd_filt2(0)}',
164  
-    r'\Y1': '${self.cwd_filt2(1)}',
165  
-    r'\Y2': '${self.cwd_filt2(2)}',
166  
-    r'\Y3': '${self.cwd_filt2(3)}',
167  
-    r'\Y4': '${self.cwd_filt2(4)}',
168  
-    r'\Y5': '${self.cwd_filt2(5)}',
  169
+    r'\Y0': '{cwd_y[0]}',
  170
+    r'\Y1': '{cwd_y[1]}',
  171
+    r'\Y2': '{cwd_y[2]}',
  172
+    r'\Y3': '{cwd_y[3]}',
  173
+    r'\Y4': '{cwd_y[4]}',
  174
+    r'\Y5': '{cwd_y[5]}',
169 175
     # Hostname up to first .
170 176
     r'\h': HOSTNAME_SHORT,
171 177
     # Full hostname
@@ -184,253 +190,189 @@ def multiple_replace(dict, text):
184 190
     r'\$': ROOT_SYMBOL,
185 191
     }
186 192
 
187  
-# A copy of the prompt_specials dictionary but with all color escapes removed,
188  
-# so we can correctly compute the prompt length for the auto_rewrite method.
189  
-prompt_specials_nocolor = prompt_specials_color.copy()
190  
-prompt_specials_nocolor['%n'] = '${self.cache.prompt_count}'
191  
-prompt_specials_nocolor[r'\#'] = '${self.cache.prompt_count}'
192  
-
193  
-# Add in all the InputTermColors color escapes as valid prompt characters.
194  
-# They all get added as \\C_COLORNAME, so that we don't have any conflicts
195  
-# with a color name which may begin with a letter used by any other of the
196  
-# allowed specials.  This of course means that \\C will never be allowed for
197  
-# anything else.
198  
-input_colors = coloransi.InputTermColors
199  
-for _color in dir(input_colors):
200  
-    if _color[0] != '_':
201  
-        c_name = r'\C_'+_color
202  
-        prompt_specials_color[c_name] = getattr(input_colors,_color)
203  
-        prompt_specials_nocolor[c_name] = ''
204  
-
205  
-# we default to no color for safety.  Note that prompt_specials is a global
206  
-# variable used by all prompt objects.
207  
-prompt_specials = prompt_specials_nocolor
208  
-
209 193
 #-----------------------------------------------------------------------------
210 194
 # More utilities
211 195
 #-----------------------------------------------------------------------------
212 196
 
213  
-def str_safe(arg):
214  
-    """Convert to a string, without ever raising an exception.
215  
-
216  
-    If str(arg) fails, <ERROR: ... > is returned, where ... is the exception
217  
-    error message."""
218  
-
219  
-    try:
220  
-        out = str(arg)
221  
-    except UnicodeError:
222  
-        try:
223  
-            out = arg.encode('utf_8','replace')
224  
-        except Exception,msg:
225  
-            # let's keep this little duplication here, so that the most common
226  
-            # case doesn't suffer from a double try wrapping.
227  
-            out = '<ERROR: %s>' % msg
228  
-    except Exception,msg:
229  
-        out = '<ERROR: %s>' % msg
230  
-        #raise  # dbg
231  
-    return out
  197
+def cwd_filt(depth):
  198
+    """Return the last depth elements of the current working directory.
232 199
 
233  
-#-----------------------------------------------------------------------------
234  
-# Prompt classes
235  
-#-----------------------------------------------------------------------------
  200
+    $HOME is always replaced with '~'.
  201
+    If depth==0, the full path is returned."""
236 202
 
237  
-class BasePrompt(object):
238  
-    """Interactive prompt similar to Mathematica's."""
239  
-
240  
-    def _get_p_template(self):
241  
-        return self._p_template
242  
-
243  
-    def _set_p_template(self,val):
244  
-        self._p_template = val
245  
-        self.set_p_str()
246  
-
247  
-    p_template = property(_get_p_template,_set_p_template,
248  
-                          doc='Template for prompt string creation')
249  
-
250  
-    def __init__(self, cache, sep, prompt, pad_left=False):
251  
-
252  
-        # Hack: we access information about the primary prompt through the
253  
-        # cache argument.  We need this, because we want the secondary prompt
254  
-        # to be aligned with the primary one.  Color table info is also shared
255  
-        # by all prompt classes through the cache.  Nice OO spaghetti code!
256  
-        self.cache = cache
257  
-        self.sep = sep
258  
-
259  
-        # regexp to count the number of spaces at the end of a prompt
260  
-        # expression, useful for prompt auto-rewriting
261  
-        self.rspace = re.compile(r'(\s*)$')
262  
-        # Flag to left-pad prompt strings to match the length of the primary
263  
-        # prompt
264  
-        self.pad_left = pad_left
265  
-
266  
-        # Set template to create each actual prompt (where numbers change).
267  
-        # Use a property
268  
-        self.p_template = prompt
269  
-        self.set_p_str()
270  
-
271  
-    def set_p_str(self):
272  
-        """ Set the interpolating prompt strings.
273  
-
274  
-        This must be called every time the color settings change, because the
275  
-        prompt_specials global may have changed."""
276  
-
277  
-        import os,time  # needed in locals for prompt string handling
278  
-        loc = locals()
279  
-        try:
280  
-            self.p_str = ItplNS('%s%s%s' %
281  
-                                ('${self.sep}${self.col_p}',
282  
-                                 multiple_replace(prompt_specials, self.p_template),
283  
-                                 '${self.col_norm}'),self.cache.shell.user_ns,loc)
284  
-
285  
-            self.p_str_nocolor = ItplNS(multiple_replace(prompt_specials_nocolor,
286  
-                                                         self.p_template),
287  
-                                        self.cache.shell.user_ns,loc)
288  
-        except:
289  
-            print "Illegal prompt template (check $ usage!):",self.p_template
290  
-            self.p_str = self.p_template
291  
-            self.p_str_nocolor = self.p_template
292  
-
293  
-    def write(self, msg):
294  
-        sys.stdout.write(msg)
295  
-        return ''
  203
+    cwd = os.getcwd().replace(HOME,"~")
  204
+    out = os.sep.join(cwd.split(os.sep)[-depth:])
  205
+    return out or os.sep
296 206
 
297  
-    def __str__(self):
298  
-        """Return a string form of the prompt.
299  
-
300  
-        This for is useful for continuation and output prompts, since it is
301  
-        left-padded to match lengths with the primary one (if the
302  
-        self.pad_left attribute is set)."""
303  
-
304  
-        out_str = str_safe(self.p_str)
305  
-        if self.pad_left:
306  
-            # We must find the amount of padding required to match lengths,
307  
-            # taking the color escapes (which are invisible on-screen) into
308  
-            # account.
309  
-            esc_pad = len(out_str) - len(str_safe(self.p_str_nocolor))
310  
-            format = '%%%ss' % (len(str(self.cache.last_prompt))+esc_pad)
311  
-            return format % out_str
312  
-        else:
313  
-            return out_str
  207
+def cwd_filt2(depth):
  208
+    """Return the last depth elements of the current working directory.
314 209
 
315  
-    # these path filters are put in as methods so that we can control the
316  
-    # namespace where the prompt strings get evaluated
317  
-    def cwd_filt(self, depth):
318  
-        """Return the last depth elements of the current working directory.
  210
+    $HOME is always replaced with '~'.
  211
+    If depth==0, the full path is returned."""
319 212
 
320  
-        $HOME is always replaced with '~'.
321  
-        If depth==0, the full path is returned."""
  213
+    full_cwd = os.getcwd()
  214
+    cwd = full_cwd.replace(HOME,"~").split(os.sep)
  215
+    if '~' in cwd and len(cwd) == depth+1:
  216
+        depth += 1
  217
+    drivepart = ''
  218
+    if sys.platform == 'win32' and len(cwd) > depth:
  219
+        drivepart = os.path.splitdrive(full_cwd)[0]
  220
+    out = drivepart + '/'.join(cwd[-depth:])
322 221
 
323  
-        cwd = os.getcwd().replace(HOME,"~")
324  
-        out = os.sep.join(cwd.split(os.sep)[-depth:])
325  
-        if out:
326  
-            return out
327  
-        else:
328  
-            return os.sep
  222
+    return out or os.sep
329 223
 
330  
-    def cwd_filt2(self, depth):
331  
-        """Return the last depth elements of the current working directory.
332  
-
333  
-        $HOME is always replaced with '~'.
334  
-        If depth==0, the full path is returned."""
335  
-
336  
-        full_cwd = os.getcwd()
337  
-        cwd = full_cwd.replace(HOME,"~").split(os.sep)
338  
-        if '~' in cwd and len(cwd) == depth+1:
339  
-            depth += 1
340  
-        drivepart = ''
341  
-        if sys.platform == 'win32' and len(cwd) > depth:
342  
-            drivepart = os.path.splitdrive(full_cwd)[0]
343  
-        out = drivepart + '/'.join(cwd[-depth:])
  224
+#-----------------------------------------------------------------------------
  225
+# Prompt classes
  226
+#-----------------------------------------------------------------------------
344 227
 
345  
-        if out:
346  
-            return out
  228
+lazily_evaluate = {'time': LazyEvaluate(time.strftime, "%H:%M:%S"),
  229
+                   'cwd': LazyEvaluate(os.getcwd),
  230
+                   'cwd_last': LazyEvaluate(lambda: os.getcwd().split(os.sep)[-1]),
  231
+                   'cwd_x': [LazyEvaluate(lambda: os.getcwd().replace("%s","~"))] +\
  232
+                            [LazyEvaluate(cwd_filt, x) for x in range(1,6)],
  233
+                   'cwd_y': [LazyEvaluate(cwd_filt2, x) for x in range(6)]
  234
+                   }
  235
+        
  236
+
  237
+class PromptManager(Configurable):
  238
+    """This is the primary interface for producing IPython's prompts."""
  239
+    shell = Instance('IPython.core.interactiveshell.InteractiveShellABC')
  240
+    
  241
+    color_scheme_table = Instance(coloransi.ColorSchemeTable)
  242
+    color_scheme = Unicode('Linux', config=True)
  243
+    def _color_scheme_changed(self, name, new_value):
  244
+        self.color_scheme_table.set_active_scheme(new_value)
  245
+        for pname in ['in', 'in2', 'out', 'rewrite']:
  246
+            # We need to recalculate the number of invisible characters
  247
+            self.update_prompt(pname)
  248
+    
  249
+    lazy_evaluate_fields = Dict(help="""
  250
+        This maps field names used in the prompt templates to functions which
  251
+        will be called when the prompt is rendered. This allows us to include
  252
+        things like the current time in the prompts. Functions are only called
  253
+        if they are used in the prompt.
  254
+        """)
  255
+    def _lazy_evaluate_fields_default(self): return lazily_evaluate.copy()
  256
+    
  257
+    in_template = Unicode('In [\\#]: ', config=True)
  258
+    in2_template = Unicode('   .\\D.: ', config=True)
  259
+    out_template = Unicode('Out[\\#]: ', config=True)
  260
+    rewrite_template = Unicode("------> ", config=True)
  261
+    
  262
+    justify = Bool(True, config=True, help="""
  263
+        If True (default), each prompt will be right-aligned with the
  264
+        preceding one.
  265
+        """)
  266
+    
  267
+    # We actually store the expanded templates here:
  268
+    templates = Dict()
  269
+    
  270
+    # The number of characters in the last prompt rendered, not including
  271
+    # colour characters.
  272
+    width = Int()
  273
+    
  274
+    # The number of characters in each prompt which don't contribute to width
  275
+    invisible_chars = Dict()
  276
+    def _invisible_chars_default(self):
  277
+        return {'in': 0, 'in2': 0, 'out': 0, 'rewrite': 0}
  278
+    
  279
+    def __init__(self, shell, config=None):
  280
+        super(PromptManager, self).__init__(shell=shell, config=config)
  281
+        
  282
+        # Prepare colour scheme table
  283
+        self.color_scheme_table = coloransi.ColorSchemeTable([PColNoColors,
  284
+                                    PColLinux, PColLightBG], self.color_scheme)
  285
+        
  286
+        # Prepare templates
  287
+        self.update_prompt('in', self.in_template)
  288
+        self.update_prompt('in2', self.in2_template)
  289
+        self.update_prompt('out', self.out_template)
  290
+        self.update_prompt('rewrite', self.rewrite_template)
  291
+        self.on_trait_change(self._update_prompt_trait, ['in_template',
  292
+                            'in2_template', 'out_template', 'rewrite_template'])
  293
+    
  294
+    def update_prompt(self, name, new_template=None):
  295
+        """This is called when a prompt template is updated. It processes
  296
+        abbreviations used in the prompt template (like \#) and calculates how
  297
+        many invisible characters (ANSI colour escapes) the resulting prompt
  298
+        contains.
  299
+        
  300
+        It is also called for each prompt on changing the colour scheme. In both
  301
+        cases, traitlets should take care of calling this automatically.
  302
+        """
  303
+        if new_template is not None:
  304
+            self.templates[name] = multiple_replace(prompt_abbreviations, new_template)
  305
+        invis_chars = len(self.render(name, color=True, just=False)) - \
  306
+                            len(self.render(name, color=False, just=False))
  307
+        self.invisible_chars[name] = invis_chars
  308
+    
  309
+    def _update_prompt_trait(self, traitname, new_template):
  310
+        name = traitname[:-9]   # Cut off '_template'
  311
+        self.update_prompt(name, new_template)
  312
+    
  313
+    def render(self, name, color=True, just=None, **kwargs):
  314
+        """
  315
+        Render the selected prompt.
  316
+        
  317
+        Parameters
  318
+        ----------
  319
+        name : str
  320
+          Which prompt to render. One of 'in', 'in2', 'out', 'rewrite'
  321
+        color : bool
  322
+          If True (default), include ANSI escape sequences for a coloured prompt.
  323
+        just : bool
  324
+          If True, justify the prompt to the width of the last prompt. The
  325
+          default is stored in self.justify.
  326
+        **kwargs :
  327
+          Additional arguments will be passed to the string formatting operation,
  328
+          so they can override the values that would otherwise fill in the
  329
+          template.
  330
+        
  331
+        Returns
  332
+        -------
  333
+        A string containing the rendered prompt.
  334
+        """
  335
+        if color:
  336
+            scheme = self.color_scheme_table.active_colors
  337
+            if name=='out':
  338
+                colors = color_lists['normal']
  339
+                colors.number, colors.prompt, colors.normal = \
  340
+                        scheme.out_number, scheme.out_prompt, scheme.normal
  341
+            elif name=='rewrite':
  342
+                colors = color_lists['normal']
  343
+                # We need a non-input version of these escapes
  344
+                colors.number = scheme.in_number.replace("\001","").replace("\002","")
  345
+                colors.prompt = scheme.in_prompt.replace("\001","").replace("\002","")
  346
+                colors.normal = scheme.normal
  347
+            else:
  348
+                colors = color_lists['inp']
  349
+                colors.number, colors.prompt, colors.normal = \
  350
+                        scheme.in_number, scheme.in_prompt, scheme.in_normal
  351
+                if name=='in2':
  352
+                    colors.prompt = scheme.in_prompt2
347 353
         else:
348  
-            return os.sep
349  
-
350  
-    def __nonzero__(self):
351  
-        """Implement boolean behavior.
352  
-
353  
-        Checks whether the p_str attribute is non-empty"""
354  
-
355  
-        return bool(self.p_template)
356  
-
357  
-
358  
-class Prompt1(BasePrompt):
359  
-    """Input interactive prompt similar to Mathematica's."""
360  
-
361  
-    def __init__(self, cache, sep='\n', prompt='In [\\#]: ', pad_left=True):
362  
-        BasePrompt.__init__(self, cache, sep, prompt, pad_left)
363  
-
364  
-    def set_colors(self):
365  
-        self.set_p_str()
366  
-        Colors = self.cache.color_table.active_colors # shorthand
367  
-        self.col_p = Colors.in_prompt
368  
-        self.col_num = Colors.in_number
369  
-        self.col_norm = Colors.in_normal
370  
-        # We need a non-input version of these escapes for the '--->'
371  
-        # auto-call prompts used in the auto_rewrite() method.
372  
-        self.col_p_ni = self.col_p.replace('\001','').replace('\002','')
373  
-        self.col_norm_ni = Colors.normal
374  
-
375  
-    def __str__(self):
376  
-        self.cache.last_prompt = str_safe(self.p_str_nocolor).split('\n')[-1]
377  
-        return str_safe(self.p_str)
378  
-
379  
-    def auto_rewrite(self):
380  
-        """Return a string of the form '--->' which lines up with the previous
381  
-        input string. Useful for systems which re-write the user input when
382  
-        handling automatically special syntaxes."""
383  
-
384  
-        curr = str(self.cache.last_prompt)
385  
-        nrspaces = len(self.rspace.search(curr).group())
386  
-        return '%s%s>%s%s' % (self.col_p_ni,'-'*(len(curr)-nrspaces-1),
387  
-                              ' '*nrspaces,self.col_norm_ni)
388  
-
389  
-
390  
-class PromptOut(BasePrompt):
391  
-    """Output interactive prompt similar to Mathematica's."""
392  
-
393  
-    def __init__(self, cache, sep='', prompt='Out[\\#]: ', pad_left=True):
394  
-        BasePrompt.__init__(self, cache, sep, prompt, pad_left)
395  
-        if not self.p_template:
396  
-            self.__str__ = lambda: ''
397  
-
398  
-    def set_colors(self):
399  
-        self.set_p_str()