Allow configuring (vs just toggling) v1's output line prefix

[bitprophet]

Description

It seems that the output prefix should be a bit more flexible than it currently is. The current format is:

[hoststring] Message

This should be flexible, meaning both format as well as the ability to turn it off, or make it print one line per host/task combo. (That particular aspect has been spun off into #281.)

An ideal situation would be to use python's built-in template strings. (or to be very forward thinking format strings, but those require 2.6). The template can be put in evn.output.prefix or similar. Using the templates it is easy to just define output format based on exposed variables.

These could include:

• hostname
• hoststring
• comand output signifier [1][2]
• time [1]
• other stuff

It could also include color code formatting.

[1] some variables will need to have a helper variable, e.g. strftime format for time
[2] output signifier could be something simple like a ** making the output something like:

[hostname] Log msg
[hostname] ** output from cmd

An example format string (with helpers) could look like:

env.output.prefix = "[${hostname} - ${time}] ${cmd_sig}"
env.output.timeformat = "H:M"
env.output.cmd_sig = "**"

resulting in:

[www1.foo.com - 17:05] Executing hostname
[www1.foo.com - 17:05] ** www1.foo.com

---

Originally submitted by Erich Heine (sophacles) on 2010-05-14 at 05:56pm EDT

Attachments

• fabric.patch

Relations

• Related to puts show_prefix #164: puts show_prefix
• Related to Make line prefix optional #281: Make line prefix optional
• Related to Support full logging to file #57: Support full logging to file
• Related to Improved prompt detection and passthrough #7: Improved prompt detection and passthrough

[bitprophet]

Erich Heine (sophacles) posted:

---

OK, so formatting fail... it should read:

These could include:

• hostname
• hoststring
• command output signifier [1][2]
• time [1]
• other stuff

---

on 2010-05-14 at 05:58pm EDT

[bitprophet]

Jeff Forcier (bitprophet) posted:

---

Tweaked formatting a bit for you. Thanks for making the ticket.

I agree that the forward compatible string formatting would be best but I don't want to rely on 2.6 yet so that's unfortunately out.

Amusingly a translated format string brings us back to the older magical string interpolation, but since it's required for dynamic stuff like this and is limited to a single spot of the code, may not be so bad.

An alternative is a call chain, an array of callbacks, e.g. it might start out as env.prefix = [lambda: env.host_string] and one could then, in their fabfile's module level, do env.prefix.append(lambda: datetime.now.strftime("whatevs")). This feels somewhat more Pythonic to me at the expense of losing some control over the exact prefix string format.

---

on 2010-05-14 at 07:28pm EDT

[bitprophet]

Erich Heine (sophacles) posted:

---

Nice! I like it anytime callbacks can be incorporated anywhere. The thing we are discussing at this point tho is an "almost but not quite" version of python's logging module. Perhaps that should be looked at for either inspiration or instead of a one-off. Refreshing myself on the module, I notice there are a few things that could help in other places too. There are handlers and formatters, and it supports the notion of levels, so theoretically this could do everything fabric already aims to do regarding output. The biggest problem would be prompting, but if the chain of events goes (remote process output)->(paramiko.stdout/stderr) -> (prompt/expect handling) -> logger it may not be a big deal.

Anyway, some food for thought for the morning.

---

on 2010-05-15 at 11:25am EDT

[bitprophet]

Jeff Forcier (bitprophet) posted:

---

Good point. We do have tickets open that refer to using the logging module, and I think some of them did mention using it for stdout as well as to-file logging. Will add them as related items in case that has any bearing on this specific issue (I haven't really played with the logging module much, but given how logs are, I do expect it'll have prefix options...)

---

on 2010-05-15 at 02:02pm EDT

[bitprophet]

Jeff Forcier (bitprophet) posted:

---

Keep in mind that a fully empty prefix should also be easy to effect, though offhand that should simply be an empty-string prefix setting, so not a big deal.

For example, now that #7 is basically done, fully interactive stuff is going to want prefixing turned off completely lest it look terribly ugly or even broken.

---

on 2010-06-21 at 10:34pm EDT

[bitprophet]

Jeff Forcier (bitprophet) posted:

---

Attaching a related patch from Pawel Michalski, based off of the 0.9.3 codebase.

---

on 2011-01-16 at 09:23am EST

[bitprophet]

Jeff Forcier (bitprophet) posted:

---

BUNP

---

on 2011-06-21 at 08:33pm EDT

[andresriancho]

What's required to see this merged into master?

[morrissimo]

👍 for this - especially the customizable datetime prefix bits.

[bitprophet]

The main hurdle is that changes to critical parts of the codebase (such as this) aren't typically worth the support/backwards-compat burden, when an in-the-pipe rewrite with this sort of flexibility in mind is coming (Fab 2). The chance of this (if it were cleaned up for current master & merged) causing issues is relatively high :(

See also the roadmap.

[chrisspen]

Allowing this to be configurable would be nice. I currently re-run commands on the same host but specifying different shell variables to select different databases. It would be useful if I could configure it to show [hostname:database] so if the tasks stops due to some error (e.g. a migration wasn't run on a database causing a later data-load to fail) I know which database it refers to instead of having to guess and/or manually check one of several dozen databases.

[humitos]

It would be good to have the ability to turn the prefix off also. For example, I'm using this task to connect to a remote Django shell and try many things on the server directly:

def shell_plus():
    with cd(env.project):
        with virtualenv(env.venv):
            run("./manage.py shell")

Right now, I have a lot of prefix information that I don't need and it's just noise.

[humitos]

Well, I found the env.output_prefix settings to turn this off, but there is no docs about that :( . I found it looking at 3b5ab4f

[bitprophet]

One impl of this is in #1140.

[ror6ax]

Currently env.output.prefix="[]" returns AttributeError:

   env.output.prefix="[]"
  File "/usr/lib/python2.6/site-packages/fabric/utils.py", line 204, in __getattr__
    raise AttributeError(key)
AttributeError: output

[bitprophet]

That looks like a typo, should be env.output_prefix - underscore, not dot :)

[anhvngo]

Is there any way to put the boolean value in fabricic file ?
I tried : output_prefix=False
but it understand False as string 'False'.

[bitprophet]

@anhvanngouiowa No, that's a limitation of fabricrc files :( strings only. FWIW version 2.0 will have a much less awful config mechanism allowing this sort of setting to be configured usefully.

[Constantin07]

I also need a way to customise the output. When this patch is going to be merged into upstream ?

[bitprophet]

Roll this into pyinvoke/invoke#15

[bitprophet]

On second thought, just make it more obvious that this would be a nice small feature add for the 1.x line.