Logging & other output concerns #15
Comments
bitprophet
changed the title
|
So what's the work need to be done here? Just use Python's logging module instead of I assume you want to log the output of the command, and with logging formatting you can control whether the command itself is shown and whether the hostname is shown? So something like: c.run('ls')Would output something like: And: import logging
logging.basicConfig(format="%(command)s\n%(output)s")
c.run('ls')Would output something like: and import logging
logging.basicConfig(format="[%(hostname)] %(command)s\n%(output)s")
c.run('ls')Would output something like: Am I on the right track here? |
|
in stressant, i add a logging output level called i call the logging function once per line, that said, not once per command. maybe that's bad, but i need this to show that stuff on the console, so i had no way around that that i found elegant. |
|
I've created my own from textwrap import indent
class Color:
# Terminal text color escape characters
PURPLE = '\033[95m'
CYAN = '\033[96m'
DARKCYAN = '\033[36m'
BLUE = '\033[94m'
GREEN = '\033[92m'
YELLOW = '\033[93m'
RED = '\033[91m'
BOLD = '\033[1m'
UNDERLINE = '\033[4m'
END = '\033[0m'
def run(conn, command, sudo_pass=None):
"""
At the time of writing, Fabric 2 doesn't have a way to show the hostname in
front of the command. So we do it manually here.
This GitHub issue is working on solving this in Fabric 2:
https://github.com/pyinvoke/invoke/issues/15
"""
print(f"{Color.BOLD}[{conn.host}] {command}{Color.END}\n")
if sudo_pass:
result = conn.sudo(command, hide=True, echo=False, password=sudo_pass)
else:
result = conn.run(command, hide=True, echo=False)
output = indent(result.stdout or result.stderr, ' ')
print(output)
@contextmanager
def cd(conn, path):
"""
For the same reason we define our own `run()` command, we also have our own
`cd()` command so we can log elaborate output to the user.
Because this method should be a contextmanager, we can't use the original
`cd()` method, but we can mimic it's functionality, by updating the value
of `Connection.command_cwds`.
"""
print(f'\n{Color.BOLD}[{conn.host}] entering dir {path}{Color.END}\n')
conn.command_cwds.append(path)
try:
yield
finally:
print(f'\n{Color.BOLD}[{conn.host}] leaving dir {path}{Color.END}\n')
conn.command_cwds.pop() |
|
after fiddling around with fabric for a few more days, one thing i'm missing is a for example, take this script: it has a if fab had a (well, that's not true: that script is complicated by the fact that it operates on three different servers, so the task would at least need to be rewire just so, but still...) update: i added #706 to show what i mean. probably incomplete... i wonder if another part of this would be to convert a precious few of the current |
This is useful for fabfiles that use `logging.info` as an intermediate
between 'no output' and 'full debug'.
This is not ideal: I don't like that if/elif check to set the log
level. But I cannot figure out how to do the equivalent of this
argparse snippet:
parser.add_argument('--verbose', '-v', dest='log_level',
action='store_const', const='info', default='warning')
parser.add_argument('--debug', '-d', dest='log_level',
action='store_const', const='debug', default='warning')
I suspect I might be missing some invoke.yml configuration bits as well.
See: pyinvoke#15
|
here's another hack I used: class VerboseProgram(Fab):
# cargo-culted from fab's main.py
def __init__(self, *args, **kwargs):
super().__init__(*args,
executor_class=Executor,
config_class=Config,
**kwargs)
def core_args(self):
core_args = super().core_args()
extra_args = [
Argument(
names=('verbose', 'v'),
kind=bool,
default=False,
help="be more verbose"
),
]
return core_args + extra_args
def parse_core(self, argv):
super().parse_core(argv)
if self.args.debug.value:
logging.getLogger('').setLevel(logging.DEBUG)
elif self.args.verbose.value:
logging.getLogger('').setLevel(logging.INFO)
# reset formatter
logging.debug('reseting logging formats')
for h in logging.getLogger('').handlers:
f = logging.Formatter()
h.setFormatter(f)
# override default logging policies in submodules
#
# without this, we get debugging info from paramiko with --verbose
for mod in 'fabric', 'paramiko', 'invoke':
logging.getLogger(mod).setLevel('WARNING')kind of clunky, but it seemed better than hacking up my own wrapper to a task, which is what i was doing before (and required duplicating the argument parsing logic). |
This is useful for fabfiles that use `logging.info` as an intermediate
between 'no output' and 'full debug'.
This is not ideal: I don't like that if/elif check to set the log
level. But I cannot figure out how to do the equivalent of this
argparse snippet:
parser.add_argument('--verbose', '-v', dest='log_level',
action='store_const', const='info', default='warning')
parser.add_argument('--debug', '-d', dest='log_level',
action='store_const', const='debug', default='warning')
I suspect I might be missing some invoke.yml configuration bits as well.
See: pyinvoke#15
This is useful for fabfiles that use `logging.info` as an intermediate
between 'no output' and 'full debug'.
This is not ideal: I don't like that if/elif check to set the log
level. But I cannot figure out how to do the equivalent of this
argparse snippet:
parser.add_argument('--verbose', '-v', dest='log_level',
action='store_const', const='info', default='warning')
parser.add_argument('--debug', '-d', dest='log_level',
action='store_const', const='debug', default='warning')
I suspect I might be missing some invoke.yml configuration bits as well.
See: pyinvoke#15
Turning this into a general "how to do logging and useful output controls for invoke and fabric 2" ticket.
Use cases
Older thoughts
Thoughts for now, jumping off of the description of fabric/fabric#57 & current state of things in Invoke master:
invoke.utils.debug(w/ setup to add other levels easily) that logs to a package-level logger. Could probably be expanded some, it's mostly used in the parsing and config modules.runis called, on which host (in Fabric), with which command (tho this may want truncating...), run-time, etc - but not the stdout/stderr contents (though sizes of those, similar to HTTP log formats, is probably good).Resultobjects.run's stream arguments, and to configure logging to taste.Original/outdated description
Right now stdout/stderr is printed in our Popen subclass byte-by-byte to allow interaction etc. However, eventually we'll want real logging, meaning an ability to throw things line-by-line into a log library, meaning an extra buffer at the Popen level that does additional Things with lines as they are tallied up.
I suppose there should be an option to turn off the bytewise in favor of just linewise, but that could be a 2nd tier thing or saved for when we start layering Fab 2 on top of this.
The text was updated successfully, but these errors were encountered: