New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Condense/simplify CLI output #32

Closed
tristanpemble opened this Issue Aug 28, 2017 · 5 comments

Comments

Projects
None yet
3 participants
@tristanpemble
Contributor

tristanpemble commented Aug 28, 2017

The "parallel" output is pretty cool to watch, but it's verbose and hard to follow. When things break it's quite difficult to figure out what broke and why, while text is flying around the screen.

I believe there's a middle ground of informative and actionable that could be found with the CLI output.

@tristanpemble tristanpemble changed the title from Condense CLI output to Condense/simplify CLI output Aug 28, 2017

@plombardi89 plombardi89 added the Feature label Aug 28, 2017

@rhs

This comment has been minimized.

Contributor

rhs commented Aug 28, 2017

I totally agree with you, but I'm also not sure what direction to go with this. Here are the challenges I have in displaying what is going on:

  1. There are multiple things happening in parallel, i.e. essentially every container is built in parallel, and this means you potentially get interleaved queries to local/remote docker registry and potentially interleaved output from the docker build command.

  2. There's output from multiple distinct commands (kubectl apply vs docker build vs docker registry vs forge itself).

  3. As a general principle I want it to be easy to figure out when forge is at fault vs one of the things it depends on (e.g. if my docker password is wrong, it should be obvious I need to poke at my docker related config).

I took a brief survey of packer, ansible, and gradle (thanks to @plombardi89's suggestions) and they seem to deal with a number of similar problems. Based on their output I can think of the following potential improvements (these aren't mutually exclusive):

  1. Offer (perhaps by default) a simple summary mode that gives you a progress bar per service, and then an error summary at the end. This would be vaguely reminiscent of py.test output. The basic idea is you get progress indication while the command is running, and then a simple log of actionable outputs (error messages) as different steps finish. Here is a simple mockup:
$ forge deploy
svc-A  [####################################]  100%
svc-B  [################################----]   90%  0d 00:00:01
svc-C  [####################################]  100%
svc-D  [####################################]  100%

Errors:
... # detailed info about errors here, this would likely be error output from a subcommand, e.g. kubectl apply failed on input, or the docker build failed
  1. We could offer an option to output a completely linear log format, in which case we simply need to be clear when context switches occur. The most difficult example of this is displaying output from concurrent docker build commands. Something like this could work for that:
$ forge deploy
...
>> svc-A:php/Dockerfile:build <<
some build output
some more build output
>> svc-A:nginx/Dockerfile:build <<
some parallel build output from nginx container
some more parallel build output from nginx container
>> svc-A:php/Dockerfile:build <<
some even more build output
...

2.5 Same as (2) but formatted to be more machine friendly, e.g. prefix each line with context rather than just indicating context switches.

  1. One way to deal with clearly differentiating output from different sources (whether different commands or different parallel tracks) is to assign different colors to different sources, e.g. always show docker build output in one color, always show kubectl output in another color, and always show forge output in a third color.

Those are my thoughts, but I'd be interested if you have any ideas on what you'd like to see here or any examples of tools you like that deal with similar problems?

@tristanpemble

This comment has been minimized.

Contributor

tristanpemble commented Aug 28, 2017

I like the progress bars, and was thinking a similar thing. I'm not sure how you would be able to measure progress, but if you think that they can be somewhat meaningful then it's probably a good direction. In the "success path", it would be great -- all you would see is the progress bars moving and a nice "Done!" message.

Stern has great output for tailing output logs from multiple pods:

The colored prefixing could be something to draw inspiration from.

I feel like if errors streamed out in a style similar to Stern, as they happen, below the progress bars, it would be useful. The progress bars could also correlate the color with the log messages, and when it fails, the progress bar could indicate so.

@tristanpemble

This comment has been minimized.

Contributor

tristanpemble commented Aug 28, 2017

Something roughly like this maybe:

screen shot 2017-08-28 at 1 49 30 pm

And this for failure:

screen shot 2017-08-28 at 1 51 29 pm

@rhs

This comment has been minimized.

Contributor

rhs commented Aug 28, 2017

Thanks, this is helpful. I will see how close I can get. As you mention, the progress indication might be a bit tricky, but I can think of a few things to try there, and worst case I think it would still be an improvement even with relatively coarse grained progress updates.

@rhs

This comment has been minimized.

Contributor

rhs commented Sep 29, 2017

I just released a forge 0.3.1 with revamped output based on this thread. I think I was able to capture the spirit of the multiplexed output discussed.

I didn't end up getting anything workable with scrollbars. Basically after playing with it for a bit, I came to the following way of thinking:

Pretty much everything forge does is mostly instantaneous (on a functioning network) with the exception of docker build and docker push, and both of those things have extremely high variance. A first time build or push can take many minutes if no layers are cached, or they can take a few seconds, and there isn't really a good way for me to predict this. A scrollbar is therefore very likely to get stuck in one spot and make it appear is if forge is hanging under some circumstances.

So to deal with this I ended up going with a fairly simple scheme where by default docker build and docker push output are relayed (multiplexed if necessary) to the user, followed by a success/error summary at the end. The verbose option operates similarly except outputs all commands and http interactions, not just docker build & push.

I think this is definitely better than before, although I'm sure more usage will elicit more useful feedback. ;-) I'm gonna call this issue done, but please submit more issues if you have any thoughts on improving the new output.

@rhs rhs closed this Sep 29, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment