what is our cohesive logging and output UX experience

[tayloramurphy]

Opened this to discuss and document what our goals are for the user experience on the command line. Triggered by discussion in #6103

The command line should be offer completely functionality for working with your Meltano project, it should be nice to use, and it shouldn't try to be too cute.

• We will not invest in a full TUI. https://charm.sh/ is cool but is likely a distraction.
• We will make the experience better with better output structure and formatting with tools like https://github.com/Textualize/rich-cli
• We will make the experience better by having and applying clear design guidelines docs: CLI design guidelines  #6539
• Visual enhancements will be used thoughtfully and sparingly
• Users will be able to run all the actions they would need to build, manage, and run their meltano project via the command line, but they may not get all of the information they need there (think not putting everything that's on the Hub on the CLI

Inspirations:

• https://blog.developer.atlassian.com/10-design-principles-for-delightful-clis/
• https://clig.dev/

Related:

• Improve readability of Meltano's console output #7016

[tayloramurphy]

@meltano/engineering @pnadolny13 would love your opinions on guiding principles around CLI output and UX.

[edgarrmondragon]

+1 on clig.dev guidelines, since they seem to produce better CLI experiences. This advice

Prefer flags to args. It’s a bit more typing, but it makes it much clearer what is going on. It also makes it easier to make changes to how you accept input in the future. Sometimes when using args, it’s impossible to add new input without breaking existing behavior or creating ambiguity.
clig.dev

was useful when designing meltano lock.

---

One of the things users seem like the least from Meltano's output is tracebacks and error messages. We should capture all known exceptions as succinct and clear one-line messages, and only unknown exceptions should display the full traceback.

Our current output format makes it hard to even parse what the error was. That's what probably drives some users to post screenshots or walls of output in Slack, because they don't even know what failed. It also makes it hard to search for the same error message in GitHub issues.

[pandemicsyn]

We should start being pretty strict about what we write to the console by default and limiting it to critical progress updates, or actionable info.

It's a tricky balance because, day to day being terse is great - but when you start hitting issues or when you're just getting started, terse is rough. Having extra info available at a glance, like what stream's are being skipped or what env is active is handy at times.

I think we should consider a pattern where we always write to two spots.

1. The console with fairly terse and to the point events only (unless --debug is enabled)
2. A meltano.log. This probably still shouldn't default to DEBUG level though, just INFO. We already technically do this today, we just bury the log in a terrible location and give folks no way to control it.

That would let us design an experience something like:

$ meltano run tap-gitlab target-postgres dbt:run
-> Run is being logged to meltano.log with run_id=BLAH_BLAH_BLAH_42
Invoking block 1 of 2 comprised_of=['tap-gitlab', 'target-jsonl'] 
Finished block 1 of 2 comprised_of=['tap-gitlab', 'target-jsonl'] 
Invoking block 2 of 2 comprised_of=dbt:run
Finished block 2 of 2 comprised_of=dbt:run

The actual content can be whatever we want or structured however we want. The point is for that run statement, on a successful execution, you most likely don't need more than 4 lines by default. If something goes haywire, or you want to see what happened behind the scenes, you can look at meltano.log.

That first line also spits out a unique run_id , so if you're savvy, you can also grep BLAH_BLAH_BLAH_42 meltano.log to get logs only for this execution.

If we want to be really fancy we could create a meltano log cli command to let users manage/view that log file:

$ meltano log cat --run-id=BLAH_BLAH_BLAH_42
<all the usual logs>

$ meltano log tail --run-id=BLAH_BLAH_BLAH_42 -n 10
<last 10 lines of that run id>

Having a log command would translate nicely to also support invoking a command via a Meltano API instance.

[pandemicsyn]

Kind of an aside, but there's definitely an easy/interim fix out there of going through our existing output and re-leveling some log lines, making sure we use named loggers where needed, and then switching to a default logging config that's more selective about what the default log level is for certain modules or paths. e.g. meltano.core.runner defaults to only WARN and above while meltano.cli.elt defaults to INFO and above.

[tayloramurphy]

@pandemicsyn I generally like the simplified proposal, but I do worry about swallowing info that the plugins are sharing. Specifically in the dbt case, they have a fairly terse output as well, listing whether or not specific models have run or not, but that I'd still like to see on the command line.

Perhaps a --verbose (or --terse) option would make sense where the terse output is just what Meltano is doing like you've indicated and verbose is still INFO but everything from Meltano and plugins?

I do like the idea of improving where and how we store the logs though 👍

[pandemicsyn]

Specifically in the dbt case, they have a fairly terse output as well, listing whether or not specific models have run or not, but that I'd still like to see on the command line.

@tayloramurphy Definitely understand that, and you (the product you) should be able to be opinionated about what we do and don't display by default. Its hard for us to be super granular, since we're just ingesting a bunch of random output. If you said "I want to show these specific types of lines from the dbt output" that's probably do able but tricky and would take some hackery.

At broad levels though, you can say "meltano run should be terse for this class of plugins, but for transformers please still display all the output" and we can fairly easily make that happen for you.

[pandemicsyn]

@tayloramurphy oh! Just realized, for dbt specifically, and other things that are soon going to be extensions. This probably gets easier. The extension developer can be opinionated and wrangle, format, and filter output the output it produces - and meltano for the most part doesn't have to deal with it directly. There's some hand wavy assumptions in there, but in general - I think the extension spec (if extended to include output) and the extension sdk can help a lot here.

[tayloramurphy]

@pandemicsyn beat me to it :) I was going to say that the extension definition should handle this more!

This does make it easier because we can then be very clear about Meltano output and show where extension/component output is interleaved.

[pandemicsyn]

Mostly sharing to see if it triggers cool ideas, but I like how some of the frontend tools mix eye candy and raw ouput. For example, this is the output when adding a new integration to the astro framework:

Things I like:

1. It shows a diff of the change its going to make to my config (essentially the equivalent of the meltano.yml)
2. It shows me what command its going to run behind the scenes
3. Still gives me the raw output from the npm command, but I think level filtered.
4. If something blows up, shows the errors inline when it can, and usually, where to find the log. (like when pip has issues, the npm experience leaves a lot to be desired though when things fail)