kpt fn error reporting and CLI output

[droot]

kpt fn render should report errors with useful diagnostic info that helps the end-users as well as kpt developers. We should explore if we kpt can support error reporting explained in this post by Rob Pike. The approach in that post tries to strike the right balance with needs of both the personas.

In addition, being able to trace input/output for each function execution of the pipeline can assist in function level debugging as well.

[droot]

This is addressed with #1634

[frankfarzan]

Reopening this since we need to hash out the UX for error reporting and CLI output.

A list of all the data involved in running functions:

1. What function provides:
   1. ResourceList.results
   2. ResourceList.items
   3. stderr
   4. exit code
2. Orchestrator provides metadata:
   1. What function is running (Specially important when running a pipeline)

We need to figure out the best UX for providing all or subset of the following to the user across:

1. CLI output
   1. stdout
   2. stderr
2. FunctionResults files

As part of this, need to consider happy paths and failure paths:

1. Happy path: The pipeline ran successfully
2. Failure paths:
   1. Function exceptional failures (e.g. function segfaulted)
   2. Validator failures which are WAI (kubeval found an issue)
   3. Orchestrator failed for some reason (e.g. IO failures or user input issues)

Some principles to follow:

1. Don’t pollute the CLI output unnecessarily (Especially in the happy path)
2. Don't leak internal implementation details. For example, for validator failures, the user shouldn't see a stack trace with internal identifiers.
3. Assume any function in the pipeline will have large amount of results and stderr. Truncation may be necessary (May be controlled through a flag)
4. Provide realtime information about what’s happening, especially for long-running operations. The CLI should not appear stuck for long periods of time with no progress.
5. Follow and add to the style guide to ensure consistency across commands: go/kpt-errors-style-guide

[frankfarzan]

#1783 should be addressed as part of this.

[frankfarzan]

As part this, need to add e2e test cases for both render and eval: go/kpt-test-cases

[mortent]

Related to this, the current output while a pipeline is running is a little confusing. It prints

package ".": running function "gcr.io/kpt-fn/kubeval:v0.1": 

with the caret blinking at the end of the line. It looks like we expect some kind of user input.

[frankfarzan]

A design proposal for the UI:

https://docs.google.com/document/d/1mWZyYcCWmFUZuNNG3O0r-Y-FOk256JS-1mVpNRxeuMg/edit?hl=en#

[natasha41575]

Adding that we should address the concerns raised here:
#1839 (review)

[Shell32-Natsu]

The works remained for this task are:

1. Aggregate structured results
2. Create results file.
3. Hash output for the behavior with CLI stdin and stdout

These tasks are coved by umbrella issue #1390. Subtasks: #1920, #1915, #1873

[Shell32-Natsu]

Close this with #1783 . The only thing remained in m2 is #1915. Please create new issues for bugs or features.