Add --debug option to Notation CLI Sign and Verify commands to help with troubleshooting and logging

[iamsamirzon]

As a user, I want the Notation Sign and Verify commands to generate additional info why a given CLI command is failing and/or to create logs of the actions being performed by a command.

When the Notation CLI reports an error that a user can get more detail about the error by running the command again with the --debug option. With this option, the Notation CLI outputs details about every step it takes. The details in the output can help a user to determine when the error occurs and provides clues about where it started. A user should be able to output to a text file for later review, or to post it as part of bug report.

When you include the --debug option, some of the details included in the logs could be.

General

• Registry auth failure details
• Plugin request and responses

Signing workflow

• String to sign
• Signature envelope content

Verification workflow

• Trust store resolution
• Trust policy evaluation
• Details of verification result
• display signature envelope content as part of debug with verify command.

--debug for both sign and verify should also emit signature either in plain text(for jws) or and encoded format(for cose).

[yizha1]

Could you give an example of the following items?

• Parsing the provided parameters
• The formatted output

[gokarnm]

I've updated the list of log items (not exhaustive), and removed "Parse parameters" and "Formatted output".

[dtzar]

Per discussion on call today, I think this makes sense in RC.1 for just the sign and verify commands. 2nd priority may be the login.

[dtzar]

@iamsamirzon Can you please rescope this item to just sign/verify?

[iamsamirzon]

Modified the issue description to just focus on Sign and Verify commands for RC-1. This limited scope was discussed and agreed in the NV2 meeting on 9/8/2022

[iamsamirzon]

@yizha1 - Now that we are doing refactoring, could you adjust the scope on this item down from 3 weeks to 0.8 weeks like you shared yesterday? Also, lets only focus on the use cases called out in this issue and not include "HTTP/REQ/RESP/POST" like things

[yizha1]

Some debug outputs are addressed in the following PRs:

• PR for notation sign: spec: update cli sign spec for tag to digest translation #439
• PR for notation verify: spec: update cli verify spec for UX improvement #440

More suggestions on --debug outputs are welcome. I can make a summary and update the spec accordingly.

Implementation can be kicked off after the spec is updated.

@priteshbandi @iamsamirzon @vaninrao10 @gokarnm @FeynmanZhou @toddysm

[priteshbandi]

--debug for both sign and verify should also emit signature either in plain text(for jws) or and encoded format(for cose).

[yizha1]

update milestone to rc.2 per discussion.

[vaninrao10]

We need to outline or link to the issues to define 1. what work is remaining in this space for rc-2 and 2. What work is completed in rc-1.

[yizha1]

@vaninrao10 I think the debug log for notation sign and notation verify are almost done. We need another issue to add debug log for other notation commands.

[yizha1]

@patrickzheng200 anything left for notation sign and notation verify? If not, we can close this issue

[patrickzheng200]

Closing this issue as it's completed in RC.1.