feat: add support for json output for notation verify

[byronchien]

allows json output for notation verify.

fixes notaryproject/roadmap#67, #498

Example output:

chienb@a07817b52895 notation % ./bin/notation verify $IMAGE --output json
{
    "reference": "localhost:5000/net-monitor@sha256:5a07385af4e6b6af81b0ebfd435aedccdfa3507f0609c658209e1aba57159b2b",
    "userMetadata": {
        "foo": "bar"
    },
    "result": "Success"
}

Note: PR is on top of changes that haven't been merged into main yet, so there's duplicate code from this PR adding metadata support.

suggested order for review: notation-go #261 => notation-core-go #111 => notation #527 (this one) =>notation #528

[byronchien]

order to review:
#507
#527
#528

[shizhMSFT]

There is no issue linked with this PR. Could you link at least one issue to this PR so that reviewers can have context?

Since this is a new feature, could you add E2E tests accordingly?

[byronchien]

There is no issue linked with this PR. Could you link at least one issue to this PR so that reviewers can have context?

Since this is a new feature, could you add E2E tests accordingly?

linked the related metadata PR and issue in the initial comment. will add E2E tests for this PR

[codecov-commenter]

Codecov Report

Merging #527 (79b3217) into main (4f573af) will decrease coverage by 0.15%.
The diff coverage is 3.70%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main     #527      +/-   ##
==========================================
- Coverage   36.34%   36.20%   -0.15%     
==========================================
  Files          30       30              
  Lines        1607     1616       +9     
==========================================
+ Hits          584      585       +1     
- Misses       1002     1010       +8     
  Partials       21       21              

Impacted Files	Coverage Δ
cmd/notation/verify.go	23.20% <3.70%> (-0.94%)	⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[patrickzheng200]

LGTM

[shizhMSFT]

What if the verification fails?

[byronchien]

for verification failure, no json is written to stdout, and the failure is logged to stderr.

[shizhMSFT]

@byronchien If JSON is printed only if the verification passes, what's the meaning of showing "result": "Success"?

/cc @priteshbandi @yizha1

[shizhMSFT]

When verification fails, I expect that the result would be failure or failed, and the failure reason will be included in the JSON object so that other scripts can parse it correctly.

[shizhMSFT]

Since this PR is merged, I've created #546 to track.

[priteshbandi]

result can have two values, one is skip and other one is success. IMO for failure case displaying JSON for wont be useful because it wont contain any useful information which automation/script can use(non-zero exit code should suffice to show failure)

[shizhMSFT]

The JSON output is usually consumed by scripts or programs. How can them obtain the structured error message and detailed verification outcomes?

[priteshbandi]

To display error message/detailed verification output we need more than result field something like resultReason because result will only say failure. Also, in current state it wont be of much help since as there is only one genuine/expected failure use case i.e failed signature verification for all the signatures. Apart from genuine/expected error, all other errors should always be emitted as stdErr.

Are you suggest we should emit json for expected failure use cases or all failure use cases?

---

Moved conversation to #546 (comment)