Identify possible overlay error message categories and apply error message template

[cari-lynn]

What are "helpful error messages"? Ask yourself these five questions to start:

Does the error...

1. focus my attention on the source of the problem?
2. give all of the relevant facts (and none of the irrelevant ones)?
3. consistently use a set of accessible terms?
4. make clear what's wrong?
5. help me figure out how to fix it?

What were the results from the initial round of user testing?

During the first rounds of user testing, we found that our proposed error messages had addressed questions 1-4, but lacked any guidance on how to fix the problem. Since we know that our proposed changes to the error messages are improvements (based on user tests), we think it would be smartest to implement these improvements iteratively, rather than waiting until we have the "perfect" error message.

How

During the user testing stage, we worked out a general template on how we want the new error messages to look, however we only drafted complete errors in three different scenarios.
Template Error Message:

 ytt: Error: Overlaying (in order: <overlay file>):
Expected:
 <overlay file>:<line failure> (<match type>, '<node value>')  to match <expected match number> item in
 <template file>:<line failure> (<match type> in '<parent of match>') but matched <actual match number>.

 <overlay file>:<line failure> 
 <context from overlay file consisting of:
   - document start (---)
   - parent tree down to line of failure
   - annotation matcher above the failure line
   - ellipsis (...) to skip lines such as siblings and unimportant context
  >

 <template file>:<line failure> 
 <context from the template file consisting of:
   - document start (---)
   - parent tree down to lines where the overlay matched
   - ellipsis (...) to skip lines such as siblings and unimportant context 
  >

Since an overlay can match via an expression, the domain of overlay error messages is vast, making it difficult to implement changes. In order to address this, a HackMD Doc was created with a subset of scenarios and proposed errors. This document will also act as the acceptance criteria for the following implementation story.

Acceptance Criteria

At this time, this document is not the source of truth, and we are looking identify more error scenarios. Once an example error message is fleshed out for each major category that is identified, there should be enough information for implementation to begin.

Notes

• First round user interview doc: https://hackmd.io/qOdnXOdHQbmP6i7wTOgKMg?view
• Proposed error message doc: https://hackmd.io/mVrrCDjJQJapmUVUUBeDlw?view
• If you would like to provide feed back on the proposed error message, please capture that in a HackMD document so that we can include your thoughts during the next round of user testing.

[cari-lynn]

Issue moved to vmware-tanzu/carvel-ytt #287 via ZenHub

[cari-lynn]

Closed this because it was opened in the incorrect repo