add diffs back when errors happen during testing#42
add diffs back when errors happen during testing#42jakecoffman wants to merge 3 commits intomainfrom
Conversation
|
We've been getting along nicely without this and relying on the diff of the output, so I think I'll just close this for now. |
|
Thanks for #58, it def felt like we needed some sort of middle ground between super-noisy-diff and none at all... One other thought is that if there's a problem, adding a hint to the log saying "to see the diff, do X" because if we start recommending this in place of dry-run to drive-by users, we're going to get lots of questions... |
|
Maybe when there's an error while running in |
|
Personally, that feels too clever for me... just a simple "if you want to see the diff, run with |
Following up #40 this adds output of a diff between expected and actual when there is a missed assertion of the output.
I went with a unified-diff style diff this time.
To keep an excessive amount of output from hitting the terminal I added explicit checks to the content of the DependencyFiles, so it will simply say there's no match there. In our smoke tests we still have a "Diff" step which will reveal the problem, and locally we can run with
-oto output the actual results for diffing if needed.I'll probably run this locally for a little while to make sure it's good and maybe add a test so I'll just put this up as draft for now.