-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AssertionFailedException: What's the benefit of the expected value? #21
Comments
Also the actual state can be much more complex than to put it into a simple value:
It would be great if upon failure I could directly open the accompanying files like HTML dump or the screenshot from the following sources:
So what we really need is a bunch of debug information upon failure. Even more mission critical for integration and system tests where you are often not able to reproduce the failure locally. So perhaps an exception could carry just an extra object DebugInformation which is:
This DebugInformation could be contained in any of the existing exceptions. The more and more and I think about a DebugInformationObject the more I like it. We could even have specialized DebugInformationObjects. One for console output (the default, I guess), one for Eclipse, one for IntelliJ Idea, one for Jenkins, ... Just like a clipboard it could have different flavors and the assertion framework developer might decide to support different flavors as debug information. Ok, think I should now wait and see what you think about this idea. Is it worth to give it a try? |
It's certainly true that the current |
Introducing a more generic way to provide additional information for debugging upon assertion failures. The DebugInformation is key-value based and can contain actual any information. In addition to this extended the ValueWrapper (now renamed to ValueDescriptor as it *describes* the value) so that it does some effort to serialize objects applying some strategies. This way we could also provide images as for example for UI tests. The current state neither includes documentation, more tests and misses the commonly used exception for all exceptions provided inside here. This approach also addresses issues: * ota4j-team#20 support for exception creation strategies * ota4j-team#4 detach AssertionError from exceptions here * ota4j-team#9 partially, as you the debug information provided here should eventually merge into a standard test report.
* Introduces DebuggableException as base for all exceptions which should provide debug information. * Makes AssertionFailedException a DebuggableException Keeps actual/expected constructors for now. * MultiFailuresException: Adjust newline to system line.separator * Repackages DebuggableException-classes * Introduces test helpers like for serialization and to do assertions on exceptions.
* Refactored ExceptionTestHelper to provide better/easier to understand support for testing DebuggableExceptions. * Introduces test for DebuggableException * Renames methods of DebugInformation to be similar to Map interface.
* Provide a lot more tests. * Remove unused methods from AssertionFailedException about actual/expected.
Adjust Javadoc as the statements about actual/expected values are not valid anymore.
As the decision about ota4j-team#4 is not yet clear and some points (support of old frameworks/tests) this is an alternative introducing the AssertionError again to the hierarchy... with the drawback of duplicated code.
What I like about IDEs knowing about actual and expected: They can clearly show them in console and provide a "diff" function. Currently they have to parse the exception message and guess expected and actual value from it.
But this only makes sense, if we expect a certain value... but already fails for negation, such as not null, or not equal to, ... With the current implementation of AssertionFailedError I see no other option than to leave the expected value empty. But then IDEs and other reporting tools will have less options to highlight the expectation.
So perhaps it is better to misuse the expected value in these cases and provide a descriptive string like "not null", "not equal to". Example:
An IDE might consider it is nice to provide a "diff" button. But in this case the diff would not be of any value.
At least the use case of "expected" should be clearly defined so that assertion framework developers deal with it in the very same way as IDE developers expect it to be.
Side Note: Perhaps it might make sense to even provide some on-demand-diff hook if the assertion frameworks knows better how to display the difference.
The text was updated successfully, but these errors were encountered: