Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare test results #5

Open
3 tasks
heinezen opened this issue Dec 24, 2022 · 0 comments
Open
3 tasks

Compare test results #5

heinezen opened this issue Dec 24, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@heinezen
Copy link
Collaborator

TestCase should implement a compare() method that can be used to compare the results of two checks. Comparing results can be useful for the following scenarios:

  1. Reproducing a test run (e.g. for checking if a detected issue has been fixed)
  2. Diffing results of two tests of the same type (e.g. for comparing the APIs reaction to slighlty different API requests)

compare() should be implemented as a classmethod that gets passed two report objects. It should then do the following:

  • Check for differences in the issue field.
  • Diff the value field. Complexity for this may vary depending on the test case, since there can be optional fields and nested values.
  • Return a comparison object (as a dict). The object should contain a flag that indicates whether the results are a match/mismatch and the created diff.
@heinezen heinezen added the enhancement New feature or request label Dec 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant