Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LocalBearTestHelper: Add assertObjectsEqual #4310

Merged
merged 1 commit into from Feb 21, 2018

Conversation

yash-nisar
Copy link
Member

@yash-nisar yash-nisar commented Jun 2, 2017

Add assertObjectsEqual(..) method that compares individual
fields of the Result object and yields better, easy to understand
messages in case of an attribute mismatch.

Closes #4302

For short term contributors: we understand that getting your commits well
defined like we require is a hard task and takes some learning. If you
look to help without wanting to contribute long term there's no need
for you to learn this. Just drop us a message and we'll take care of brushing
up your stuff for merge!

Checklist

  • I read the commit guidelines and I've followed
    them.
  • I ran coala over my code locally. (All commits have to pass
    individually.
    It is not sufficient to have "fixup commits" on your PR,
    our bot will still report the issues for the previous commit.) You will
    likely receive a lot of bot comments and build failures if coala does not
    pass on every single commit!

After you submit your pull request, DO NOT click the 'Update Branch' button.
When asked for a rebase, consult coala.io/rebase
instead.

Please consider helping us by reviewing other peoples pull requests as well:

The more you review, the more your score will grow at coala.io and we will
review your PRs faster!

@yash-nisar
Copy link
Member Author

status/blocked on coala_utils release which will include https://gitlab.com/coala/coala-utils/merge_requests/42

@yash-nisar
Copy link
Member Author

CC @Makman2

@yash-nisar
Copy link
Member Author

Note that the tests will fail because we still don't have the __compare_fields attribute which will only be added after https://gitlab.com/coala/coala-utils/merge_requests/42 is merged and a new version of coala_utils is released(maybe).

@@ -185,10 +201,11 @@ def check_results(self,
create_tempfile=create_tempfile,
tempfile_kwargs=tempfile_kwargs,
settings=settings)
self.addTypeEqualityFunc(list, self.assertObjectsEqual)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. This should be run somewhere in the beginning of a base test class or so, otherwise it takes effect only in check_validity
  2. list doesn't use generate_eq or generate_repr, so this is not really working

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we're basically comparing list of Result objects, so whenever we pass lists to the assertEqual(...) for comparison, it will call the assertObjectsEqual(...) method IIUC. What I had initially done was put Result object instead of list since we wanna call that method whenever we're comparing Result objects, see below :
Snippet: self.addTypeEqualityFunc(list, self.assertObjectsEqual)
Output:

E   AssertionError: 'mixed spaces and tabs' != 'mixed spaces and tab'
E   - mixed spaces and tabs
E   ?                     -
E   + mixed spaces and tab
E    : message_base mismatch

Snippet: self.addTypeEqualityFunc(Result, self.assertObjectsEqual)
Output:

E   AssertionError: Lists differ: [<Result object(id=0xf33dc5cf898c48b4b7114cffd86e8172, origin='[490 chars]438>] != [<Result object(id=0xf211aef4abb24ecb8cc36beebbd848a8, origin='[489 chars]d30>]
E   
E   First differing element 0:
E   <Result object(id=0xf33dc5cf898c48b4b7114cffd86e8172, origin='[489 chars]a438>
E   <Result object(id=0xf211aef4abb24ecb8cc36beebbd848a8, origin='[488 chars]2d30>
E   
E   Diff is 2240 characters long. Set self.maxDiff to None to see it.

CC @Makman2

Copy link
Member Author

@yash-nisar yash-nisar Jun 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I create an __init__ method inside our LocalBearTestHelper class and put addTypeEqualityFunc(..) there ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another alternative that I've found is we pass the Result object to assertEqual which is something like(Error handling of empty lists, and so on will be done..) :

        if not check_order:
            self.assertEqual(sorted(bear_output)[0], sorted(results)[0])
        else:
            self.assertEqual(bear_output[0], results[0])

Then self.addTypeEqualityFunc(Result, self.assertObjectsEqual) snippet is justified IIUC.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ouch haven't thought about that we pass results with lists... here we could just iterate over all passed results and compare them one by one with assertEqual. And we still register addTypeEqualityFunc with Result globally (or an ABC-type of classes using generate_eq and generate_ordering which would be the ideal solution).

Hm this is better than nothing, but ideally we would be able to have nicer outputs for collections... don't know something yet, maybe you also get some ideas how to make self.assertObjectsEqual working automatically everywhere in our tests, even those ones not testing only bears.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ouch haven't thought about that we pass results with lists... here we could just iterate over all passed results and compare them one by one with assertEqual. And we still register addTypeEqualityFunc with Result globally (or an ABC-type of classes using generate_eq and generate_ordering which would be the ideal solution).

Aren't we doing this already in #4310 (comment) ?

Hm this is better than nothing, but ideally we would be able to have nicer outputs for collections... don't know something yet, maybe you also get some ideas how to make self.assertObjectsEqual working automatically everywhere in our tests, even those ones not testing only bears

Our main concern ATM is the not so useful output of bears so IMO we should get this basic functionality merged and those not testing only bears can be looked after later on ? What say ?

@gitmate-bot
Copy link
Collaborator

Hey! This pull request hasn't been updated for a while :/ It would be nice if we could get this going again!

@@ -185,10 +201,11 @@ def check_results(self,
create_tempfile=create_tempfile,
tempfile_kwargs=tempfile_kwargs,
settings=settings)
self.addTypeEqualityFunc(list, self.assertObjectsEqual)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could actually use Iterable instead of only list, so we support any kind of iterable 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe for this case we should explicitly use assertObjectsEqual instead of registering it for tests. Makes it easier to understand^^

else:
self.assertEqual(bear_output, results, msg=msg)
self.assertEqual(bear_output, results)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we delete the msg variable above as we don't seem to use it any more?

@yash-nisar
Copy link
Member Author

Will fix things suggested by the review comments in the next iteration and iterate over the list containing more than multiple results.

@yash-nisar yash-nisar changed the title LocalBearTestHelper: Add assertObjectsEqual WIP LocalBearTestHelper: Add assertObjectsEqual Jun 19, 2017
@yash-nisar
Copy link
Member Author

Updated @Makman2

@@ -25,7 +26,7 @@
'aspect',
'additional_info',
'debug_msg')
class Result:
class Result(Comparables):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to register Result to be a deriving type of Comparables inside the generate_eq/generate_ordering decorators, not here 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I just got to know the difference between the 2 ways of subclassing, the real one and the virtual one. I had actually referred to https://pymotw.com/3/abc/ which doesn't explicitly state the difference. My bad . :)

@gitmate-bot gitmate-bot added size/S and removed size/M labels Jun 21, 2017
@yash-nisar yash-nisar force-pushed the improve-testing-api branch 2 times, most recently from 38dbea6 to 07bd68d Compare June 22, 2017 10:11
@gitmate-bot gitmate-bot added size/M and removed size/S labels Jun 22, 2017
@yash-nisar yash-nisar force-pushed the improve-testing-api branch 2 times, most recently from b994c82 to 122dffa Compare June 22, 2017 13:06
@yash-nisar
Copy link
Member Author

How does this look ? @Makman2

@gitmate-bot gitmate-bot removed the process/approved The PR is approved and will be merged soon label Feb 2, 2018
@yash-nisar
Copy link
Member Author

@Makman2 Ready for merge ?

@gitmate-bot
Copy link
Collaborator

Sorry @yash-nisar, you do not have the necessary permission levels to perform the action.

@Makman2
Copy link
Member

Makman2 commented Feb 14, 2018

ack 14642d8

@gitmate-bot gitmate-bot added process/approved The PR is approved and will be merged soon and removed process/pending review labels Feb 14, 2018
@Makman2
Copy link
Member

Makman2 commented Feb 21, 2018

@gitmate-bot rebase

@gitmate-bot
Copy link
Collaborator

Hey! I'm GitMate.io! This pull request is being rebased automatically. Please DO NOT push while rebase is in progress or your changes would be lost permanently ⚠️

Add ``assertObjectsEqual(..)`` method that compares individual
fields of the Result object and yields better, easy to understand
messages in case of an attribute mismatch.

Closes coala#4302
@gitmate-bot
Copy link
Collaborator

Automated rebase with GitMate.io was successful! 🎉

@gitmate-bot gitmate-bot added process/pending review process/approved The PR is approved and will be merged soon and removed process/approved The PR is approved and will be merged soon process/pending review labels Feb 21, 2018
@Makman2
Copy link
Member

Makman2 commented Feb 21, 2018

@gitmate-bot ff

@gitmate-bot
Copy link
Collaborator

Hey! I'm GitMate.io! This pull request is being fastforwarded automatically. Please DO NOT push while fastforward is in progress or your changes would be lost permanently ⚠️

@gitmate-bot
Copy link
Collaborator

Automated fastforward with GitMate.io was successful! 🎉

@gitmate-bot gitmate-bot added process/approved The PR is approved and will be merged soon and removed process/approved The PR is approved and will be merged soon labels Feb 21, 2018
@gitmate-bot gitmate-bot merged commit 31ad864 into coala:master Feb 21, 2018
@gitmate-bot gitmate-bot added the process/approved The PR is approved and will be merged soon label Feb 21, 2018
@gitmate-bot gitmate-bot removed the process/approved The PR is approved and will be merged soon label Jun 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tests difficulty/medium review carefully! size/S status/blocked The issue requires other referenced issues/PRs to be solved/merged before being worked on status/stale
Development

Successfully merging this pull request may close these issues.

Testing API doesn't yield useful results
6 participants