Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit tests for completion models #1572

Merged
merged 20 commits into from Jul 1, 2016

Conversation

@rcorre
Copy link
Contributor

@rcorre rcorre commented Jun 11, 2016

This establishes a pattern that can probably be used to test any of the
completion models.

See #999.

Let me know if this pattern looks good, and I'll start testing the other models.


This change is Reviewable

@rcorre rcorre force-pushed the rcorre:completion_tests branch from 0de4324 to f9bf839 Jun 12, 2016
@rcorre
Copy link
Contributor Author

@rcorre rcorre commented Jun 12, 2016

Just realized there is an easy(ish) way to stub cmdutils.cmd_dict. Gonna do that, ignore this for now.

@rcorre rcorre force-pushed the rcorre:completion_tests branch from f9bf839 to bc234da Jun 13, 2016
assert actual["Commands"] == [('drop', 'drop all user data', ''),
('rock', "Alias for 'roll'", ''),
('roll', 'never gonna give you up', 'rr'),
('stop', 'stop qutebrowser', 's')]

This comment has been minimized.

@The-Compiler

The-Compiler Jun 13, 2016
Member

😆 👍

...
}
"""
completions = {}

This comment has been minimized.

@The-Compiler

The-Compiler Jun 13, 2016
Member

Maybe this should be a collections.OrderedDict so future tests could also check the order of completions?

config_stub.data['aliases'] = {'rock': 'roll'}
key_config_stub.set_bindings_for('normal', {'s': 'stop', 'rr': 'roll'})
actual = _get_completions(miscmodels.CommandCompletionModel())
assert "Commands" in actual

This comment has been minimized.

@The-Compiler

The-Compiler Jun 13, 2016
Member

assert actual.keys() == ["Commands"] maybe to ensure there are no other categories?


"""Tests for completion models."""

import pytest

This comment has been minimized.

@The-Compiler

The-Compiler Jun 13, 2016
Member

unused import

import pytest

from qutebrowser.completion.models import miscmodels
from qutebrowser.commands import cmdutils

This comment has been minimized.

@The-Compiler

The-Compiler Jun 13, 2016
Member

unused import

@The-Compiler
Copy link
Member

@The-Compiler The-Compiler commented Jun 13, 2016

Nice! I added a few comments, but looks like a simple good way to test completion models indeed.

@rcorre rcorre force-pushed the rcorre:completion_tests branch from bc234da to a97e430 Jun 13, 2016
(CategoryName: [(name, desc, misc), ...]),
(CategoryName: [(name, desc, misc), ...]),
...
]

This comment has been minimized.

@rcorre

rcorre Jun 13, 2016
Author Contributor

Just went with a list for the whole thing here so you can check the order of the categories too.

@rcorre rcorre force-pushed the rcorre:completion_tests branch 4 times, most recently from 09570e3 to acb46d3 Jun 14, 2016
@codecov-io
Copy link

@codecov-io codecov-io commented Jun 17, 2016

Current coverage is 82.21%

Merging #1572 into master will increase coverage by 0.74%

@@             master      #1572   diff @@
==========================================
  Files           106        106          
  Lines         15371      15374     +3   
  Methods           0          0          
  Messages          0          0          
  Branches       2462       2463     +1   
==========================================
+ Hits          12523      12640   +117   
+ Misses         2378       2260   -118   
- Partials        470        474     +4   

Powered by Codecov. Last updated by f6fbb09...501319c

@The-Compiler
Copy link
Member

@The-Compiler The-Compiler commented Jun 17, 2016

Is this ready to review/merge?

@rcorre
Copy link
Contributor Author

@rcorre rcorre commented Jun 17, 2016

At this point I might as well finish the rest of the models. I think I just have the config models left. I've been grouping the models into separate commits in case its easier to review that way.

@rcorre rcorre force-pushed the rcorre:completion_tests branch 3 times, most recently from c7a242d to befd1f2 Jun 23, 2016
@rcorre
Copy link
Contributor Author

@rcorre rcorre commented Jun 27, 2016

@The-Compiler assuming the CI tests pass, I think this is ready to go.

@rcorre
Copy link
Contributor Author

@rcorre rcorre commented Jun 27, 2016

Looks like something is awry with the osx env

@The-Compiler
Copy link
Member

@The-Compiler The-Compiler commented Jun 30, 2016

Sorry for the delay - after failing to properly fix this, I now marked OS X tests as allowed to fail again for the time being... Reviewing this now!


def __iter__(self):
"""Iterate over all set values."""
return self.values.__iter__()

This comment has been minimized.

@The-Compiler

The-Compiler Jun 30, 2016
Member

nitpick: This should be iter(self.values) (just like you wouldn't do self.values.__getitem__(key) below) 😉

This comment has been minimized.

@rcorre

rcorre Jun 30, 2016
Author Contributor

Copied from the real config section code :)
I can go ahead and change it there too

This comment has been minimized.

@The-Compiler

The-Compiler Jun 30, 2016
Member

Oh, my bad! 😆 Thanks for fixing it!

return self.values[key]


class FakeSettingValue:

This comment has been minimized.

@The-Compiler

The-Compiler Jun 30, 2016
Member

Not sure if it really pays off to have a fake for this - couldn't you easily use the real SettingValue object without pulling it too much unrelated code?

Though I guess this makes it easier to customize valid_values and default for tests?

This comment has been minimized.

@rcorre

rcorre Jun 30, 2016
Author Contributor

Though I guess this makes it easier to customize valid_values and default for tests?

Exactly. If I use a real SettingValue, then I have to use a real ConfigType, and my test becomes flaky with regards to changes in configtypes (e.g. if someone added an extra option to IgnoreCase).

@@ -400,9 +452,114 @@ def set_bindings_for(self, section, bindings):
self.bindings[section] = bindings


class FakeHistoryEntry:

This comment has been minimized.

@The-Compiler

The-Compiler Jun 30, 2016
Member

Same as above, I think you could just use webkit.history.Entry here?

rcorre added 15 commits Jun 17, 2016
This involved adding a few stubs as well as expanding the FakeWebView.
test_init in test_history was leaving a web-history in the objreg,
which was interfering with the completion tests.
Had to add the `raw` parameter to ConfigStub.get as the setting option
completion model passes this argument explicitly (although the docs say
only raw=True is supported).
Validate that quickmarks and bookmarks can be deleted from the url
completion menu.
Reduce duplicate code by mocking out a QTreeView around the model and
setting its index.
It now has 100% test coverage.
Corrent two places this was used.
Use webkit.history.Entry in tests instead of FakeHistoryEntry.
Add the view created during the tests to qtot so it gets cleaned up
after the test exits.
@rcorre rcorre force-pushed the rcorre:completion_tests branch from 501319c to 4178f73 Jun 30, 2016
@rcorre
Copy link
Contributor Author

@rcorre rcorre commented Jun 30, 2016

Rebased on master to pick up the osx test 'fix'. The last 4 commits are from this code review. I've still got some more un-mocking to do, I think.

@The-Compiler
Copy link
Member

@The-Compiler The-Compiler commented Jun 30, 2016

Definitely. It depends on whether you really think of these 'unit' tests as 'isolation' tests, or whether they could have some elements of an integration test. I was leaning towards the former, but it definitely feels like I mocked out too much.

Yeah, I'm usually leaning a bit more towards the latter.

I think the benefit is clear: More straightforward and often more maintainable tests.

The drawbacks I can think of:

  • Slightly slower tests - I don't think it makes much of a difference, unless
    you have things like network access, which should definitely be mocked out.
  • Unrelated code changes cause unit tests to fail - I think here the benefits
    simply outweight this drawback, and if you mocked things out, you need to
    take care of keeping the mocks up-to-date as well
  • Coverage can be a bit skewed, i.e. a module can appear covered even though
    it's just from other tests "smearing" to that module, and it's not actually
    tested well. To take care of that, I added a mode to run test files
    one-by-one to check_coverage.py, though I haven't actually checked if that
    still works for a while 😆
'web-history-max-items': 2}
model = urlmodel.UrlCompletionModel()
# delete item (1, 0) -> (bookmarks, 'https://github.com' )
view = _mock_view_index(model, 1, 0)

This comment has been minimized.

@The-Compiler

The-Compiler Jun 30, 2016
Member

I think you need to do qtbot.add_widget(view) here too.

'web-history-max-items': 2}
model = urlmodel.UrlCompletionModel()
# delete item (0, 1) -> (quickmarks, 'ddg' )
view = _mock_view_index(model, 0, 1)

This comment has been minimized.

@The-Compiler

The-Compiler Jun 30, 2016
Member

I think you need to do qtbot.add_widget(view) here too.

rcorre added 2 commits Jul 1, 2016
Don't really need to mock these out for tests as the real classes are
simple enough.
@rcorre
Copy link
Contributor Author

@rcorre rcorre commented Jul 1, 2016

I'm pretty much in complete agreement about mocking. I've seen tests that mock out so much they don't even seem to be testing anything ...

I managed to eliminate the fake setting sections and values; I kept around FakeConfigType because its an easy way to mock completions.

I remember now why I didn't use a real Command: the constructor takes the handler method and tears it apart to populate its fields, and that's way more than I wanted to test here. I can take a look at test_cmdutils/test_argparser and see how rigorous they are.

I've got a test for the Completer in the pipeline as well -- should I push that up here or would you rather get this through and review that in another PR?

@The-Compiler The-Compiler self-assigned this Jul 1, 2016
@The-Compiler The-Compiler merged commit d45acb0 into qutebrowser:master Jul 1, 2016
1 of 2 checks passed
1 of 2 checks passed
continuous-integration/travis-ci/pr The Travis CI build failed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
@The-Compiler
Copy link
Member

@The-Compiler The-Compiler commented Jul 1, 2016

Let's get this in first - thanks!

@rcorre rcorre deleted the rcorre:completion_tests branch Jul 1, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants