Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tests to work with latest test dependencies (mock, vcrpy in particular) #1217

Closed
koobs opened this issue May 18, 2019 · 4 comments
Closed
Labels
Improvement This is regarding an improvement to an existing feature Task This is a task related to the library Tests This is regarding the library's tests
Milestone

Comments

@koobs
Copy link

koobs commented May 18, 2019

Several tests fail when testing with the latest versions of test dependencies:

mock 3.x (3.0.4)
vcrpy 2.x (2.0.1)

These are the versions that exist (or are likely to exist) in downstream OS package sets, as they do for FreeBSD Ports in this case.

Most appear to fail with the same class of error, as follows:

TweepError: Failed to send request: No match for the request (<Request (GET) https://api.twitter.com/1.1/followers/ids.json?cursor=-1&id=tweepytest>) was found. Can't overwrite existing cassette ('cassettes/testcursorcu
rsoritems.json') in your current record mode ('none').

If you would like the full set, let me know and I'll include them in a followup.

It would be great if testing was against the latest versions of their test dependencies to pick up bugs/incompatibility issues as early as possible. This ensures downstream consumers (packagers in particular) are able to extensively and reliably QA upstream packages, and ensure a great experience for users.

To achieve this I'd suggest (in a branch):

  • Removing test_requirements.txt in favour of setup.py:tests_require (similar to 778bd7a for requirements.txt/install_requires)
  • Keep test dependencies unpinned (>=) so latest versions are used
  • Note: Add tox dependency to extras_require { 'dev': ['tox'] } (not tests_require) as its not a compulsory requirement (unlike nose, mock vcrpy otoh, which are 'required'). coverage flake and other development only dependencies can go in here too.
  • Use tweepy[dev] to install tox where upstream requires it (travis, wherever)
  • Fix issues associated with mock 3.x, vcrpy 2.x, etc
@Harmon758 Harmon758 added Improvement This is regarding an improvement to an existing feature Task This is a task related to the library Tests This is regarding the library's tests labels May 18, 2019
@Harmon758
Copy link
Member

Harmon758 commented Jul 22, 2019

Regarding test dependencies, I haven't had any issues using the latest versions for all of them, including the ones you specified.

That error you're encountering is due to c880c90, where the default test account was switched from @tweepytest to @TheTweepyTester and cassettes were updated accordingly. That commit missed updating the default testing configuration:

username = os.environ.get('TWITTER_USERNAME', 'tweepytest')
Therefore, the default testing configuration was unable to find cassettes that used the username, tweepytest.

However, c573a28 has updated the default test account and corresponding cassettes again, along with the default testing configuration this time, so this shouldn't be an issue anymore. For more details, see #1242.

As for your additional points, I did and do plan on addressing them as follows:

  • Unpin test requirements: 7f12532
    (in test_requirements.txt and tox.ini)
  • Replace test_requirements.txt with tests_require in setup.py: 1b4d154
  • Separate dev requirements from test requirements using extras_require in setup.py: 18c39a6
    (specifically, separate tox and add coveralls)

This will also likely result in the following changes:

  • Specify nose.collector as the test suite in setup.py: 5413d21
  • Use setup.cfg to specify nosetests verbosity and tests to run: 7d17d6c
    (merging and replacing travis-tests.cfg and explicit specification in tox.ini)
  • Mirror test requirements using extras_require in setup.py: 1b4d154, 9cb5ae6
    (might be necessary to facilitate the ability to install test requirements without running tests
    and specifically, for Travis CI and tox)

And include additional changes I'm planning:

@Harmon758
Copy link
Member

Harmon758 commented Oct 16, 2019

I've now completed almost all of the changes I had planned.

The only thing remaining to do is to drop usage of nose. Even the documentation for the library itself recommends against continuing to use it. Thus, I've already removed the only explicit usage of the library with 4312997. I don't see the point in migrating to nose2 and switching to another unnecessary library, so all that remains is to replace it with unittest. This should be relatively simple, only requiring setup.py to be updated to remove nose as a test requirement and suite and tox.ini and .travis.yml to be updated to run a unittest command with the specific test modules currently in setup.cfg.

However, I don't see any reason to drop it entirely right now, as it's still working and requires little to no overhead, while providing a nice interface for tox and Travis CI and clean visual output for tests and coverage. Hence, I'm going to leave it as is for now until such a point where it's necessary to make changes.

@Harmon758
Copy link
Member

Harmon758 commented Sep 28, 2021

Following up, I've removed nose usage now, with b4c06a4, as it will stop working with Python 3.10 (nose-devs/nose#1099).

I anticipate using pytest in the future, to facilitate testing with asyncio / aiohttp, etc.

@koobs
Copy link
Author

koobs commented Oct 25, 2021

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Improvement This is regarding an improvement to an existing feature Task This is a task related to the library Tests This is regarding the library's tests
Projects
None yet
Development

No branches or pull requests

2 participants