Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow running tests against the installed version #5949

Merged
merged 2 commits into from Jun 14, 2023

Conversation

wRAR
Copy link
Member

@wRAR wRAR commented Jun 13, 2023

Fixes #4914 by using get_testenv() only for python3 -m scrapy.cmdline and finding files under tests via different mechanisms.

For some reason running these as-installed tests in the Debian autopkgtest chroot always fails on tests/test_crawl.py::CrawlTestCase::test_start_requests_laziness with "All start requests consumed before any download happened". I've looked at the test and the spider it runs and I don't understand if it just relies on mockserver replying faster than start_requests schedules requests or there is something more stable. When I extracted this spider and ran it via scrapy crawlspider on a separately started mockserver it also often (but not always) fails with the same problem. It's possible that the test is just flaky and depends on performance of various components, but I don't remember if it fails on some CI runs.

@codecov
Copy link

codecov bot commented Jun 13, 2023

Codecov Report

Merging #5949 (508b50d) into master (33b418d) will increase coverage by 0.05%.
The diff coverage is 100.00%.

❗ Current head 508b50d differs from pull request most recent head 493ea43. Consider uploading reports for the commit 493ea43 to get more accurate results

@@            Coverage Diff             @@
##           master    #5949      +/-   ##
==========================================
+ Coverage   88.81%   88.86%   +0.05%     
==========================================
  Files         162      162              
  Lines       11184    11199      +15     
  Branches     1819     1821       +2     
==========================================
+ Hits         9933     9952      +19     
+ Misses        965      962       -3     
+ Partials      286      285       -1     
Impacted Files Coverage Δ
scrapy/exporters.py 100.00% <100.00%> (ø)
scrapy/extensions/feedexport.py 95.80% <100.00%> (+0.16%) ⬆️

... and 2 files with indirect coverage changes

Copy link
Member

@Gallaecio Gallaecio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

@wRAR
Copy link
Member Author

wRAR commented Jun 14, 2023

Actually test_start_requests_laziness failed in build-time tests too, so whatever it is, it's unrelated to these changes and this mode of running tests, so merging this.

@wRAR wRAR merged commit 58300e0 into scrapy:master Jun 14, 2023
23 checks passed
@wRAR wRAR deleted the tests-as-installed branch June 14, 2023 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

As-installed tests support
2 participants