Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up CI with Azure Pipelines #4458

Merged
merged 19 commits into from Jul 16, 2020
Merged

Set up CI with Azure Pipelines #4458

merged 19 commits into from Jul 16, 2020

Conversation

dangra
Copy link
Member

@dangra dangra commented Mar 27, 2020

No description provided.

@kmike
Copy link
Member

kmike commented Mar 27, 2020

I'm not sure I know what I'm doing, but opened simplistix/sybil#20 :)

@codecov
Copy link

codecov bot commented Mar 30, 2020

Codecov Report

Merging #4458 into master will decrease coverage by 0.08%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #4458      +/-   ##
==========================================
- Coverage   84.82%   84.73%   -0.09%     
==========================================
  Files         163      163              
  Lines        9976     9979       +3     
  Branches     1485     1484       -1     
==========================================
- Hits         8462     8456       -6     
- Misses       1251     1258       +7     
- Partials      263      265       +2     
Impacted Files Coverage Δ
scrapy/utils/url.py 100.00% <100.00%> (ø)
scrapy/utils/py36.py 20.00% <0.00%> (-80.00%) ⬇️
scrapy/utils/spider.py 66.66% <0.00%> (-11.12%) ⬇️
scrapy/utils/trackref.py 82.85% <0.00%> (-2.86%) ⬇️

@Gallaecio
Copy link
Member

Gallaecio commented Mar 30, 2020

The errors look different, I guess that’s an improvement.

@Gallaecio
Copy link
Member

Gallaecio commented Apr 16, 2020

2f510fd fixed 1 test in Python 3.7
94ee686 fixed 3 tests in Python 3.7

@Gallaecio
Copy link
Member

Gallaecio commented Apr 16, 2020

@elacuesta test_ipv6_alternative_name_resolver fails on Windows; it looks like it does not log any error there, and it just successfully crawls the page. Any idea?

PS: That is now the only test failing in Python 3.7.

@elacuesta
Copy link
Member

elacuesta commented Apr 20, 2020

@elacuesta test_ipv6_alternative_name_resolver fails on Windows; it looks like it does not log any error there, and it just successfully crawls the page. Any idea?

PS: That is now the only test failing in Python 3.7.

The failing assertions are about looking for ConnectionRefusedError/ConnectError with the alternative resolver. I ran a debug pipeline (I don't have access to a Windows machine) and it seems to me like it fails because of the assumption that there is no web server listening on the local host. Apparently there is something in the test environment indeed listening on http://[::1] so the test fails because the connection is not refused.
I think the test should not care about the presence of ConnectionRefusedError exceptions but about the absence of DNSLookupError, I can fix that 👍

@elacuesta
Copy link
Member

elacuesta commented Apr 20, 2020

Job successful for py37: https://scrapinghub.visualstudio.com/scrapy/_build/results?buildId=25&view=logs&j=00b6a206-eb91-585b-3d14-cf1a4d7b1970

@Gallaecio
Copy link
Member

Gallaecio commented Apr 23, 2020

Weird, now test_response_ip_address fails with […] FailTest: 'INFO: Host: not.a.real.domain' not in […].

Edit: Oh, it must be a new test. Looking into it…

@Gallaecio
Copy link
Member

Gallaecio commented Jul 3, 2020

Finally! 😓

@Gallaecio Gallaecio changed the title [WIP] Set up CI with Azure Pipelines Set up CI with Azure Pipelines Jul 3, 2020
tests/CrawlerRunner/ip_address.py Outdated Show resolved Hide resolved
tests/CrawlerRunner/ip_address.py Outdated Show resolved Hide resolved
Gallaecio and others added 2 commits Jul 6, 2020
Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
Copy link
Member

@elacuesta elacuesta left a comment

Nice 💪

scrapy/utils/url.py Outdated Show resolved Hide resolved
wRAR
wRAR approved these changes Jul 15, 2020
@kmike kmike merged commit 0f2f1ac into master Jul 16, 2020
2 checks passed
@elacuesta elacuesta deleted the azure-pipelines branch Jul 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants