Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not process cookies from headers #4823

Merged
merged 2 commits into from Oct 8, 2020

Conversation

elacuesta
Copy link
Member

@elacuesta elacuesta commented Oct 1, 2020

Fixes #4717

Revert the behaviour introduced in #2400, because of the bug reported in #4717. Once we merge this, we'll need to either reopen #1992 or create a new issue to keep track of the work towards supporting cookies in the Cookie request header (#4812, for instance).

Tasks:

  • Update cookies middleware implementation
  • Mark existing tests as xfail
  • Docs: add warnings about the need to define cookies in the Request.cookies attribute (Request.headers attribute and DEFAULT_REQUEST_HEADERS setting)

@codecov
Copy link

codecov bot commented Oct 1, 2020

Codecov Report

Merging #4823 into master will increase coverage by 0.30%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #4823      +/-   ##
==========================================
+ Coverage   87.52%   87.82%   +0.30%     
==========================================
  Files         160      160              
  Lines        9761     9718      -43     
  Branches     1440     1432       -8     
==========================================
- Hits         8543     8535       -8     
+ Misses        955      924      -31     
+ Partials      263      259       -4     
Impacted Files Coverage Δ
scrapy/downloadermiddlewares/cookies.py 94.80% <100.00%> (-0.94%) ⬇️
scrapy/spiders/crawl.py 93.97% <0.00%> (-0.65%) ⬇️
scrapy/link.py 100.00% <0.00%> (ø)
scrapy/extensions/feedexport.py 95.22% <0.00%> (ø)
scrapy/downloadermiddlewares/httpproxy.py 100.00% <0.00%> (ø)
scrapy/utils/conf.py 93.75% <0.00%> (+0.05%) ⬆️
scrapy/core/downloader/__init__.py 92.48% <0.00%> (+1.50%) ⬆️
scrapy/utils/test.py 60.93% <0.00%> (+4.35%) ⬆️
scrapy/pipelines/files.py 71.67% <0.00%> (+6.99%) ⬆️
scrapy/core/downloader/tls.py 97.14% <0.00%> (+11.09%) ⬆️

@elacuesta elacuesta marked this pull request as ready for review Oct 6, 2020
@Gallaecio
Copy link
Member

Gallaecio commented Oct 6, 2020

Since I’m a coward not willing to just merge: what about adding xfail_strict: true to pytest.ini? (https://docs.pytest.org/en/latest/skipping.html#strict-parameter). I though that’s what xfail would do, but it turns out by default xfail means that the tests always pass regardless of their outcome.

If it cannot be applied globally, you can always apply it on those 3 tests, and we can open a separate issue to look into why it cannot be applied globally at the moment.

wRAR
wRAR approved these changes Oct 8, 2020
@wRAR wRAR merged commit 9f8c393 into scrapy:master Oct 8, 2020
1 check passed
@elacuesta elacuesta deleted the cookies-revert-header branch Oct 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Request cookies not updated with values from previous response
3 participants