Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix overwriting repeated headers in HTTP2 #5777

Merged
merged 3 commits into from
Jan 11, 2023

Conversation

Malkiz223
Copy link
Contributor

@Malkiz223 Malkiz223 commented Dec 30, 2022

This commit fixes the overwriting bug of repeated header values using HTTP2

raw_response = \
"""
HTTP 1.1 200 OK
Set-Cookie: first=1
Set-Cookie: second=2
"""

# Old logic
>>> response.headers.getlist(b'Set-Cookie')
[b'second=2']

# New logic
>>> response.headers.getlist(b'Set-Cookie')
[b'first=1', b'second=2']

@Malkiz223 Malkiz223 changed the title Fix overwriting repeated headers Fix overwriting repeated headers in HTTP2 Dec 30, 2022
@codecov
Copy link

codecov bot commented Dec 31, 2022

Codecov Report

Merging #5777 (59f0980) into master (3252097) will decrease coverage by 0.13%.
The diff coverage is 100.00%.

❗ Current head 59f0980 differs from pull request most recent head 520be4f. Consider uploading reports for the commit 520be4f to get more accurate results

@@            Coverage Diff             @@
##           master    #5777      +/-   ##
==========================================
- Coverage   88.87%   88.74%   -0.14%     
==========================================
  Files         162      162              
  Lines       10980    10978       -2     
  Branches     1796     1624     -172     
==========================================
- Hits         9759     9742      -17     
- Misses        940      953      +13     
- Partials      281      283       +2     
Impacted Files Coverage Δ
scrapy/core/http2/stream.py 91.32% <100.00%> (-0.58%) ⬇️
scrapy/robotstxt.py 75.30% <0.00%> (-13.59%) ⬇️
scrapy/downloadermiddlewares/cookies.py 95.78% <0.00%> (-2.11%) ⬇️
scrapy/shell.py 67.91% <0.00%> (-0.75%) ⬇️
scrapy/http/response/__init__.py 97.40% <0.00%> (-0.04%) ⬇️
scrapy/http/request/__init__.py 97.75% <0.00%> (-0.03%) ⬇️
scrapy/commands/check.py 71.01% <0.00%> (ø)

@Gallaecio
Copy link
Member

Looks good.

How do you feel about including a test for the fix?

@Malkiz223
Copy link
Contributor Author

Thank you very much for your answer.

I agree with your policy of writing tests, but this turned out to be too complicated for me.
I spent several hours writing the test, but could not make it correct.

Copy link
Member

@Gallaecio Gallaecio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@wRAR wRAR merged commit 4a5ef81 into scrapy:master Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants