Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ints in Cookies are not getting parsed correctly #5252

Closed
doctorrmcb opened this issue Oct 1, 2021 · 0 comments · Fixed by #5253
Closed

ints in Cookies are not getting parsed correctly #5252

doctorrmcb opened this issue Oct 1, 2021 · 0 comments · Fixed by #5253

Comments

@doctorrmcb
Copy link

doctorrmcb commented Oct 1, 2021

Description

Using an int in a cookie causes the spider to not work. Replacing the int with a str fixes the problem.

Steps to Reproduce

Failing Code Example:

request = scrapy.Request(
      url = url, 
      callback = callback
)
request.cookies['some_cookie'] = 1

Successful Code Example:

request = scrapy.Request(
      url = url, 
     callback = callback
)
request.cookies['some_cookie'] = '1'

Expected behavior: I expect that an int should be parsed to a str without me needing to do it myself.

Actual behavior: Get following error:

Traceback (most recent call last):
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\twisted\internet\defer.py", line 1661, in _inlineCallbacks
    result = current_context.run(gen.send, result)
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\core\downloader\middleware.py", line 36, in process_request
    response = yield deferred_from_coro(method(request=request, spider=spider))
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\downloadermiddlewares\cookies.py", line 32, in process_request
    for cookie in self._get_request_cookies(jar, request):
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\downloadermiddlewares\cookies.py", line 109, in _get_request_cookies
    response = Response(request.url, headers={"Set-Cookie": formatted})
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\http\response\__init__.py", line 32, in __init__
    self.headers = Headers(headers or {})
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\http\headers.py", line 11, in __init__
    super().__init__(seq)
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\utils\datatypes.py", line 20, in __init__
    self.update(seq)
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\utils\datatypes.py", line 56, in update
    super().update(iseq)
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\utils\datatypes.py", line 55, in <genexpr>
    iseq = ((self.normkey(k), self.normvalue(v)) for k, v in seq)
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\http\headers.py", line 26, in normvalue
    return [self._tobytes(x) for x in value]
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\http\headers.py", line 26, in <listcomp>
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\downloadermiddlewares\cookies.py", line 108, in <genexpr>
    formatted = filter(None, (self._format_cookie(c, request) for c in cookies))
  File "C:\Users\Michael\AppData\Local\Programs\Python\Python39\lib\site-packages\scrapy\downloadermiddlewares\cookies.py", line 87, in _format_cookie
    decoded[key] = cookie[key].decode("utf8")
AttributeError: 'int' object has no attribute 'decode'

Reproduces how often: [What percentage of the time does it reproduce?]

Versions

Scrapy : 2.5.0
lxml : 4.6.3.0
libxml2 : 2.9.5
cssselect : 1.1.0
parsel : 1.6.0
w3lib : 1.22.0
Twisted : 21.7.0
Python : 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)]
pyOpenSSL : 21.0.0 (OpenSSL 1.1.1l 24 Aug 2021)
cryptography : 35.0.0
Platform : Windows-10-10.0.19043-SP0

Additional context

from scrapy.http import Request

Failing Code Example:

request = scrapy.Request(
      url = url, 
      callback = callback
)
request.cookies['some_cookie'] = 1

Successful Code Example:

request = scrapy.Request(
      url = url, 
     callback = callback
)
request.cookies['some_cookie'] = '1'

(edited to add code formatting to the traceback)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants