Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dont_merge_cookies docs are incomplete #2998

Closed
kmike opened this issue Nov 9, 2017 · 2 comments · Fixed by #4028
Closed

dont_merge_cookies docs are incomplete #2998

kmike opened this issue Nov 9, 2017 · 2 comments · Fixed by #4028

Comments

@kmike
Copy link
Member

kmike commented Nov 9, 2017

dont_merge_cookies docs are incomplete: they say that

When some site returns cookies (in a response) those are stored in the cookies for that domain and will be sent again in future requests. That’s the typical behaviour of any regular web browser. However, if, for some reason, you want to avoid merging with existing cookies you can instruct Scrapy to do so by setting the dont_merge_cookies key to True in the Request.meta.

But this flag not only prevents merging of cookies, but also prevents sending of them:

def process_request(self, request, spider):

Maybe a separate issue, but it'd be nice to have separate flags for sending and merging of cookies.

//cc @whalebot-helmsman

@kmike kmike added the docs label Nov 9, 2017
@raphapassini
Copy link
Contributor

You think that this is a change only related to the docs? Maybe we should add a new option called dont_send_cookies?

@RyQcan
Copy link

RyQcan commented Jun 7, 2019

I mean that if i set dont_merge_cookies=true, scrapy will dont send any cookie to server.
But i just want that scrapy don't merge my cookies with the response's cookies, and continue to send my cookies to server.
In scrapy/downloadermiddlewares/cookies.py

def process_request(self, request, spider):
        if request.meta.get('dont_merge_cookies', False):
            return

        cookiejarkey = request.meta.get("cookiejar")
        jar = self.jars[cookiejarkey]
        cookies = self._get_request_cookies(jar, request)
        for cookie in cookies:
            jar.set_cookie_if_ok(cookie, request)

        # set Cookie header
        request.headers.pop('Cookie', None)
        jar.add_cookie_header(request)
        self._debug_cookie(request, spider)

    def process_response(self, request, response, spider):
        if request.meta.get('dont_merge_cookies', False):
            return response

        # extract cookies from Set-Cookie and drop invalid/expired cookies
        cookiejarkey = request.meta.get("cookiejar")
        jar = self.jars[cookiejarkey]
        jar.extract_cookies(response, request)
        self._debug_set_cookie(response, spider)

        return response

if I set dont_merge_cookies=true,scrapy will do nothing.

in issue2124
@Ferrer-Jeremy think code can be modify to

def process_request(self, request, spider):
        cookiejarkey = request.meta.get("cookiejar")
        jar = self.jars[cookiejarkey]

        if request.meta.get('dont_merge_cookies', False):
            jar = CookieJar()

        cookies = self._get_request_cookies(jar, request)
        for cookie in cookies:
            jar.set_cookie_if_ok(cookie, request)

        # set Cookie header
        request.headers.pop('Cookie', None)
        jar.add_cookie_header(request)
        self._debug_cookie(request, spider)

it can work in right way.
So,,,now i want to know why do not fix this bug...

And my English is so poor,sorry,,,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants