MozillaCookieJar ignores HttpOnly cookies

[douyuan]

BPO	2190
Nosy	@loewis, @terryjreedy, @orsenthil, @dlenski, @jdetrey, @jacobneiltaylor
PRs	bpo-2190: make MozillaCookieJar handle HTTP-only cookies #22798 bpo-38976: Add support for HTTP Only flag in MozillaCookieJar #17471
Files	_MozillaCookieJar.diff: a quick & dirty fix httponly.patch: patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-11-18.05:27:37.695>
created_at = <Date 2008-02-25.16:39:05.656>
labels = ['type-feature', 'library', '3.10']
title = 'MozillaCookieJar ignores HttpOnly cookies'
updated_at = <Date 2020-11-18.05:27:37.695>
user = 'https://bugs.python.org/douyuan'

bugs.python.org fields:

activity = <Date 2020-11-18.05:27:37.695>
actor = 'orsenthil'
assignee = 'none'
closed = True
closed_date = <Date 2020-11-18.05:27:37.695>
closer = 'orsenthil'
components = ['Library (Lib)']
creation = <Date 2008-02-25.16:39:05.656>
creator = 'douyuan'
dependencies = []
files = ['9552', '37625']
hgrepos = []
issue_num = 2190
keywords = ['patch']
message_count = 17.0
messages = ['62985', '74822', '109819', '109958', '110058', '110121', '233571', '300920', '367468', '367469', '379003', '379312', '379398', '381034', '381039', '381312', '381313']
nosy_count = 10.0
nosy_names = ['loewis', 'terry.reedy', 'jjlee', 'orsenthil', 'douyuan', 'python-dev', 'dlenski', 'jdetrey', 'mt0321', 'Jacob Taylor']
pr_nums = ['22798', '17471']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue2190'
versions = ['Python 3.10']

[douyuan]

HttpOnly cookie in Firefox's cookies.txt begins with "#HttpOnly_" now,
just like a comment, e.g.:

#HttpOnly_.rad.live.com TRUE / FALSE 1258200001 FC09 FB=
#HttpOnly_service.ilib.cn FALSE / FALSE 1209905939
.ASPXANONYMOUS
JMeD5-atyAEkAAAAYjZlNDUyNDAtOGQ4ZC00NTEyLTljN2EtMzNkODM3M2JjMjFivtX6ikB7Iv0jRJBJs9ftggv_a2k

Since no obvious need, there are no patches for save method and
cookielib.Cookie class.

[jjlee]

I think firefox 3 no longer writes cookies.txt (it writes cookies.sqlite
instead).

Can anybody point out a version of firefox that wrote this HttpOnly
information to cookies.txt, so the patch can be tested?

[terryjreedy]

MozillaCookieJar is now a class in http.cookiejar, so patch would need update. Is this still used enough to bother?

[douyuan]

Firefox no longer use cookies.txt. I think this patch is useless.

[terryjreedy]

Would you suggest removing MozillaCookieJar from the module?
(Through the normal warn-deprecate-remove process.)

[jjlee]

Is deprecation really necessary? lynx still uses that format. lynx doesn't write the header that MozillaCookieJar insists on being present, but a trivial subclass can read cookies files written by lynx.

[jdetrey]

Dear all,

In fact, this cookie.txt format is still used by curl. For instance, see

https://github.com/bagder/curl/blob/curl-7_39_0/lib/cookie.c#L644

which clearly shows support for the "#HttpOnly_" prefix. Therefore, supporting this format in http.cookiejar.MozillaCookieJar seems quite relevant to me.

Attached is an updated patch.

Kind regards,
Jérémie.

[mt0321]

Can this issue be reopened? As Jérémie stated, curl uses this format and outputs cookie files using the #HttpOnly_ prefix. I also found at least one project that is working around lack of this support:
https://code.google.com/archive/p/git-repo/
https://gerrit.googlesource.com/git-repo/+/master/subcmds/sync.py#995
# Python doesn't understand cookies with the #HttpOnly_ prefix
# Since we're only using them for HTTP, copy the file temporarily,
# stripping those prefixes away.

One potential improvement for the proposed patch: instead of just stripping out #HttpOnly_, this attribute should be set on the Cookie that is created, within the 'rest' dict (rest={'HttpOnly': True}). The Morsel class is already aware of this attribute, as is the 'requests.cookies' module.

[dlenski]

Also confused about why this was closed.

This format is still frequently used. In the absence of a solution in the standard library, I'm using this kludge to strip the leading #HttpOnly_.

    from tempfile import NamedTemporaryFile
    from http.cookiejar import MozillaCookieJar
    from contextlib import contextmanager

    def fix_cookie_jar_file(orig_cookiejarfile):
        with NamedTemporaryFile(mode='w+') as cjf:
            with open(orig_cookiejarfile, 'r') as ocf:
            for l in ocf:            
                cjf.write(l[10:] if l.startswith('#HttpOnly_') else l)
            cjf.seek(0)
        yield cjf.name

    MozillaCookieJar(filename=fix_cookie_jar_file(orig_cookiejarfile))

[terryjreedy]

This issue was closed as useless for Firefox in 2010 by the original poster, msg109958. My participation here is only as tracker triager, as I only have a consumer knowledge of cookies. Unfortunately, there is no core developer expert for http, let alone the http.cookiejar. The person who once handled some cookie related patches is no longer active.

Adding a patch to a closed issue is somewhat useless. In any case, a possible revised PR would be needed. My suggestion is to ask on python-ideas whether this enhancement might be accepted now and whether better to reopen this issue or open a new one.

[dlenski]

I've got a patch that will address both loading and saving of "HTTP-only" cookies: master...dlenski:patch-1

Testing/feedback before I submit as a PR would be very welcome.