Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Writing cookies: Prefix wildcard domains with "." #295

Merged
merged 2 commits into from

3 participants

@mmorearty

There was some discussion related to this issue in issue #293. Making this change will allow for better interop with other libraries that read cookie files (see below). What follows is a whole lot of justification for a very small change :)

If a cookie matches a wildcard domain, e.g. (anything).example.com, then the line written out to a Netscape-formatted cookies.txt file must begin with a dot, e.g. .example.com not example.com. This is redundant with the second field of the line, a TRUE/FALSE value indicating whether the domain is a wildcard, but that's the way it goes.

This is a little tricky, because there is no formal spec for the Netscape cookie file format. (There is, of course, a formal spec the format of the HTTP Set-Cookie header, but that's not the same thing as the cookie file.)

The unnecessary duplication (leading dot in the first field, plus TRUE/FALSE in the second field) seems to lead to all kinds of incompatibilities between different cookie libraries. For comparison:

  • curl/libcurl: When reading a cookie file, they allow, but ignore, the leading dot on the domain; they always respect the TRUE/FALSE flag in the second field. When writing a cookie file, if the second field is TRUE, they always put a leading "." on the domain.
  • Python: Requires that the presence or absence of the leading dot matches the TRUE/FALSE value in the second field; otherwise, rejects the whole cookie file.
  • Perl: The second field (TRUE/FALSE) is ignored. The presence or absence of a leading dot on the first field is used to determine whether

The upshot of all this is that curl will be able to correctly read cookie files that were generated by the current Mechanize, but Python and Perl scripts will not.

All of the above libraries (curl, Python, and Perl), when writing a cookie file, ensure that the presence or absence of a leading "." on the domain matches the TRUE/FALSE in the second field. Seems to me that Mechanize should do the same.

Mechanize's current behavior when reading a cookie file is fine: It allows a leading dot, but that is overridden by the TRUE/FALSE of the second field. That matches curl's behavior.

@mmorearty mmorearty Writing cookies: Prefix wildcard domains with "."
If a cookie matches a wildcard domain, e.g. "(anything).example.com",
then the line written out to a Netscape-formatted cookies.txt file must
begin with a dot, e.g. ".example.com" not "example.com".  This is
redundant with the second field of the line, a TRUE/FALSE value
indicating whether the domain is a wildcard, but that's the way it goes.
dae2733
@drbrain
Owner

A test should accompany this change to ensure we don't break it in the future

@mmorearty mmorearty Add test for cookies that match subdomains
A test to go with commit dae2733: Tests that when writing a cookies.txt
file:

* Cookies that only match exactly the domain specified must not have a
  leading dot, and must have FALSE as the second field.
* Cookies that match subdomains must have a leading dot, and must have
  TRUE as the second field.
85bda8f
@leejarvis leejarvis merged commit 4afa07c into sparklemotion:master

1 check passed

Details default The Travis build passed
@leejarvis leejarvis referenced this pull request from a commit
@leejarvis leejarvis updated changes for #295 d509938
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Feb 19, 2013
  1. @mmorearty

    Writing cookies: Prefix wildcard domains with "."

    mmorearty authored Mike Morearty committed
    If a cookie matches a wildcard domain, e.g. "(anything).example.com",
    then the line written out to a Netscape-formatted cookies.txt file must
    begin with a dot, e.g. ".example.com" not "example.com".  This is
    redundant with the second field of the line, a TRUE/FALSE value
    indicating whether the domain is a wildcard, but that's the way it goes.
Commits on Feb 20, 2013
  1. @mmorearty

    Add test for cookies that match subdomains

    mmorearty authored Mike Morearty committed
    A test to go with commit dae2733: Tests that when writing a cookies.txt
    file:
    
    * Cookies that only match exactly the domain specified must not have a
      leading dot, and must have FALSE as the second field.
    * Cookies that match subdomains must have a leading dot, and must have
      TRUE as the second field.
This page is out of date. Refresh to see the latest.
Showing with 39 additions and 1 deletion.
  1. +1 −1  lib/mechanize/cookie_jar.rb
  2. +38 −0 test/test_mechanize_cookie_jar.rb
View
2  lib/mechanize/cookie_jar.rb
@@ -192,7 +192,7 @@ def dump_cookiestxt(io)
"#{Mechanize::VERSION} https://github.com/sparklemotion/mechanize.\n\n"
to_a.each do |cookie|
io.puts([
- cookie.domain,
+ (cookie.for_domain? ? "." : "") + cookie.domain,
cookie.for_domain? ? "TRUE" : "FALSE",
cookie.path,
cookie.secure ? "TRUE" : "FALSE",
View
38 test/test_mechanize_cookie_jar.rb
@@ -508,4 +508,42 @@ def test_secure_cookie
assert_equal('Foo1', @jar.cookies(nurl).map { |c| c.name }.sort.join(' ') )
assert_equal('Foo1 Foo2', @jar.cookies(surl).map { |c| c.name }.sort.join(' ') )
end
+
+ def test_save_cookies_cookiestxt_subdomain
+ top_url = URI 'http://rubyforge.org/'
+ subdomain_url = URI 'http://admin.rubyforge.org/'
+
+ # cookie1 is for *.rubyforge.org; cookie2 is only for rubyforge.org, no subdomains
+ cookie1 = Mechanize::Cookie.new(cookie_values)
+ cookie2 = Mechanize::Cookie.new(cookie_values(:name => 'Boo', :for_domain => false))
+
+ @jar.add(top_url, cookie1)
+ @jar.add(top_url, cookie2)
+
+ assert_equal(2, @jar.cookies(top_url).length)
+ assert_equal(1, @jar.cookies(subdomain_url).length)
+
+ in_tmpdir do
+ @jar.save_as("cookies.txt", :cookiestxt)
+
+ jar = Mechanize::CookieJar.new
+ jar.load("cookies.txt", :cookiestxt) # HACK test the format
+ assert_equal(2, jar.cookies(top_url).length)
+ assert_equal(1, jar.cookies(subdomain_url).length)
+
+ # Check that we actually wrote the file correctly (not just that we were
+ # able to read what we wrote):
+ #
+ # * Cookies that only match exactly the domain specified must not have a
+ # leading dot, and must have FALSE as the second field.
+ # * Cookies that match subdomains must have a leading dot, and must have
+ # TRUE as the second field.
+ cookies_txt = File.readlines("cookies.txt")
+ assert_equal(1, cookies_txt.grep( /^rubyforge\.org\tFALSE/ ).length)
+ assert_equal(1, cookies_txt.grep( /^\.rubyforge\.org\tTRUE/ ).length)
+ end
+
+ assert_equal(2, @jar.cookies(top_url).length)
+ assert_equal(1, @jar.cookies(subdomain_url).length)
+ end
end
Something went wrong with that request. Please try again.