Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl doesn't send the last line cookie if there is no trailing newline #4946

Closed
ajfg93 opened this issue Feb 18, 2020 · 7 comments
Closed

curl doesn't send the last line cookie if there is no trailing newline #4946

ajfg93 opened this issue Feb 18, 2020 · 7 comments
Labels

Comments

@ajfg93
Copy link

@ajfg93 ajfg93 commented Feb 18, 2020

I did this

  1. Set a cookie in the browser http://httpbin.org/cookies/set/name/value
  2. I use the EditThisCookie Chrome extension to output the cookie into Netscape HTTP Cookie format.

filename: cookie

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by EditThisCookie
httpbin.org	FALSE	/	FALSE	0	name	value
  1. In Powershell, I run
curl "http://httpbin.org/cookies" -H "Proxy-Connection: keep-alive" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9" -H "Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7" --compressed --insecure -b cookie --verbose -o bb.html

The cookie is not sent according the verbose information (No HTTP cookie header is shown in request)

I expected the following

The cookie I saved should appear in request headers. The cookie should be sent.

curl/libcurl version

curl 7.68.0 (x86_64-pc-win32) libcurl/7.68.0 OpenSSL/1.1.1d (Schannel) zlib/1.2.11 brotli/1.0.7 WinIDN libssh2/1.9.0 nghttp2/1.40.0
Release-Date: 2020-01-08
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile MultiSSL NTLM SPNEGO SSL SSPI TLS-SRP brotli libz

operating system

Windows7

Solution

Adding an extra blank line at the end of the cookie file then it works. The cookie will be sent.

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by EditThisCookie
httpbin.org FALSE   /   FALSE   0   name    value

I don't know if this can be counted as a bug. But it did cause me a few hours to find the problem and the documentation doesn't say anything about this requirement(an blank line at the end) for local cookie files, if I don't miss any details.

@bagder
Copy link
Member

@bagder bagder commented Feb 18, 2020

Do you actually mean an extra newline or do you mean a newline ? Because I can reproduce your problem if I make the cookie line in the file not have a newline at all, but I only need one newline at the end of that cookie line to make it work. (I tested on Linux though, and I won't be surprise if this is different on Windows).

@ajfg93
Copy link
Author

@ajfg93 ajfg93 commented Feb 18, 2020

Do you actually mean an extra newline or do you mean a newline ? Because I can reproduce your problem if I make the cookie line in the file not have a newline at all, but I only need one newline at the end of that cookie line to make it work. (I tested on Linux though, and I won't be surprise if this is different on Windows).

I think I mean a newline.

As I posted in the question body

This doesn't work:

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by EditThisCookie
httpbin.org	FALSE	/	FALSE	0	name	value

This works:

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by EditThisCookie
httpbin.org	FALSE	/	FALSE	0	name	value


the cursor is here.

@bagder
Copy link
Member

@bagder bagder commented Feb 18, 2020

I think I'll go with updating the documentation to say that a newline at the end is required. It is really the most sensible way we know that we have read a complete and not a partial line.

@danielgustafsson
Copy link
Member

@danielgustafsson danielgustafsson commented Feb 18, 2020

I'm unable to find any definitive file format specification for the Netscape cookie file. I am inclined to document a trailing newline as being required though, and possibly issue a warning or error for other non-comment lines not ending in a newline?

@danielgustafsson
Copy link
Member

@danielgustafsson danielgustafsson commented Feb 18, 2020

I think I'll go with updating the documentation to say that a newline at the end is required.

👍

bagder added a commit that referenced this issue Feb 18, 2020
... so that we know we got the whole and not a partial line.

Also, changed the formatting of the fields away from a table again since
the table format requires a github-markdown tool version that we don't
run on the web server atm.

Reported-by: Sunny Bean
Fixes #4946
@ajfg93
Copy link
Author

@ajfg93 ajfg93 commented Feb 18, 2020

I'm unable to find any definitive file format specification for the Netscape cookie file. I am inclined to document a trailing newline as being required though, and possibly issue a warning or error for other non-comment lines not ending in a newline?

See if this helps:

https://unix.stackexchange.com/questions/36531/format-of-cookies-when-using-wget/210282

http://www.cookiecentral.com/faq/

@bagder
Copy link
Member

@bagder bagder commented Feb 18, 2020

I'm unable to find any definitive file format specification for the Netscape cookie file

I'm convinced there never was one, we all just copied what they apparently used back in the day... And whatever we'll find today is just people who have tried to document what has been observed. Pretty much like we document how we use and create the file, as I expect that curl and wget are nowadays the two major consumers and producers of this style of cookie file.

@bagder bagder changed the title curl doesn't send the last line cookie if I don't add an extra blank line at the end of the cookie file curl doesn't send the last line cookie if there is no trailing newline Feb 18, 2020
@bagder bagder closed this in 67d26a3 Feb 19, 2020
@lock lock bot locked as resolved and limited conversation to collaborators May 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants
You can’t perform that action at this time.