Fix netrc parsing on Windows #258

Closed
wants to merge 1 commit into
from

Conversation

Projects
None yet
5 participants
@orgads
Contributor

orgads commented May 5, 2015

Open it in text mode to avoid considering CR as part of the line

Fix netrc parsing on Windows
Open it in text mode to avoid considering CR as part of the line

@orgads orgads referenced this pull request in git-for-windows/git May 5, 2015

Closed

_netrc doesn't work on https push #132

@dscho

This comment has been minimized.

Show comment
Hide comment
@dscho

dscho May 5, 2015

Contributor

Excellent! (For context: I am the Git for Windows maintainer and would love to see this change integrated soon)

Contributor

dscho commented May 5, 2015

Excellent! (For context: I am the Git for Windows maintainer and would love to see this change integrated soon)

@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay May 5, 2015

Member

Mode t isn't widely recognized. It's not necessary unless global _fmode was used to set the default mode to binary. Something has changed in the way you build curl that causes this problem, I think.

Member

jay commented May 5, 2015

Mode t isn't widely recognized. It's not necessary unless global _fmode was used to set the default mode to binary. Something has changed in the way you build curl that causes this problem, I think.

@orgads

This comment has been minimized.

Show comment
Hide comment
@orgads

orgads May 6, 2015

Contributor

According to cygwin docs ("The default Cygwin behavior", section b), if the file path contains backslashes the default is binary. In netrc.c DIR_CHAR, which is a backslash on WIN32, is used as a separator, and curl_getenv might also return a path with backslashes.

Contributor

orgads commented May 6, 2015

According to cygwin docs ("The default Cygwin behavior", section b), if the file path contains backslashes the default is binary. In netrc.c DIR_CHAR, which is a backslash on WIN32, is used as a separator, and curl_getenv might also return a path with backslashes.

@jay jay self-assigned this May 6, 2015

@dscho

This comment has been minimized.

Show comment
Hide comment
@dscho

dscho May 6, 2015

Contributor

Mode t isn't widely recognized.

That might be true (although all the Windows, Linux and MacOSX compilers I know -- GCC, clang, Visual C++, icc, Borland C -- support it, or at least handle it gracefully). However, it would benefit all Windows-based projects using cURL, as cURL would not have to be patched to support CR/LF line endings in $HOME/_netrc.

Contributor

dscho commented May 6, 2015

Mode t isn't widely recognized.

That might be true (although all the Windows, Linux and MacOSX compilers I know -- GCC, clang, Visual C++, icc, Borland C -- support it, or at least handle it gracefully). However, it would benefit all Windows-based projects using cURL, as cURL would not have to be patched to support CR/LF line endings in $HOME/_netrc.

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder May 6, 2015

Member

Unfortunately I believe "t" is not defined by POSIX: http://pubs.opengroup.org/onlinepubs/009695399/functions/fopen.html

... instead it says: "If the string is one of the following, the file shall be opened in the indicated mode. Otherwise, the behavior is undefined."

To play really safe, I would say that a little #ifdef dance would be appropriate there.

Member

bagder commented May 6, 2015

Unfortunately I believe "t" is not defined by POSIX: http://pubs.opengroup.org/onlinepubs/009695399/functions/fopen.html

... instead it says: "If the string is one of the following, the file shall be opened in the indicated mode. Otherwise, the behavior is undefined."

To play really safe, I would say that a little #ifdef dance would be appropriate there.

jay added a commit that referenced this pull request May 6, 2015

netrc: Read in text mode when cygwin
Use text mode when cygwin to eliminate trailing carriage returns.

Bug: #258
@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay May 6, 2015

Member

Thanks, I was able to reproduce in cygwin. A modified version of your patch landed in 33058a1.

Member

jay commented May 6, 2015

Thanks, I was able to reproduce in cygwin. A modified version of your patch landed in 33058a1.

@jay jay closed this May 6, 2015

@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay May 7, 2015

Member

@orgads Can you confirm this works for you? I read in the git-for-windows issue that someone else tested and it works for them but I would like to know about your specific case. It works for my Cygwin POSIX-only build but it looks like you're using a build that's a POSIX/WinAPI mix.

@bagder I'm thinking of making the same change for curlrc to handle the case where the build is Cygwin POSIX-only but the user's curlrc is CRLF, to ensure it's translated properly. Any objection?

Member

jay commented May 7, 2015

@orgads Can you confirm this works for you? I read in the git-for-windows issue that someone else tested and it works for them but I would like to know about your specific case. It works for my Cygwin POSIX-only build but it looks like you're using a build that's a POSIX/WinAPI mix.

@bagder I'm thinking of making the same change for curlrc to handle the case where the build is Cygwin POSIX-only but the user's curlrc is CRLF, to ensure it's translated properly. Any objection?

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder May 8, 2015

Member

@jay: sounds perfectly logical and fine to me

Member

bagder commented May 8, 2015

@jay: sounds perfectly logical and fine to me

@orgads

This comment has been minimized.

Show comment
Hide comment
@orgads

orgads May 8, 2015

Contributor

@jay: __CYGWIN__ doesn't seem to be defined in mingw. Change it to WIN32 maybe?

Contributor

orgads commented May 8, 2015

@jay: __CYGWIN__ doesn't seem to be defined in mingw. Change it to WIN32 maybe?

@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay May 8, 2015

Member

Orgad I thought you were using Cygwin. How are you building? The only way I can reproduce is in Cygwin. The curl and libcurl packages available from the cygwin package manager are what I tested with. You seem to be using a build that's using both POSIX and the Windows API I think, because your netrc is "_netrc" and that underscore style is only for builds in Windows that aren't POSIX-only, which means libcurl uses the Windows API.

Using WIN32 by itself won't solve it since it isn't defined if POSIX-only; though I could use defined(__CYGWIN__) || defined(WIN32) which won't break anything. I'm still at a loss for what POSIX-only system is being used when WIN32 is defined. Can you tell me if the fix as it is now works for what you described in your original report, because if so that means __CYGWIN__ is defined when you build.

Member

jay commented May 8, 2015

Orgad I thought you were using Cygwin. How are you building? The only way I can reproduce is in Cygwin. The curl and libcurl packages available from the cygwin package manager are what I tested with. You seem to be using a build that's using both POSIX and the Windows API I think, because your netrc is "_netrc" and that underscore style is only for builds in Windows that aren't POSIX-only, which means libcurl uses the Windows API.

Using WIN32 by itself won't solve it since it isn't defined if POSIX-only; though I could use defined(__CYGWIN__) || defined(WIN32) which won't break anything. I'm still at a loss for what POSIX-only system is being used when WIN32 is defined. Can you tell me if the fix as it is now works for what you described in your original report, because if so that means __CYGWIN__ is defined when you build.

@orgads

This comment has been minimized.

Show comment
Hide comment
@orgads

orgads May 10, 2015

Contributor

I'm building on msys2 environment. See PR there: Alexpux/MINGW-packages#613

Contributor

orgads commented May 10, 2015

I'm building on msys2 environment. See PR there: Alexpux/MINGW-packages#613

@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay May 29, 2015

Member

I still am not able to reproduce this in msys2 with _netrc though I believe it's likely. So far:

I've installed msys2 using the 64-bit installer. There are 3 environments available in msys2: msys2 (64-bit), mingw32 (mingw-w64 32-bit) and mingw64 (mingw-w64 64-bit). mingw32 has /mingw32/bin first in the path and mingw64 has /mingw64/bin first in the path. Packages can be installed for msys2 or mingw32 or mingw64. If in one of the mingw shells and a mingw-specific version of a program is not installed then the msys version which is next in the path will be used. For example if I have the msys2 curl package installed but not the mingw64 curl package then when I run curl in mingw64 it will run the msys2 version.

Then I followed the advice in their wiki to build the packages on my own.

I've built and installed the mingw-w64 version of curl but removed the patch that reportedly fixes the issue. Without the patch everything is fine in the curl tool.

So next I looked at git. I installed the msys2 git package but that uses .netrc. Indeed the issue is present there but that's to be expected. msys2 targets are built with __CYGWIN__ so my patch covers that. But I've hit a roadblock because I can't get the mingw git package working. I get this:

$ git clone https://github.com/bagder/curl.git
Cloning into 'curl'...
ssh: Could not resolve hostname https: Name or service not known
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

So I tried building it from source and that had the same problem. I tried building the git nightly from source but that won't build. They don't have time to support mingw git package from what I was told.

So the mystery remains how @orgads git loads _netrc. Which git are you using?

I think though I have a good idea now what caused this issue. I looked at msysgit's source (what mingw git package is made from) and I can see in the version they use msysgit 1.9.4.1 the default file mode is set to binary:

_CRT_fmode = _O_BINARY

And basically the same thing eventually made its way upstream since then. In the latest git 2.4.2 the default for all fmode is binary as well.

So that seems to me likely to be the issue, though I still prefer to reproduce to be certain. And it's an interesting issue, I think. What obligation do we have as a shared library if the application overrides the default open mode and changes it to binary? Is the onus on the application or on libcurl? If it's libcurl the remedy for this seems to be change each fopen call where we need to read/write text to explicitly specify t for WIN32.

Member

jay commented May 29, 2015

I still am not able to reproduce this in msys2 with _netrc though I believe it's likely. So far:

I've installed msys2 using the 64-bit installer. There are 3 environments available in msys2: msys2 (64-bit), mingw32 (mingw-w64 32-bit) and mingw64 (mingw-w64 64-bit). mingw32 has /mingw32/bin first in the path and mingw64 has /mingw64/bin first in the path. Packages can be installed for msys2 or mingw32 or mingw64. If in one of the mingw shells and a mingw-specific version of a program is not installed then the msys version which is next in the path will be used. For example if I have the msys2 curl package installed but not the mingw64 curl package then when I run curl in mingw64 it will run the msys2 version.

Then I followed the advice in their wiki to build the packages on my own.

I've built and installed the mingw-w64 version of curl but removed the patch that reportedly fixes the issue. Without the patch everything is fine in the curl tool.

So next I looked at git. I installed the msys2 git package but that uses .netrc. Indeed the issue is present there but that's to be expected. msys2 targets are built with __CYGWIN__ so my patch covers that. But I've hit a roadblock because I can't get the mingw git package working. I get this:

$ git clone https://github.com/bagder/curl.git
Cloning into 'curl'...
ssh: Could not resolve hostname https: Name or service not known
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

So I tried building it from source and that had the same problem. I tried building the git nightly from source but that won't build. They don't have time to support mingw git package from what I was told.

So the mystery remains how @orgads git loads _netrc. Which git are you using?

I think though I have a good idea now what caused this issue. I looked at msysgit's source (what mingw git package is made from) and I can see in the version they use msysgit 1.9.4.1 the default file mode is set to binary:

_CRT_fmode = _O_BINARY

And basically the same thing eventually made its way upstream since then. In the latest git 2.4.2 the default for all fmode is binary as well.

So that seems to me likely to be the issue, though I still prefer to reproduce to be certain. And it's an interesting issue, I think. What obligation do we have as a shared library if the application overrides the default open mode and changes it to binary? Is the onus on the application or on libcurl? If it's libcurl the remedy for this seems to be change each fopen call where we need to read/write text to explicitly specify t for WIN32.

@jay jay reopened this May 29, 2015

@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay May 30, 2015

Member

I was able to reproduce what I think is happening by modifying simple.c and adding #include <fcntl.h> and in main adding _fmode = _O_BINARY; right after variable declaration, and load netrc curl_easy_setopt(curl, CURLOPT_NETRC, CURL_NETRC_REQUIRED); after curl_easy_init. I linked against the mingw-w64 64-bit version of libcurl:
x86_64-w64-mingw32-gcc -o test_netrc test_netrc.c -lcurl
The password that was sent to the server contained the carriage return.

mingw-w64 does not support _CRT_fmode. Adding it will not change the default file open mode. To confirm I checked their source and can't find it. It looks like that has been known for a while. This is relevant because it implies only msysgit 1.9.4 built using original mingw --and not the later mingw-w64 used by msys2-- would have this issue. So I think it's likely Orgad was using a git built with original mingw and not a git package provided by msys2 which would have used mingw-w64. Or maybe some source other than msysgit. Edit: msysgit 1.9.4 also sets _fmode to binary so the crossed out sentence above is incorrect. And Orgad replied below he's using git-for-windows which is forked from msysgit.

I'm satisfied that the cause is an application setting the default file mode to binary. I'm ready to move on to a solution but I could really use some input.

Member

jay commented May 30, 2015

I was able to reproduce what I think is happening by modifying simple.c and adding #include <fcntl.h> and in main adding _fmode = _O_BINARY; right after variable declaration, and load netrc curl_easy_setopt(curl, CURLOPT_NETRC, CURL_NETRC_REQUIRED); after curl_easy_init. I linked against the mingw-w64 64-bit version of libcurl:
x86_64-w64-mingw32-gcc -o test_netrc test_netrc.c -lcurl
The password that was sent to the server contained the carriage return.

mingw-w64 does not support _CRT_fmode. Adding it will not change the default file open mode. To confirm I checked their source and can't find it. It looks like that has been known for a while. This is relevant because it implies only msysgit 1.9.4 built using original mingw --and not the later mingw-w64 used by msys2-- would have this issue. So I think it's likely Orgad was using a git built with original mingw and not a git package provided by msys2 which would have used mingw-w64. Or maybe some source other than msysgit. Edit: msysgit 1.9.4 also sets _fmode to binary so the crossed out sentence above is incorrect. And Orgad replied below he's using git-for-windows which is forked from msysgit.

I'm satisfied that the cause is an application setting the default file mode to binary. I'm ready to move on to a solution but I could really use some input.

@orgads

This comment has been minimized.

Show comment
Hide comment
@orgads

orgads May 31, 2015

Contributor

I'm using git-for-windows.

It does set _fmode to _O_BINARY as you noticed.

What input do you miss?

Contributor

orgads commented May 31, 2015

I'm using git-for-windows.

It does set _fmode to _O_BINARY as you noticed.

What input do you miss?

@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay May 31, 2015

Member

What input do you miss?

Thanks for the follow up. I need input from other people involved in the project. We could handle this by explicitly specifying t every fopen call we want to read lines of text. The problem with doing that is we can't be sure t won't cause an fopen that doesn't recognize it to fail. Most will ignore it as far as I know, but I don't know if all will. Do any fail? We could test fopen with t in configure I guess.

Opening with t regardless already appears once in the source code for the curl tool but I don't know if it should. To work around the issue there is what I committed earlier which is basically an ifdef platform fopen(file, "rt") else fopen file,"r") #endif.

I don't want to pepper the libcurl source with ifdefs every time an fopen is needed to read text. I don't know if everyone is going to want to adhere to putting "t" every time we need to read text, either. What I could instead do is replace fopen with a function or a function-like macro.

#if defined(WIN32) || defined(__CYGWIN__) || defined(SOMETHING_ELSE_THAT_NEEDS_T)
#undef fopen
#define fopen rpl_win_fopen
#endif
and then
rpl_win_fopen {
if(!strstr(mode,"b")) {
// if we're not explicitly binary append a 't' for windows users in case the application switched the default mode.
}
}

Others may have better ideas.

Member

jay commented May 31, 2015

What input do you miss?

Thanks for the follow up. I need input from other people involved in the project. We could handle this by explicitly specifying t every fopen call we want to read lines of text. The problem with doing that is we can't be sure t won't cause an fopen that doesn't recognize it to fail. Most will ignore it as far as I know, but I don't know if all will. Do any fail? We could test fopen with t in configure I guess.

Opening with t regardless already appears once in the source code for the curl tool but I don't know if it should. To work around the issue there is what I committed earlier which is basically an ifdef platform fopen(file, "rt") else fopen file,"r") #endif.

I don't want to pepper the libcurl source with ifdefs every time an fopen is needed to read text. I don't know if everyone is going to want to adhere to putting "t" every time we need to read text, either. What I could instead do is replace fopen with a function or a function-like macro.

#if defined(WIN32) || defined(__CYGWIN__) || defined(SOMETHING_ELSE_THAT_NEEDS_T)
#undef fopen
#define fopen rpl_win_fopen
#endif
and then
rpl_win_fopen {
if(!strstr(mode,"b")) {
// if we're not explicitly binary append a 't' for windows users in case the application switched the default mode.
}
}

Others may have better ideas.

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder May 31, 2015

Member

@jay, here's my suggestion on how we could proceed with the code:

#ifdef [the t-needing platforms]
#define FOPEN_READTEXT "rt"
#define FOPEN_WRITETEXT "wt"
#else
#define FOPEN_READTEXT "r"
#define FOPEN_WRITETEXT "w"
#endif

... and then we have all fopens in the code base that read text files done like this (seems to be 9 places in src/ and lib/ right now):

fopen(filename, FOPEN_READTEXT);

... and the version that writes text like this (I believe 4 places):

fopen(filename, FOPEN_WRITETEXT);

Member

bagder commented May 31, 2015

@jay, here's my suggestion on how we could proceed with the code:

#ifdef [the t-needing platforms]
#define FOPEN_READTEXT "rt"
#define FOPEN_WRITETEXT "wt"
#else
#define FOPEN_READTEXT "r"
#define FOPEN_WRITETEXT "w"
#endif

... and then we have all fopens in the code base that read text files done like this (seems to be 9 places in src/ and lib/ right now):

fopen(filename, FOPEN_READTEXT);

... and the version that writes text like this (I believe 4 places):

fopen(filename, FOPEN_WRITETEXT);

jay added a commit that referenced this pull request Jun 1, 2015

curl_setup: Add macros for FOPEN_READTEXT, FOPEN_WRITETEXT
- Change fopen calls to use FOPEN_READTEXT instead of "r" or "rt"
- Change fopen calls to use FOPEN_WRITETEXT instead of "w" or "wt"

This change is to explicitly specify when we need to read/write text.
Unfortunately 't' is not part of POSIX fopen so we can't specify it
directly. Instead we now have FOPEN_READTEXT, FOPEN_WRITETEXT.

Prior to this change we had an issue on Windows if an application that
uses libcurl overrides the default file mode to binary. The default file
mode in Windows is normally text mode (translation mode) and that's what
libcurl expects.

Bug: #258 (comment)
Reported-by: Orgad Shaneh
@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay Jun 1, 2015

Member

@bagder Ok I made those changes in e8423f9 but I left alone the mode settings in the vms wrappers that use fopen. Although this is only a libcurl issue as far as I know I made the changes in the curl tool as well.

I think having to specify FOPEN_READTEXT , FOPEN_WRITETEXT may be easy to forget. It may need to be enforced or a warning somewhere (CONTRIBUTE?) so developers know it's required now to explicitly read/write text.

Member

jay commented Jun 1, 2015

@bagder Ok I made those changes in e8423f9 but I left alone the mode settings in the vms wrappers that use fopen. Although this is only a libcurl issue as far as I know I made the changes in the curl tool as well.

I think having to specify FOPEN_READTEXT , FOPEN_WRITETEXT may be easy to forget. It may need to be enforced or a warning somewhere (CONTRIBUTE?) so developers know it's required now to explicitly read/write text.

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Jun 1, 2015

Member

Yes it is certainly easy to forget/miss. It would be nice to write a test for it. One (crude) way would be to just scan for fopen() and "r" in the source, but perhaps we can come up with a clever way to abuse the memdebug fopen() wrapper for it?

Member

bagder commented Jun 1, 2015

Yes it is certainly easy to forget/miss. It would be nice to write a test for it. One (crude) way would be to just scan for fopen() and "r" in the source, but perhaps we can come up with a clever way to abuse the memdebug fopen() wrapper for it?

@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay Jun 1, 2015

Member

Do you mean scan like add a regex in checksrc? There would have to be exclusions for the vms wrappers. I don't know how to leverage the fopen wrapper in memdebug for this.

Member

jay commented Jun 1, 2015

Do you mean scan like add a regex in checksrc? There would have to be exclusions for the vms wrappers. I don't know how to leverage the fopen wrapper in memdebug for this.

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Jun 2, 2015

Member

Doing it with checksrc is probably what makes the most sense, and it already has a whitelisting concept so it should work fine. I'll work on it!

Member

bagder commented Jun 2, 2015

Doing it with checksrc is probably what makes the most sense, and it already has a whitelisting concept so it should work fine. I'll work on it!

bagder added a commit that referenced this pull request Jun 2, 2015

checksrc: detect fopen() for text without the FOPEN_* macros
Follow-up to e8423f9 with discussionis in
#258

This check scans for fopen() with a mode string without 'b' present, as
it may indicate that an FOPEN_* define should rather be used.
@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Jun 2, 2015

Member

Merged in commit 33ee411. I think we can consider this issue done now!

Member

bagder commented Jun 2, 2015

Merged in commit 33ee411. I think we can consider this issue done now!

@bagder bagder closed this Jun 2, 2015

@gvanem

This comment has been minimized.

Show comment
Hide comment
@gvanem

gvanem Jun 2, 2015

Member

@bagder wrote:

> #ifdef [the t-needing platforms]
> #define FOPEN_READTEXT "rt"
> #define FOPEN_WRITETEXT "wt"

I saw the commit for this in curl_setup.h. Isn't MSDOS also in the the t-needing platforms ?

Member

gvanem commented Jun 2, 2015

@bagder wrote:

> #ifdef [the t-needing platforms]
> #define FOPEN_READTEXT "rt"
> #define FOPEN_WRITETEXT "wt"

I saw the commit for this in curl_setup.h. Isn't MSDOS also in the the t-needing platforms ?

jay added a commit that referenced this pull request Jun 2, 2015

@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay Jun 2, 2015

Member

@gvanem Changed in 5943250. I can't find documentation for it but a simple program in Turbo C that calls fopen("file", "rt") works. I don't build libcurl for MS-DOS but if you do and you could test this change that would be great.

Member

jay commented Jun 2, 2015

@gvanem Changed in 5943250. I can't find documentation for it but a simple program in Turbo C that calls fopen("file", "rt") works. I don't build libcurl for MS-DOS but if you do and you could test this change that would be great.

jsonn pushed a commit to jsonn/pkgsrc that referenced this pull request Jun 30, 2015

spz
update of curl to version 7.43.0. Upstream RELEASE_NOTES:
Curl and libcurl 7.43.0

 Public curl releases:         147
 Command line options:         176
 curl_easy_setopt() options:   219
 Public functions in libcurl:  58
 Contributors:                 1291

This release includes the following changes:

 o Added CURLOPT_PROXY_SERVICE_NAME[11]
 o Added CURLOPT_SERVICE_NAME[12]
 o New curl option: --proxy-service-name[13]
 o Mew curl option: --service-name [14]
 o New curl option: --data-raw [5]
 o Added CURLOPT_PIPEWAIT [15]
 o Added support for multiplexing transfers using HTTP/2, enable this
   with the new CURLPIPE_MULTIPLEX bit for CURLMOPT_PIPELINING [16]
 o HTTP/2: requires nghttp2 1.0.0 or later
 o scripts: add zsh.pl for generating zsh completion
 o curl.h: add CURL_HTTP_VERSION_2

This release includes the following bugfixes:

 o CVE-2015-3236: lingering HTTP credentials in connection re-use [30]
 o CVE-2015-3237: SMB send off unrelated memory contents [31]
 o nss: fix compilation failure with old versions of NSS [1]
 o curl_easy_getinfo.3: document 'internals' in CURLINFO_TLS_SESSION
 o schannel.c: Fix possible SEC_E_BUFFER_TOO_SMALL error
 o Curl_ossl_init: load builtin modules [2]
 o configure: follow-up fix for krb5-config [3]
 o sasl_sspi: Populate domain from the realm in the challenge [4]
 o netrc: support 'default' token
 o README: convert to UTF-8
 o cyassl: Implement public key pinning
 o nss: implement public key pinning for NSS backend
 o mingw build: add arch -m32/-m64 to LDFLAGS
 o schannel: Fix out of bounds array [6]
 o configure: remove autogenerated files by autoconf
 o configure: remove --automake from libtoolize call
 o acinclude.m4: fix shell test for default CA cert bundle/path
 o schannel: fix regression in schannel_recv [7]
 o openssl: skip trace outputs for ssl_ver == 0 [8]
 o gnutls: properly retrieve certificate status
 o netrc: Read in text mode when cygwin [9]
 o winbuild: Document the option used to statically link the CRT [10]
 o FTP: Make EPSV use the control IP address rather than the original host
 o FTP: fix dangling conn->ip_addr dereference on verbose EPSV
 o conncache: keep bundles on host+port bases, not only host names
 o runtests.pl: use 'h2c' now, no -14 anymore
 o curlver: introducing new version number (checking) macros
 o openssl: boringssl build brekage, use SSL_CTX_set_msg_callback [17]
 o CURLOPT_POSTFIELDS.3: correct variable names [18]
 o curl_easy_unescape.3: update RFC reference [19]
 o gnutls: don't fail on non-fatal alerts during handshake
 o testcurl.pl: allow source to be in an arbitrary directory
 o CURLOPT_HTTPPROXYTUNNEL.3: only works with a HTTP proxy
 o SSPI-error: Change SEC_E_ILLEGAL_MESSAGE description [20]
 o parse_proxy: switch off tunneling if non-HTTP proxy [21]
 o share_init: fix OOM crash
 o perl: remove subdir, not touched in 9 years
 o CURLOPT_COOKIELIST.3: Add example
 o CURLOPT_COOKIE.3: Explain that the cookies won't be modified [22]
 o CURLOPT_COOKIELIST.3: Explain Set-Cookie without a domain [23]
 o FAQ: How do I port libcurl to my OS?
 o openssl: Use TLS_client_method for OpenSSL 1.1.0+
 o HTTP-NTLM: fail auth on connection close instead of looping [24]
 o curl_setup: Add macros for FOPEN_READTEXT, FOPEN_WRITETEXT [25]
 o curl_getdate.3: update RFC reference
 o curl_multi_info_read.3: added example
 o curl_multi_perform.3: added example
 o curl_multi_timeout.3: added example
 o cookie: Stop exporting any-domain cookies [26]
 o openssl: remove dummy callback use from SSL_CTX_set_verify()
 o openssl: remove SSL_get_session()-using code
 o openssl: removed USERDATA_IN_PWD_CALLBACK kludge
 o openssl: removed error string #ifdef
 o openssl: Fix verification of server-sent legacy intermediates [27]
 o docs: man page indentation and syntax fixes
 o docs: Spelling fixes
 o fopen.c: fix a few compiler warnings
 o CURLOPT_OPENSOCKETFUNCTION: return error at once [28]
 o schannel: Add support for optional client certificates
 o build: Properly detect OpenSSL 1.0.2 when using configure
 o urldata: store POST size in state.infilesize too [29]
 o security:choose_mech remove dead code
 o rtsp_do: remove dead code
 o docs: many HTTP URIs changed to HTTPS
 o schannel: schannel_recv overhaul [32]

This release includes the following known bugs:

 o see docs/KNOWN_BUGS (http://curl.haxx.se/docs/knownbugs.html)

This release would not have looked like this without help, code, reports and
advice from friends like these:

  Alessandro Ghedini, Alexander Dyagilev, Anders Bakken, Anthony Avina,
  Ashish Shukla, Bert Huijben, Brian Chrisman, Brian Prodoehl, Chris Araman,
  Dagobert Michelsen, Dan Fandrich, Daniel Melani, Daniel Stenberg,
  Dmitry Eremin-Solenikov, Drake Arconis, Egon Eckert, Frank Meier, Fred Stluka,
  Gisle Vanem, Grant Pannell, Isaac Boukris, Jens Rantil, Joel Depooter,
  Kamil Dudka, Linus Nielsen Feltzing, Linus Nielsen Feltzing Feltzing,
  Liviu Chircu, Marc Hoersken, Michael Osipov, Oren Souroujon, Orgad Shaneh,
  Patrick Monnerat, Patrick Rapin, Paul Howarth, Paul Oliver, Rafayel Mkrtchyan,
  Ray Satiro, Sean Boudreau, Tatsuhiro Tsujikawa, Tomas Tomecek, Viktor Szakáts,
  Ville Skyttä, Yehezkel Horowitz,
  (43 contributors)

        Thanks! (and sorry if I forgot to mention someone)

References to bug reports and discussions on issues:

 [1] = http://curl.haxx.se/mail/lib-2015-04/0095.html
 [2] = curl/curl#206
 [3] = curl/curl@5b66860#commitcomment-10473445
 [4] = curl/curl#141
 [5] = curl/curl#198
 [6] = http://curl.haxx.se/mail/lib-2015-04/0199.html
 [7] = curl/curl#244
 [8] = curl/curl#219
 [9] = curl/curl#258
 [10] = curl/curl#254
 [11] = http://curl.haxx.se/libcurl/c/CURLOPT_PROXY_SERVICE_NAME.html
 [12] = http://curl.haxx.se/libcurl/c/CURLOPT_SERVICE_NAME.html
 [13] = http://curl.haxx.se/docs/manpage.html#--proxy-service-name
 [14] = http://curl.haxx.se/docs/manpage.html#--service-name
 [15] = http://curl.haxx.se/libcurl/c/CURLOPT_PIPEWAIT.html
 [16] = http://curl.haxx.se/libcurl/c/CURLMOPT_PIPELINING.html
 [17] = curl/curl#275
 [18] = curl/curl#281
 [19] = curl/curl#282
 [20] = curl/curl#267
 [21] = http://curl.haxx.se/mail/lib-2015-05/0056.html
 [22] = http://curl.haxx.se/mail/lib-2015-05/0115.html
 [23] = http://curl.haxx.se/mail/lib-2015-05/0137.html
 [24] = curl/curl#256
 [25] = curl/curl#258 (comment)
 [26] = curl/curl#292
 [27] = https://rt.openssl.org/Ticket/Display.html?id=3621&user=guest&pass=guest
 [28] = http://curl.haxx.se/mail/lib-2015-06/0047.html
 [29] = http://curl.haxx.se/mail/lib-2015-06/0019.html
 [30] = http://curl.haxx.se/docs/adv_20150617A.html
 [31] = http://curl.haxx.se/docs/adv_20150617B.html
 [32] = curl/curl#244

jgsogo added a commit to jgsogo/curl that referenced this pull request Oct 19, 2015

netrc: Read in text mode when cygwin
Use text mode when cygwin to eliminate trailing carriage returns.

Bug: curl#258

jgsogo added a commit to jgsogo/curl that referenced this pull request Oct 19, 2015

curl_setup: Add macros for FOPEN_READTEXT, FOPEN_WRITETEXT
- Change fopen calls to use FOPEN_READTEXT instead of "r" or "rt"
- Change fopen calls to use FOPEN_WRITETEXT instead of "w" or "wt"

This change is to explicitly specify when we need to read/write text.
Unfortunately 't' is not part of POSIX fopen so we can't specify it
directly. Instead we now have FOPEN_READTEXT, FOPEN_WRITETEXT.

Prior to this change we had an issue on Windows if an application that
uses libcurl overrides the default file mode to binary. The default file
mode in Windows is normally text mode (translation mode) and that's what
libcurl expects.

Bug: curl#258 (comment)
Reported-by: Orgad Shaneh

jgsogo added a commit to jgsogo/curl that referenced this pull request Oct 19, 2015

checksrc: detect fopen() for text without the FOPEN_* macros
Follow-up to e8423f9 with discussionis in
curl#258

This check scans for fopen() with a mode string without 'b' present, as
it may indicate that an FOPEN_* define should rather be used.

jgsogo added a commit to jgsogo/curl that referenced this pull request Oct 19, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment