Added flags CURL_SSLVERSION_OR_UP_TO_* to CURLOPT_SSLVERSION #1166

Closed
wants to merge 25 commits into
from

Projects

None yet

6 participants

@jkralik
Contributor
jkralik commented Dec 17, 2016

By this patch user/developer can setup range of TLS versions, that
he want to support by curl/libcurl.

curl: curl --tlsv1.1 --up-to-tls-default https://www.google.com
libcurl: curl_easy_setopt(curl, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1.1 | CURL_SSLVERSION_OR_UP_TO_DEFAULT);

@jkralik, thanks for your PR! By analyzing the history of the files in this pull request, we identified @jay, @kdudka and @bagder to be potential reviewers.

Owner
jay commented Dec 17, 2016

OR_UP_TO doesn't seem too intuitive, why not something like --tlsmin 1.0 --tlsmax 1.2

Owner
bagder commented Dec 18, 2016

Yeah, max sounds better to my ears as well rather than "or_up_to". I think that if --tlsmax 1.2 is used, the regular --tlsv* option would be treated as the minimum version. Right?

Contributor
jkralik commented Dec 18, 2016 edited

Right. --tlsv* is treated as minimum. For tool I will rewrite args "up-to-tlsv1.2,..." as --tlsmax 1.2. Do you want to refactor name of flags too? Like CURL_SSLVERSION_OR_UP_TO_TLSv1.2 =>CURL_SSLVERSION_MAX_TLSv1.2

Collaborator
kdudka commented Dec 19, 2016

Yes, CURL_SSLVERSION_MAX_TLSv1.2 sounds less cryptic to me.

docs/cmdline-opts/tls-max.d
+Use up to TLSv1.2 . The supported minimum is tlsv1.0 or tlsv1.1.
+.IP "1.3"
+Use up to TLSv1.3 . The supported minimum is tlsv1.0 or tlsv1.1 or tlsv1.2.
+.RE
kdudka
kdudka Dec 19, 2016 Collaborator

Do I understand it correctly that minimum and maximum cannot be set to the same value? Why?

jkralik
jkralik Dec 19, 2016 Contributor

Yes it is supported. But it is wired to tls-max with tlsvX.Y when you want to just tlsvX.Y. If you wish I can add this option to documentation.

kdudka
kdudka Jan 19, 2017 Collaborator

I would drop all the "The supported minimum is ..." sentences from the particular options. The statement above says it clearly enough in a generic way. The fact that the minimum enabled TLS version cannot be higher than the maximum enabled TLS version is obvious. Also the space before comma appears disruptive to me.

jkralik
jkralik Jan 23, 2017 Contributor

I agree - removed sentences.

lib/vtls/darwinssl.c
+ switch(ssl_version) {
+ case CURL_SSLVERSION_TLSv1_0:
+ return set_ssl_version_up_to(conn, sockindex, ssl_version,
+ CURL_SSLVERSION_OR_UP_TO_TLSv1_0);
kdudka
kdudka Dec 19, 2016 Collaborator

Why do you recursively call the function? Would not it be enough to (re)assign ssl_version_up_to and continue the execution?

jkralik
jkralik Dec 19, 2016 Contributor

I agree. I will rewrite it as you suggest.

@bagder bagder added the SSL/TLS label Dec 19, 2016
Contributor
jkralik commented Jan 16, 2017

Hi Kamil.
Please, would you look at the latest changes?
Is it ok, now?
Thx

Collaborator
kdudka commented Jan 19, 2017
docs/curl.1
+Use up to TLSv1.3 . The supported minimum is tlsv1.0 or tlsv1.1 or tlsv1.2.
+.RE
+
+See also \fI--tlsv1.0\fP and \fI--tlsv1.1\fP and \fI--tlsv1.2\fP. \fI--tls-max\fP requires that the underlying libcurl was built to support TLS. Added in 7.53.0.
kdudka
kdudka Jan 19, 2017 Collaborator

Please break the line such that it does not exceed 79 columns, according to:
https://github.com/curl/curl/blob/master/docs/CODE_STYLE.md

jkralik
jkralik Jan 23, 2017 Contributor

docs/curl.1 is generated document by command: docs/cmdline-opts/gen.pl mainpage

docs/libcurl/opts/CURLOPT_SSLVERSION.3
@@ -50,6 +50,18 @@ TLSv1.1 (Added in 7.34.0)
TLSv1.2 (Added in 7.34.0)
.IP CURL_SSLVERSION_TLSv1_3
TLSv1.3 (Added in 7.52.0)
+.IP CURL_SSLVERSION_MAX_DEFAULT
+Use as flag with CURL_SSLVERSION_TLSv1_x and behavior is same as
+CURL_SSLVERSION_MAX_TLSv1_2.
kdudka
kdudka Jan 19, 2017 Collaborator

Why is CURL_SSLVERSION_MAX_DEFAULT same as CURL_SSLVERSION_MAX_TLSv1_2? How would one ask libcurl to set the maximum TLS version to the TLS library default?

jkralik
jkralik Jan 23, 2017 Contributor

I look to libraries and they are three kinds:

  1. supports to get/set default maximum: nss
  2. maximum is not determined: openssl, gnutls, mbedtls
  3. doesn't support: darwinSSL, polarssl, gskit

For 1. default is determined by library.
For 2. what is default? eg: library supports TLSv3, but TLSv2 in this time is recommended.
For 3. default must be determined by us.

My suggestion:
For 1. Function <Curl_ssl_evaluate_version_max> (renamed retrieve_ssl_version_max) calls function <Curl_ssl_get_default_max_version> that returns CURL_SSLVERSION_MAX_TLSvX for certain ssl library.
For 2. I suggest use behavior as in point 3.
For 3. <Curl_ssl_get_default_max_version> returns CURL_SSLVERSION_MAX_TLSv1_2.

What do you think?

kdudka
kdudka Jan 23, 2017 Collaborator

Even if NSS was the only library that currently allows one to set sslver.max to TLS library default, it is still worth to have this feature exposed via libcurl API. We can hard-code it to TLSv1.2 for other libraries now and eventually implement propagation of the flag once other libraries introduce a similar API.

However, it needs to be hard-coded in libcurl implementation, not in its API/ABI. If both CURL_SSLVERSION_MAX_DEFAULT and CURL_SSLVERSION_MAX_TLSv1_2 were mapped to the same constant in libcurl API/ABI, it would be nearly impossible to change the default later on.

jkralik
jkralik Jan 23, 2017 Contributor

Now, CURL_SSLVERSION_MAX_DEFAULT is as separeted value and every library CURL_SSLVERSION_MAX_DEFAULT must serve self (NSS via SSL_VersionRangeGetDefault and others as CURL_SSLVERSION_MAX_TLSv1.2).

include/curl/curl.h
+#define GET_CURL_SSLVERSION(x) (x & 0xffff)
+#define GET_CURL_SSLVERSION_MAX(x) (x & 0xffff0000)
+#define SET_CURL_SSLVERSION_MAX(x, val) \
+ CURL_SSLVERSION_MAX_##x = (val << 16)
kdudka
kdudka Jan 19, 2017 Collaborator

You are changing a public header file. All the defines you introduce need to start with the CURL prefix.

jkralik
jkralik Jan 23, 2017 Contributor

Fixed

include/curl/curl.h
+ SET_CURL_SSLVERSION_MAX(TLSv1_3, 4),
+
+ SET_CURL_SSLVERSION_MAX(LAST, 5), /* never use, keep last */
+ SET_CURL_SSLVERSION_MAX(DEFAULT, 3) /* = ..._OR_UP_TO_TLSv1_2 */
kdudka
kdudka Jan 19, 2017 Collaborator

Looks wrong to me. If we changed the default in libcurl later on, all dependent software would need to be recompiled for the change to take effect. You need to assign a separate value for CURL_SSLVERSION_MAX_DEFAULT so that a rebuilt of (dynamically linked) libcurl is sufficient to change the default.

lib/vtls/ssl_hlp.c
+ }
+ }
+ return ssl_version_max_default;
+}
kdudka
kdudka Jan 19, 2017 Collaborator

Why are you introducing a new module for the helper function? It could be added to the vtls module, which all the crypto backends use already. To be honest, I am not convinced that such a helper function simplifies the code anyhow. At least in the nss backend it does not seem to be the case.

lib/vtls/nss.c
+ }
+ return CURLE_OK;
+}
+
kdudka
kdudka Jan 19, 2017 edited Collaborator

This drastically increases the code complexity, which is unnecessary IMO. I will propose a better approach once the API issues (namely CURL_SSLVERSION_MAX_DEFAULT) are resolved.

jkralik
jkralik Jan 23, 2017 Contributor

CURL_SSLVERSION_MAX_DEFAULT is now as separated value and for NSS it uses maximum version from SSL_VersionRangeGetDefault. For other libraries it is CURL_SSLVERSION_MAX_TLSv1.2.

jkralik added some commits Dec 13, 2016
@jkralik jkralik tls-max: Added flags CURL_SSLVERSION_OR_UP_TO_* to CURLOPT_SSLVERSION
By this patch user/developer can setup range of TLS versions, that
he want to support by curl/libcurl.
21143bc
@jkralik jkralik fix of SPACEBEFOREPAREN 03918ee
@jkralik jkralik added missing symbols in versions:
  CURL_SSLVERSION_OR_UP_TO_NONE,
  CURL_SSLVERSION_OR_UP_TO_FIRST,
  CURL_SSLVERSION_OR_UP_TO_TLSv1_0
8a5b3ab
@jkralik jkralik fix documentation and added curl.1 007fc8c
@jkralik jkralik fix of help: renamed up-to-tls-tlsv to up-to-tlsv d90c799
@jkralik jkralik refactoring tool curl: tls-max instead of up-to-tls* b48f5cb
@jkralik jkralik refactor: rename CURL_SSLVERSION_OR_UP_TO* to CURL_SSLVERSION_MAX* c38fb2a
@jkralik jkralik refactoring: remove recursion from set_ssl_version_min_max dc572c9
@jkralik jkralik Added flags CURL_SSLVERSION_OR_UP_TO_* to CURLOPT_SSLVERSION
By this patch user/developer can setup range of TLS versions, that
he want to support by curl/libcurl.
e4ae8ba
@jkralik jkralik Removing unnecessary sentences about minimum SSL versions 1521c82
@jkralik jkralik Renaming GET/SET_CURL_SSLVERSION to CURL_GET/SET_SSLVERSION f43ce22
@jkralik jkralik CURL_SSLVERSION_MAX_DEFAULT with own value.
For NSS it set enabled maximum version of this library
For others it set CURL_SSLVERSION_MAX_TLSv1.2
2526749
@jkralik jkralik Added recent documentation
6cd599d
@jkralik jkralik Added missing symbols (symbols-in-versions)
352116f
@jkralik jkralik Fix of documentation
f28e480
Contributor
jkralik commented Jan 25, 2017

Hi Kamil
I must reorder commits in my clone because it was mixed with curl master. Sorry for that.

  1. It is rebased.
  2. CURL_SSLVERSION_MAX_DEFAULT has own value, and for NSS it uses default from library, for others it is TLSv1.2
  3. retrieve_ssl_version_max renamed to Curl_ssl_retrieve_version_max moved to vtls.{h,c}
  4. Documentation also fixed.

I want squash all commits as one before it will be merged to master.
What do you think now?
Thx

Collaborator
kdudka commented Jan 25, 2017
docs/curl.1
@@ -0,0 +1,2726 @@
+.\" **************************************************************************
kdudka
kdudka Jan 25, 2017 Collaborator

This file was intentionally removed upstream in 4c49b83. It should not be re-created with this PR.

docs/libcurl/symbols-in-versions
@@ -709,6 +709,8 @@ CURL_FORMADD_NULL 7.9.8
CURL_FORMADD_OK 7.9.8
CURL_FORMADD_OPTION_TWICE 7.9.8
CURL_FORMADD_UNKNOWN_OPTION 7.9.8
+CURL_GET_SSLVERSION 7.53.0
+CURL_GET_SSLVERSION_MAX 7.53.0
kdudka
kdudka Jan 25, 2017 Collaborator

I do not think these helper defines should be mentioned in symbols-in-versions. They are not supposed to be used externally.

jkralik
jkralik Jan 26, 2017 Contributor

When i remove it I have issue with test 1119:


test 1119...[Verify that symbols-in-versions and headers are in sync]

perl  returned 2, when expecting 0

 exit FAILED

== Contents of files in the log/ dir after test 1119

=== Start of file stdout1119

 CURL_GET_SSLVERSION

 CURL_GET_SSLVERSION_MAX

 CURL_SET_SSLVERSION_MAX

=== End of file stdout1119

And don't find any suffix word for ignore them in a script tests/symbol-scan.pl. Can I add suffix word "_PRIVATE" or "_HELPER" to the script ?

bagder
bagder Jan 26, 2017 Owner

If they're not supposed to be used externally, then they shouldn't be in the public header, right?

If they are in the public header, they should be in symbols-in-versions I think.

jkralik added some commits Jan 26, 2017
@jkralik jkralik Removing docs/curl.1
8d5756e
@jkralik jkralik removed macro CURL_SET_SSLVERSION_MAX
f178c42
@jkralik jkralik CURL_GET_SSLVERSION, CURL_GET_SSLVERSION removed
CURLOPT_PROXY_SSLVERSION will be to support to set SSLVERSION_MAX
a218844
Contributor
jkralik commented Jan 27, 2017

Ok. I removed defines: CURL_SET_SSLVERSION_MAX, CURL_GET_SSLVERSION, CURL_GET_SSLVERSION_MAX and added feature SSLVERSION_MAX to CURLOPT_PROXY_SSLVERSION.

Collaborator
kdudka commented Feb 1, 2017

Thanks for all the updates! I am proposing some changes on top of that to deduplicate the resulting code.

The first one makes that CURL_SSLVERSION_TLSv1* constants easy to translate from the corresponding CURL_SSLVERSION_MAX_TLSv1 constants:
kdudka@a28cd25

The second one shows how the code could be deduplicated for the NSS backend:
kdudka@ee30c39

I believe that no functions like Curl_ssl_retrieve_version_max() will be needed any more if we simplify the code similarly for other backends. Please let me know if you want me to prepare such a patch. This is just a preview for eventual discussion.

kdudka and others added some commits Feb 1, 2017
@kdudka @jkralik kdudka curl.h: assign CURL_SSLVERSION_MAX_* constants systematically
... such that we can easily derive the corresponding CURL_SSLVERSION_*
constant for minimal SSL version and make it possible to share library
code that sets both the parameters.
fbdab0a
@kdudka @jkralik kdudka nss: simplify the code of SSL version mapping 0523638
@jkralik jkralik simplify sets min, max of ssl version in ssl libraries
0b7992f
Contributor
jkralik commented Feb 3, 2017

Hello folks. I drop Curl_ssl_retrieve_version_max and simplify the code similarly as Kamil for others backends.

+value from SSL library. Only library NSS currently allows to get
+maximum supported TLS version.
+(Added in 7.53.0)
+.IP CURL_SSLVERSION_MAX_TLSv1_1
kdudka
kdudka Feb 3, 2017 Collaborator

Documentation for CURL_SSLVERSION_MAX_TLSv1_0 is missing.

jkralik
jkralik Feb 3, 2017 Contributor

Fixed

docs/libcurl/opts/CURLOPT_SSLVERSION.3
+value from SSL library. Only library NSS currently allows to get
+maximum supported TLS version.
+(Added in 7.53.0)
+.IP CURL_SSLVERSION_MAX_TLSv1_1
kdudka
kdudka Feb 3, 2017 Collaborator

Same here. Documentation for CURL_SSLVERSION_MAX_TLSv1_0 is missing.

jkralik
jkralik Feb 3, 2017 Contributor

Fixed

src/tool_paramhlp.c
+ size_t i = 0;
+ if(!str)
+ return PARAM_REQUIRES_PARAMETER;
+ for(i = 0; i < sizeof(tls_max_array)/sizeof(tls_max_array[i]); i++) {
kdudka
kdudka Feb 3, 2017 edited Collaborator

sizeof(tls_max_array)/sizeof(tls_max_array[0])

jkralik
jkralik Feb 3, 2017 Contributor

good point :)

lib/vtls/darwinssl.c
+ if(ssl_version_max == CURL_SSLVERSION_MAX_NONE) {
+ ssl_version_max = ssl_version;
+ }
+
kdudka
kdudka Feb 3, 2017 Collaborator

Should not we handle CURL_SSLVERSION_MAX_DEFAULT at this point?

jkralik
jkralik Feb 3, 2017 Contributor

Fixed

Contributor
frenche commented Feb 3, 2017

This is interesting work, btw why not make the tls version a bit-mask instead of min-max, so it would be possible to do things like '--tlsv1.0 --tlsv1.2' for example.
Is that the control that the tls api / protocol allows?

lib/vtls/gtls.c
@@ -375,6 +375,98 @@ static gnutls_x509_crt_fmt_t do_file_type(const char *type)
return -1;
}
+#ifndef USE_GNUTLS_PRIORITY_SET_DIRECT
+static CURLcode
+set_ssl_version_min_max(int *protocol_priority, struct connectdata *conn)
kdudka
kdudka Feb 3, 2017 Collaborator

This can lead to stack overflow if new TLS versions were added later on but size of the array not updated. Please make set_ssl_version_min_max() take size of the array as an argument and let it check the boundaries before writing to that array.

jkralik
jkralik Feb 3, 2017 Contributor

Fixed

lib/vtls/gtls.c
+ case CURL_SSLVERSION_TLSv1_2 | CURL_SSLVERSION_MAX_DEFAULT:
+ *prioritylist = GNUTLS_CIPHERS ":-VERS-SSL3.0:-VERS-TLS-ALL:"
+ "+VERS-TLS1.2:" GNUTLS_SRP;
+ break;
kdudka
kdudka Feb 3, 2017 Collaborator

I guess we should fail by default instead of pretending success.

jkralik
jkralik Feb 3, 2017 Contributor

By default returns func CURLE_SSL_CONNECT_ERROR

lib/vtls/mbedtls.c
+ CURLcode result = CURLE_OK;
+ long ssl_version = SSL_CONN_CONFIG(version);
+ long ssl_version_max = SSL_CONN_CONFIG(version_max) >> 16;
+ if(ssl_version_max == CURL_SSLVERSION_MAX_NONE) {
kdudka
kdudka Feb 3, 2017 Collaborator

The check is not going to work as expected because ssl_version_max is already shifted. You might also want to handle CURL_SSLVERSION_MAX_DEFAULT at this point.

jkralik
jkralik Feb 3, 2017 Contributor

Fixed

Collaborator
kdudka commented Feb 3, 2017
jkralik added some commits Feb 3, 2017
@jkralik jkralik fix of shifts
fix of usin CURL_SSLVERSION_MAX_DEFAULT
426d1cd
@jkralik jkralik fix of doc: added CURL_SSLVERSION_MAX_TLSv1_0 9812598
@jkralik jkralik use first member to calc size of array
y
5a2477f
Contributor
frenche commented Feb 3, 2017
Collaborator
kdudka commented Feb 6, 2017

Looks good to me. Thanks for all the fixes! As it is two weeks after devel freeze now, I propose to wait till curl-7.53.0 is out, then rebase on top of upstream and push. Additional review comments are welcome!

@jkralik jkralik fix of windows compilation
ff5bb77
Collaborator
kdudka commented Feb 27, 2017

I have squashed the commits and rebased against latest upstream. The result is available in the tls-max branch of my git repo: kdudka@master...tls-max

Are we ready to merge this upstream?

Contributor
jkralik commented Feb 27, 2017

Hi Kamil. Do we want to set version "7.53.0" in documentation (and not "7.54.0") ?

Collaborator
kdudka commented Feb 27, 2017

Good point. 7.53.0 needs to be substituted by 7.54.0 everywhere in the patch.

Contributor
jkralik commented Feb 28, 2017

I create a patch (attached) to fix it for https://github.com/kdudka/curl/tree/tls-max. and I agree with merge.

0001-fixup-docs-set-right-version-for-specify-range-of-en.txt

@kdudka kdudka added a commit to kdudka/curl that referenced this pull request Mar 1, 2017
@jkralik @kdudka jkralik + kdudka vtls: add options to specify range of enabled TLS versions
This commit introduces the CURL_SSLVERSION_MAX_* constants as well as
the --tls-max option of the curl tool.

Closes curl#1166
87a6a97
Collaborator
kdudka commented Mar 1, 2017

Thanks for the update! I have squashed all the fixup commits:
kdudka@master...tls-max

@bagder are you fine with merging it upstream as it is?

@kdudka kdudka added a commit to kdudka/curl that referenced this pull request Mar 8, 2017
@jkralik @kdudka jkralik + kdudka vtls: add options to specify range of enabled TLS versions
This commit introduces the CURL_SSLVERSION_MAX_* constants as well as
the --tls-max option of the curl tool.

Closes curl#1166
869c315
Collaborator
kdudka commented Mar 8, 2017

Rebased on top of upstream.

Owner
bagder commented Mar 8, 2017

LGTM, ready to merge. When merged, this will make the next release truly become "7.54.0"

Collaborator
kdudka commented Mar 8, 2017

I have appended one more fixup to sync packages/OS400/curl.inc.in: kdudka@e8ef23d

@kdudka kdudka added a commit that closed this pull request Mar 8, 2017
@jkralik @kdudka jkralik + kdudka vtls: add options to specify range of enabled TLS versions
This commit introduces the CURL_SSLVERSION_MAX_* constants as well as
the --tls-max option of the curl tool.

Closes #1166
6448f98
@kdudka kdudka closed this in 6448f98 Mar 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment