Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document that --tlspassword takes a password #7378

Closed
wants to merge 1 commit into from

Conversation

@verhovsky
Copy link
Contributor

@verhovsky verhovsky commented Jul 11, 2021

No description provided.

@verhovsky
Copy link
Contributor Author

@verhovsky verhovsky commented Jul 11, 2021

I threw together this Python script to compare tool_getparam.c against the output of curl --help and also found that --request-target and -Q/--quote (and technically --stderr as well) don't document the fact that they take an argument.

import re
import subprocess

# https://github.com/curl/curl/tree/999d3d1e773d573e5edf069783417657d8390d06/src/tool_getparam.c
args = '''\
  {"*@", "url",                      ARG_STRING},
  {"*4", "dns-ipv4-addr",            ARG_STRING},
  {"*6", "dns-ipv6-addr",            ARG_STRING},
  {"*a", "random-file",              ARG_FILENAME},
  {"*b", "egd-file",                 ARG_STRING},
  {"*B", "oauth2-bearer",            ARG_STRING},
  {"*c", "connect-timeout",          ARG_STRING},
  {"*C", "doh-url"        ,          ARG_STRING},
  {"*d", "ciphers",                  ARG_STRING},
  {"*D", "dns-interface",            ARG_STRING},
  {"*e", "disable-epsv",             ARG_BOOL},
  {"*f", "disallow-username-in-url", ARG_BOOL},
  {"*E", "epsv",                     ARG_BOOL},
         /* 'epsv' made like this to make --no-epsv and --epsv to work
             although --disable-epsv is the documented option */
  {"*F", "dns-servers",              ARG_STRING},
  {"*g", "trace",                    ARG_FILENAME},
  {"*G", "npn",                      ARG_BOOL},
  {"*h", "trace-ascii",              ARG_FILENAME},
  {"*H", "alpn",                     ARG_BOOL},
  {"*i", "limit-rate",               ARG_STRING},
  {"*j", "compressed",               ARG_BOOL},
  {"*J", "tr-encoding",              ARG_BOOL},
  {"*k", "digest",                   ARG_BOOL},
  {"*l", "negotiate",                ARG_BOOL},
  {"*m", "ntlm",                     ARG_BOOL},
  {"*M", "ntlm-wb",                  ARG_BOOL},
  {"*n", "basic",                    ARG_BOOL},
  {"*o", "anyauth",                  ARG_BOOL},
#ifdef USE_WATT32
  {"*p", "wdebug",                   ARG_BOOL},
#endif
  {"*q", "ftp-create-dirs",          ARG_BOOL},
  {"*r", "create-dirs",              ARG_BOOL},
  {"*R", "create-file-mode",         ARG_STRING},
  {"*s", "max-redirs",               ARG_STRING},
  {"*t", "proxy-ntlm",               ARG_BOOL},
  {"*u", "crlf",                     ARG_BOOL},
  {"*v", "stderr",                   ARG_FILENAME},
  {"*V", "aws-sigv4",                ARG_STRING},
  {"*w", "interface",                ARG_STRING},
  {"*x", "krb",                      ARG_STRING},
  {"*x", "krb4",                     ARG_STRING},
         /* 'krb4' is the previous name */
  {"*X", "haproxy-protocol",         ARG_BOOL},
  {"*y", "max-filesize",             ARG_STRING},
  {"*z", "disable-eprt",             ARG_BOOL},
  {"*Z", "eprt",                     ARG_BOOL},
         /* 'eprt' made like this to make --no-eprt and --eprt to work
             although --disable-eprt is the documented option */
  {"*~", "xattr",                    ARG_BOOL},
  {"$a", "ftp-ssl",                  ARG_BOOL},
         /* 'ftp-ssl' deprecated name since 7.20.0 */
  {"$a", "ssl",                      ARG_BOOL},
         /* 'ssl' new option name in 7.20.0, previously this was ftp-ssl */
  {"$b", "ftp-pasv",                 ARG_BOOL},
  {"$c", "socks5",                   ARG_STRING},
  {"$d", "tcp-nodelay",              ARG_BOOL},
  {"$e", "proxy-digest",             ARG_BOOL},
  {"$f", "proxy-basic",              ARG_BOOL},
  {"$g", "retry",                    ARG_STRING},
  {"$V", "retry-connrefused",        ARG_BOOL},
  {"$h", "retry-delay",              ARG_STRING},
  {"$i", "retry-max-time",           ARG_STRING},
  {"$k", "proxy-negotiate",          ARG_BOOL},
  {"$m", "ftp-account",              ARG_STRING},
  {"$n", "proxy-anyauth",            ARG_BOOL},
  {"$o", "trace-time",               ARG_BOOL},
  {"$p", "ignore-content-length",    ARG_BOOL},
  {"$q", "ftp-skip-pasv-ip",         ARG_BOOL},
  {"$r", "ftp-method",               ARG_STRING},
  {"$s", "local-port",               ARG_STRING},
  {"$t", "socks4",                   ARG_STRING},
  {"$T", "socks4a",                  ARG_STRING},
  {"$u", "ftp-alternative-to-user",  ARG_STRING},
  {"$v", "ftp-ssl-reqd",             ARG_BOOL},
         /* 'ftp-ssl-reqd' deprecated name since 7.20.0 */
  {"$v", "ssl-reqd",                 ARG_BOOL},
         /* 'ssl-reqd' new in 7.20.0, previously this was ftp-ssl-reqd */
  {"$w", "sessionid",                ARG_BOOL},
         /* 'sessionid' listed as --no-sessionid in the help */
  {"$x", "ftp-ssl-control",          ARG_BOOL},
  {"$y", "ftp-ssl-ccc",              ARG_BOOL},
  {"$j", "ftp-ssl-ccc-mode",         ARG_STRING},
  {"$z", "libcurl",                  ARG_STRING},
  {"$#", "raw",                      ARG_BOOL},
  {"$0", "post301",                  ARG_BOOL},
  {"$1", "keepalive",                ARG_BOOL},
         /* 'keepalive' listed as --no-keepalive in the help */
  {"$2", "socks5-hostname",          ARG_STRING},
  {"$3", "keepalive-time",           ARG_STRING},
  {"$4", "post302",                  ARG_BOOL},
  {"$5", "noproxy",                  ARG_STRING},
  {"$7", "socks5-gssapi-nec",        ARG_BOOL},
  {"$8", "proxy1.0",                 ARG_STRING},
  {"$9", "tftp-blksize",             ARG_STRING},
  {"$A", "mail-from",                ARG_STRING},
  {"$B", "mail-rcpt",                ARG_STRING},
  {"$C", "ftp-pret",                 ARG_BOOL},
  {"$D", "proto",                    ARG_STRING},
  {"$E", "proto-redir",              ARG_STRING},
  {"$F", "resolve",                  ARG_STRING},
  {"$G", "delegation",               ARG_STRING},
  {"$H", "mail-auth",                ARG_STRING},
  {"$I", "post303",                  ARG_BOOL},
  {"$J", "metalink",                 ARG_BOOL},
  {"$6", "sasl-authzid",             ARG_STRING},
  {"$K", "sasl-ir",                  ARG_BOOL },
  {"$L", "test-event",               ARG_BOOL},
  {"$M", "unix-socket",              ARG_FILENAME},
  {"$N", "path-as-is",               ARG_BOOL},
  {"$O", "socks5-gssapi-service",    ARG_STRING},
         /* 'socks5-gssapi-service' merged with'proxy-service-name' and
            deprecated since 7.49.0 */
  {"$O", "proxy-service-name",       ARG_STRING},
  {"$P", "service-name",             ARG_STRING},
  {"$Q", "proto-default",            ARG_STRING},
  {"$R", "expect100-timeout",        ARG_STRING},
  {"$S", "tftp-no-options",          ARG_BOOL},
  {"$U", "connect-to",               ARG_STRING},
  {"$W", "abstract-unix-socket",     ARG_FILENAME},
  {"$X", "tls-max",                  ARG_STRING},
  {"$Y", "suppress-connect-headers", ARG_BOOL},
  {"$Z", "compressed-ssh",           ARG_BOOL},
  {"$~", "happy-eyeballs-timeout-ms", ARG_STRING},
  {"$!", "retry-all-errors",         ARG_BOOL},
  {"0",   "http1.0",                 ARG_NONE},
  {"01",  "http1.1",                 ARG_NONE},
  {"02",  "http2",                   ARG_NONE},
  {"03",  "http2-prior-knowledge",   ARG_NONE},
  {"04",  "http3",                   ARG_NONE},
  {"09",  "http0.9",                 ARG_BOOL},
  {"1",  "tlsv1",                    ARG_NONE},
  {"10",  "tlsv1.0",                 ARG_NONE},
  {"11",  "tlsv1.1",                 ARG_NONE},
  {"12",  "tlsv1.2",                 ARG_NONE},
  {"13",  "tlsv1.3",                 ARG_NONE},
  {"1A", "tls13-ciphers",            ARG_STRING},
  {"1B", "proxy-tls13-ciphers",      ARG_STRING},
  {"2",  "sslv2",                    ARG_NONE},
  {"3",  "sslv3",                    ARG_NONE},
  {"4",  "ipv4",                     ARG_NONE},
  {"6",  "ipv6",                     ARG_NONE},
  {"a",  "append",                   ARG_BOOL},
  {"A",  "user-agent",               ARG_STRING},
  {"b",  "cookie",                   ARG_STRING},
  {"ba", "alt-svc",                  ARG_STRING},
  {"bb", "hsts",                     ARG_STRING},
  {"B",  "use-ascii",                ARG_BOOL},
  {"c",  "cookie-jar",               ARG_STRING},
  {"C",  "continue-at",              ARG_STRING},
  {"d",  "data",                     ARG_STRING},
  {"dr", "data-raw",                 ARG_STRING},
  {"da", "data-ascii",               ARG_STRING},
  {"db", "data-binary",              ARG_STRING},
  {"de", "data-urlencode",           ARG_STRING},
  {"D",  "dump-header",              ARG_FILENAME},
  {"e",  "referer",                  ARG_STRING},
  {"E",  "cert",                     ARG_FILENAME},
  {"Ea", "cacert",                   ARG_FILENAME},
  {"Eb", "cert-type",                ARG_STRING},
  {"Ec", "key",                      ARG_FILENAME},
  {"Ed", "key-type",                 ARG_STRING},
  {"Ee", "pass",                     ARG_STRING},
  {"Ef", "engine",                   ARG_STRING},
  {"Eg", "capath",                   ARG_FILENAME},
  {"Eh", "pubkey",                   ARG_STRING},
  {"Ei", "hostpubmd5",               ARG_STRING},
  {"Ej", "crlfile",                  ARG_FILENAME},
  {"Ek", "tlsuser",                  ARG_STRING},
  {"El", "tlspassword",              ARG_STRING},
  {"Em", "tlsauthtype",              ARG_STRING},
  {"En", "ssl-allow-beast",          ARG_BOOL},
  {"Eo", "ssl-auto-client-cert",     ARG_BOOL},
  {"EO", "proxy-ssl-auto-client-cert", ARG_BOOL},
  {"Ep", "pinnedpubkey",             ARG_STRING},
  {"EP", "proxy-pinnedpubkey",       ARG_STRING},
  {"Eq", "cert-status",              ARG_BOOL},
  {"EQ", "doh-cert-status",          ARG_BOOL},
  {"Er", "false-start",              ARG_BOOL},
  {"Es", "ssl-no-revoke",            ARG_BOOL},
  {"ES", "ssl-revoke-best-effort",   ARG_BOOL},
  {"Et", "tcp-fastopen",             ARG_BOOL},
  {"Eu", "proxy-tlsuser",            ARG_STRING},
  {"Ev", "proxy-tlspassword",        ARG_STRING},
  {"Ew", "proxy-tlsauthtype",        ARG_STRING},
  {"Ex", "proxy-cert",               ARG_FILENAME},
  {"Ey", "proxy-cert-type",          ARG_STRING},
  {"Ez", "proxy-key",                ARG_FILENAME},
  {"E0", "proxy-key-type",           ARG_STRING},
  {"E1", "proxy-pass",               ARG_STRING},
  {"E2", "proxy-ciphers",            ARG_STRING},
  {"E3", "proxy-crlfile",            ARG_FILENAME},
  {"E4", "proxy-ssl-allow-beast",    ARG_BOOL},
  {"E5", "login-options",            ARG_STRING},
  {"E6", "proxy-cacert",             ARG_FILENAME},
  {"E7", "proxy-capath",             ARG_FILENAME},
  {"E8", "proxy-insecure",           ARG_BOOL},
  {"E9", "proxy-tlsv1",              ARG_NONE},
  {"EA", "socks5-basic",             ARG_BOOL},
  {"EB", "socks5-gssapi",            ARG_BOOL},
  {"EC", "etag-save",                ARG_FILENAME},
  {"ED", "etag-compare",             ARG_FILENAME},
  {"EE", "curves",                   ARG_STRING},
  {"f",  "fail",                     ARG_BOOL},
  {"fa", "fail-early",               ARG_BOOL},
  {"fb", "styled-output",            ARG_BOOL},
  {"fc", "mail-rcpt-allowfails",     ARG_BOOL},
  {"fd", "fail-with-body",           ARG_BOOL},
  {"F",  "form",                     ARG_STRING},
  {"Fs", "form-string",              ARG_STRING},
  {"g",  "globoff",                  ARG_BOOL},
  {"G",  "get",                      ARG_NONE},
  {"Ga", "request-target",           ARG_STRING},
  {"h",  "help",                     ARG_BOOL},
  {"H",  "header",                   ARG_STRING},
  {"Hp", "proxy-header",             ARG_STRING},
  {"i",  "include",                  ARG_BOOL},
  {"I",  "head",                     ARG_BOOL},
  {"j",  "junk-session-cookies",     ARG_BOOL},
  {"J",  "remote-header-name",       ARG_BOOL},
  {"k",  "insecure",                 ARG_BOOL},
  {"kd", "doh-insecure",             ARG_BOOL},
  {"K",  "config",                   ARG_FILENAME},
  {"l",  "list-only",                ARG_BOOL},
  {"L",  "location",                 ARG_BOOL},
  {"Lt", "location-trusted",         ARG_BOOL},
  {"m",  "max-time",                 ARG_STRING},
  {"M",  "manual",                   ARG_BOOL},
  {"n",  "netrc",                    ARG_BOOL},
  {"no", "netrc-optional",           ARG_BOOL},
  {"ne", "netrc-file",               ARG_FILENAME},
  {"N",  "buffer",                   ARG_BOOL},
         /* 'buffer' listed as --no-buffer in the help */
  {"o",  "output",                   ARG_FILENAME},
  {"O",  "remote-name",              ARG_NONE},
  {"Oa", "remote-name-all",          ARG_BOOL},
  {"Ob", "output-dir",               ARG_STRING},
  {"p",  "proxytunnel",              ARG_BOOL},
  {"P",  "ftp-port",                 ARG_STRING},
  {"q",  "disable",                  ARG_BOOL},
  {"Q",  "quote",                    ARG_STRING},
  {"r",  "range",                    ARG_STRING},
  {"R",  "remote-time",              ARG_BOOL},
  {"s",  "silent",                   ARG_BOOL},
  {"S",  "show-error",               ARG_BOOL},
  {"t",  "telnet-option",            ARG_STRING},
  {"T",  "upload-file",              ARG_FILENAME},
  {"u",  "user",                     ARG_STRING},
  {"U",  "proxy-user",               ARG_STRING},
  {"v",  "verbose",                  ARG_BOOL},
  {"V",  "version",                  ARG_BOOL},
  {"w",  "write-out",                ARG_STRING},
  {"x",  "proxy",                    ARG_STRING},
  {"xa", "preproxy",                 ARG_STRING},
  {"X",  "request",                  ARG_STRING},
  {"Y",  "speed-limit",              ARG_STRING},
  {"y",  "speed-time",               ARG_STRING},
  {"z",  "time-cond",                ARG_STRING},
  {"Z",  "parallel",                 ARG_BOOL},
  {"Zb", "parallel-max",             ARG_STRING},
  {"Zc", "parallel-immediate",       ARG_BOOL},
  {"#",  "progress-bar",             ARG_BOOL},
  {"#m", "progress-meter",           ARG_BOOL},
  {":",  "next",                     ARG_NONE},
'''

actual = {}
for line in args.splitlines():
    if not line.startswith('  {'):
        continue
    _, name, arg_type = line.strip().strip('{},').split(',')
    name, arg_type = name.strip().strip('"'), arg_type.strip()
    # print(name, arg_type)
    if name in actual:
        print('repeated argument?', name, arg_type)
    actual[name] = arg_type

output = subprocess.run(['curl', '--help'], capture_output=True, text=True).stdout
reported = {}
reported_lines = {}
for line in output.splitlines():
    if not line.startswith(' '):
        continue
    pattern = re.compile(r'^( -(?P<short>.),)? +--(?P<long>\S+) +(<(?P<arg>.+)>)?')
    match = re.match(pattern, line)
    if not match:
        # print("couldn't parse line:", line)
        continue
    reported_lines[match.group('long')] = line
    if not match.group('arg'):
        reported[match.group('long')] = None
    reported[match.group('long')] = match.group('arg')

# print(reported)

undocumented = actual.keys() - reported.keys()
extra = reported.keys() - actual.keys()
shared_opts = reported.keys() & actual.keys()

# ['ARG_FILENAME', 'ARG_BOOL', 'ARG_STRING', 'ARG_NONE']

for opt in shared_opts:
    if reported[opt] is None:
        reported_arg_type = 'ARG_BOOL'
    elif reported[opt] in ['file', 'dir', 'path', 'filename']:
        reported_arg_type = 'ARG_FILENAME'
    else:
        reported_arg_type = 'ARG_STRING'

    # I don't understand the difference.
    if actual[opt] == 'ARG_NONE':
        actual[opt] = 'ARG_BOOL'

    if actual[opt] != reported_arg_type:
        print(opt, actual[opt], reported[opt])
        print(reported_lines[opt])
        print()

Output (it's half false positives because the logic is pretty rough):

tlspassword ARG_STRING None
     --tlspassword   TLS password

preproxy ARG_STRING None
     --preproxy [protocol://]host[:port] Use this proxy first

proxy ARG_STRING None
 -x, --proxy [protocol://]host[:port] Use this proxy

stderr ARG_FILENAME None
     --stderr        Where to redirect stderr

libcurl ARG_STRING file
     --libcurl <file> Dump libcurl equivalent code of this command line

proxy-key ARG_FILENAME key
     --proxy-key <key> Private key for HTTPS proxy

request-target ARG_STRING None
     --request-target Specify the target for this request

proxy-cert ARG_FILENAME cert[:passwd]
     --proxy-cert <cert[:passwd]> Set client certificate for proxy

dump-header ARG_FILENAME filename> Write the received headers to <filename
 -D, --dump-header <filename> Write the received headers to <filename>

cert ARG_FILENAME certificate[:password]
 -E, --cert <certificate[:password]> Client certificate file and password

quote ARG_STRING None
 -Q, --quote         Send command(s) to server before transfer

egd-file ARG_STRING file
     --egd-file <file> EGD socket path for random data

key ARG_FILENAME key
     --key <key>     Private key file name
@bagder
bagder approved these changes Jul 12, 2021
@danielgustafsson
Copy link
Member

@danielgustafsson danielgustafsson commented Jul 12, 2021

Pushed to master, thanks! Feel free to open another PR with your other findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants