Cyberduck CLI: FTPS with X.509 Authentication

[PatrikZuffa]

Hi all,

I need help regarding the Cyberduck CLI commands. I found that the Cyberduck CLI automatically uses certificate with X.509 User authentication. I have my user certificate in my Local User Certificate store, I'm using the FTPS command but I get the response "FTP-SSL with username and password. No login credentials could be found in the Keychain." I guess that there is a problem or mismatch that my account on the FTPS server is not the same as the CN in the certificate (Login on FTPS server is different value that the CN in the PKI certificate).

In the GUI works all well because I select the certificate I want to use from drop down menu, but I didn't found any possibility to tell the Cyberduck CLI which certificate shall it use from my certificate repository in my user account in Windows. I could not find any mention in documentation how to choose the proper certificate from store with the CLI command.

Thanks for help and sharing thoughts,

Patrik

[kenny-ish]

Short version: duck has no option for this today, but you can still steer which certificate it uses, and the "No login credentials" message is a separate problem with an easy fix.

What the code does (current master, 73489db):

1. There is no CLI option for a client certificate. --identity is only the SSH private key file. The full option list in TerminalOptionsBuilder has nothing about certificates, and the CLI only accepts URLs: duck parses the URL with HostParser and never loads a .duck bookmark, so the "Client Certificate" setting you pick in the GUI (stored under the Client Certificate key in the bookmark file) never reaches the CLI. Passing a bookmark file instead of a URL does not work either.

2. The message you see is the password prompt, not a certificate error. LoginConnectionService.check() validates the credentials before it opens the connection, i.e. before the TLS handshake where the certificate would be selected. At that point the CLI has no certificate alias on the host object, so FTP-SSL falls back to "username and password" and asks for the password. If you pass both --username and --password, TerminalLoginService returns early and the prompt never appears. The username is the FTP login name, it does not have to match the certificate CN. If your server completes the login on USER alone (reply 230) because the TLS client certificate already authenticated you, the password value is never sent: the FTP client (commons-net) only sends PASS when the server answers 331 to USER. If your server does want a real password after the certificate, use the real one.

3. How the certificate is chosen in the CLI: on Windows, duck opens the Windows-MY keystore (your Personal store, SunMSCAPI). When the server requests a client certificate, DefaultCertificateStore.choose() takes all entries that have a private key and whose issuer matches what the server asked for, sorted alphabetically by alias, and uses the first one. There is no prompt in the CLI, which is the "automatically picks a certificate" behaviour you noticed. The chosen alias is then stored in the CLI preferences under a key of the form connection.ssl.keystore.<hostname>:<port>.<issuer list>.alias and reused on the next run (PreferencesX509KeyManager). On Windows the CLI preferences are written to %AppData%\Cyberduck\<product>.user.config; from the code the product name should be duck, so %AppData%\Cyberduck\duck.user.config, but I couldn't confirm the exact file name on a machine.

So, to make the CLI use a specific certificate:

Run once with --debug to see what duck sees and what it picked:

duck --list ftps://ftp.example.com/ --username yourftpuser --password x --debug

Look for log lines Alias in Keychain ..., Found private key for ... and Selected certificate alias .... The aliases come from the Windows store via SunMSCAPI; Cyberduck's own Windows code treats the alias as the certificate's Friendly Name, falling back to the subject's simple name (the CN) when there is none.

If the wrong certificate was picked, you have two options. Either open the user.config file mentioned above, find the connection.ssl.keystore....alias entry for your host and change its value to the alias of the certificate you want (duck reads this saved value before doing its own selection), or make the right certificate sort first: give it a Friendly Name in certmgr.msc (Personal > Certificates > right click > Properties) that comes alphabetically before the other matching certificates, delete the saved connection.ssl.keystore....alias entry so duck selects again, and re-run. Keeping only one certificate for that issuer in the Personal store also works, of course.

If you want a real --certificate <alias> option, that would be a feature request; nothing in the current CLI or in the docs provides it. I couldn't test any of this against an FTPS server that requires a client certificate, so please check the --debug output on your side.

[kenny-ish]

Hi Patrik,

Thanks for the log. It stops before duck opens the connection, so the certificate never gets a chance to be picked. These are the lines that matter, near the end:

Failed validation of credentials Credentials{user='PatrikZuffa', password='' ...
Login server.org ... FTP-SSL with username and password. No login credentials could be found in the Keychain.

duck checks for a username and a password before it connects. Your username came through, but the password is empty, so it stops at the login prompt. The TLS handshake, where the client certificate is chosen, only happens after that step. That is why there are no certificate or alias lines in the log at all.

Add --password and run it again:

duck --username PatrikZuffa --password <your FTP password> --list ftps://server.org/ --debug

If your server logs you in on the certificate alone, any non-empty value works, because the password is only sent when the server asks for one after USER. Otherwise use your real FTP password.

The log should then continue past the login into the connection, and you should see the certificate lines from my first reply (Alias in Keychain ..., Found private key for ..., Selected certificate alias ...). If it still picks the wrong certificate or none at all, please post that part of the log.

[dkocher]

Hi Patrik,

There is currently no CLI option to pick a client certificate. In the GUI the selection is stored per bookmark. The CLI has no prompt for it. If the server requests a certificate and none is saved, it takes the first matching one in your certificate store.

Two separate things are going on:

1. The Keychain message is about the FTP login, not the certificate. The certificate is used for the TLS handshake, but the server login still needs a username. The certificate's CN is not used as the login. Pass your FTP account with -u:

   duck --username ftpuser --list ftps://ftps.example.org/
   

2. To choose a certificate, set a preference for the host. The CLI reads preferences from environment variables. The name has the form connection.ssl.keystore.<host>:<port>.<issuers>.alias and the value is the certificate alias. <issuers> is the list of issuer names the server sends in its certificate request, for example [CN=My CA], or null if it sends none.

   PowerShell (names containing dots need braces):

   ${env:connection.ssl.keystore.ftps.example.org:990.[CN=My CA].alias} = "My certificate alias"
   duck --username ftpuser --list ftps://ftps.example.org/

   Command Prompt (the variable stays set for the rest of the session):

   set "connection.ssl.keystore.ftps.example.org:990.[CN=My CA].alias=My certificate alias"
   duck --username ftpuser --list ftps://ftps.example.org/
   

The host part is the name the connection resolves to, which can differ from the one you typed. Use port 990 for implicit FTPS or 21 for explicit. The issuer string must match exactly, so this is fiddly. I haven't tested this on Windows, so please let me know whether it works for you.

I think a proper --certificate <alias> option is the right fix, please follow #18553.

[PatrikZuffa]

Hi together,

many thanks for your effort. I really appreciate it. I tried to analyze the debug log but it seems that the certificate isn't chosen at all. My certificate is in my user certificate store. Could you please review and share the thoughts?

Thanks :)

Patrik

ps-output.txt