Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl tries to read known_hosts from wrong location

Closed
jeffrson opened this issue Nov 19, 2021 · 5 comments · Fixed by #8035
Closed

curl tries to read known_hosts from wrong location #8033

jeffrson opened this issue Nov 19, 2021 · 5 comments · Fixed by #8035
Assignees
Labels

Comments

@jeffrson
Copy link

jeffrson commented Nov 19, 2021

I did this

curl -u user:pass -T file.dat sftp://host.domain.tld/folder

I expected the following

upload "file.dat" to host.domain.tld/folder

what happened

although host.domain.tld is contained in known_hosts there's an error message:
curl: (60) SSL peer certificate or SSH remote key was not OK

AFAICT, curl tries to load known_hosts from %APPDATA%\.ssh (user\AppData\Roaming\.ssh), while it should look at %USERPROFILE%\.ssh (user\.ssh), which is user's actual home folder and is used by OpenSSH integrated in Windows as well.

BTW, would be great to have an example of how to get the hash for --hostpubmd5 and --hostpubsha256 in the documentation (ie, what data to hash exactly).

curl/libcurl version

[curl -V output]

curl 7.80.0 (x86_64-pc-win32) libcurl/7.80.0 OpenSSL/3.0.0 (Schannel) zlib/1.2.11 brotli/1.0.9 zstd/1.5.0 libidn2/2.3.2 libssh2/1.10.0 nghttp2/1.46.0 libgsasl/1.10.0
Release-Date: 2021-11-10
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli gsasl HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL SSPI TLS-SRP UnixSockets zstd

operating system

Windows 10 21H1

@jeffrson jeffrson changed the title curl tries to read known_hosts from wrong location (Windows) curl tries to read known_hosts from wrong location Nov 19, 2021
@bagder
Copy link
Member

bagder commented Nov 19, 2021

it should look at %USERPROFILE%.ssh

Is this practice/order documented or mandated somewhere official? curl has preferred APPDATA to USERPROFILE for finding the user's "home dir" since 8be602c, 18 years ago...

@jeffrson
Copy link
Author

jeffrson commented Nov 19, 2021

Well, that's interesting.

I'm just using curl with SFTP for the first time and I stumbled across the certificate warning. So I was going to research.

So what does it mean "official"?
Some tools want to follow XDG base directories, other tools try to mimic Linux behaviour by using %USERPROFILE% as $HOME (that's what sounds reasonable to me). Yet another tools rely on explizit %HOME%.

Also, there hasn't been "official" SSH until a few years ago in some Windows 10 update. Now, this ssh is using %USERPROFILE%\.ssh and I think it's practical to use the same data in curl as well, isn't it? It's not really useful to have two different known_hosts.

BTW, IMO it's not too well documented that curl is using %APPDATA%. I actually had used Procmon to check whether it reads my known_hosts file.

Do you know of other apps search .ssh at %APPDATA%?

Maybe you could read %USERPROFILE%\.ssh, if the other does not exist? Or if it doesn't contain known_hosts?

@bagder
Copy link
Member

bagder commented Nov 19, 2021

curl also never actually use %USERPROFILE% now, it always uses %USERPROFILE%\Application Data - if APPDATA isn't set.

@bagder bagder self-assigned this Nov 19, 2021
bagder added a commit that referenced this issue Nov 19, 2021
The homedir() function now iterates over all the environment variables
trying to access the file in question until it finds it. Last resort is
then getpwuid() if available. Previously it would first try to find a
home directory and if that was set, insist on checking only that
directory for the file.

The Windows specific checks are now done differently too and in this
order:

1 - %USERPROFILE%
2 - %APPDATA%
3 - %USERPROFILE%\\Application Data\\

The windows order is modified to match how the Windows 10 ssh tool works
when it searches for .ssh/known_hosts.

Reported-by: jeffrson on github
Fixes #8033
@bagder
Copy link
Member

bagder commented Nov 19, 2021

@jeffrson, Would you be able to try out #8035 ?

bagder added a commit that referenced this issue Nov 19, 2021
The homedir() function now iterates over all the environment variables
trying to access the file in question until it finds it. Last resort is
then getpwuid() if available. Previously it would first try to find a
home directory and if that was set, insist on checking only that
directory for the file.

The Windows specific checks are now done differently too and in this
order:

1 - %USERPROFILE%
2 - %APPDATA%
3 - %USERPROFILE%\\Application Data\\

The windows order is modified to match how the Windows 10 ssh tool works
when it searches for .ssh/known_hosts.

Reported-by: jeffrson on github
Fixes #8033
bagder added a commit that referenced this issue Nov 19, 2021
The homedir() function now iterates over all the environment variables
trying to access the file in question until it finds it. Last resort is
then getpwuid() if available. Previously it would first try to find a
home directory and if that was set, insist on checking only that
directory for the file.

The Windows specific checks are now done differently too and in this
order:

1 - %USERPROFILE%
2 - %APPDATA%
3 - %USERPROFILE%\\Application Data\\

The windows order is modified to match how the Windows 10 ssh tool works
when it searches for .ssh/known_hosts.

Reported-by: jeffrson on github
Fixes #8033
bagder added a commit that referenced this issue Nov 20, 2021
The homedir() function is now renamed into findfile() and iterates over
all the environment variables trying to access the file in question
until it finds it. Last resort is then getpwuid() if
available. Previously it would first try to find a home directory and if
that was set, insist on checking only that directory for the file. This
now returns the full file name it finds.

The Windows specific checks are now done differently too and in this
order:

1 - %USERPROFILE%
2 - %APPDATA%
3 - %USERPROFILE%\\Application Data\\

The windows order is modified to match how the Windows 10 ssh tool works
when it searches for .ssh/known_hosts.

Reported-by: jeffrson on github
Fixes #8033
Closes #8035
@jeffrson
Copy link
Author

jeffrson commented Nov 21, 2021

Great work, thank you very much!

Unfortunately I don't have any local builds for curl :-( and I'm afraid I can't set it up in the short term either.

Also I could not find a nightly or autobuild version - can you recommend one?

bagder added a commit that referenced this issue Nov 22, 2021
The homedir() function is now renamed into findfile() and iterates over
all the environment variables trying to access the file in question
until it finds it. Last resort is then getpwuid() if
available. Previously it would first try to find a home directory and if
that was set, insist on checking only that directory for the file. This
now returns the full file name it finds.

The Windows specific checks are now done differently too and in this
order:

1 - %USERPROFILE%
2 - %APPDATA%
3 - %USERPROFILE%\\Application Data\\

The windows order is modified to match how the Windows 10 ssh tool works
when it searches for .ssh/known_hosts.

Reported-by: jeffrson on github
Fixes #8033
Closes #8035
@bagder bagder closed this as completed in 2be1aa6 Nov 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants