Support reading remote files via ssh, sftp, and any other network drives

[alanhamlett]

No description provided.

[gandarez]

@alanhamlett is that a feature already implemented in python?

[alanhamlett]

Partially with https://github.com/wakatime/wakatime/issues/72 but we didn't support ssh:// nor sftp:// yet.

[gandarez]

Ok thanks. Taking this 👍

[tshu-w]

vote for this

[alanhamlett]

Thank you @gandarez for implementing remote file support with #623!!1!

[tshu-w]

Hi, are there any instructions on the format of the remote files entity, starting with ssh:// or sftp://? Where to write the hostname? I thought I can make a PR for https://github.com/wakatime/wakatime-mode/blob/master/wakatime-mode.el#L125

[tshu-w]

@gandarez @alanhamlett Hi, I try with this format ssh://hostname/path/to/file from this PR and it seems like wakatime doesn't read ssh config which complain about host name and cannot use ssh key.

[alanhamlett]

Can you normally ssh to that host without entering a password? You might need to add the authorized ssh key to your ~/.ssh/ folder. I think the default is id_rsa.

[tshu-w]

Can you normally ssh to that host without entering a password?

Yes.

$ killall ssh
$ ssh -p PORT IP ls /home/wangtianshu/test_project/test.py
/home/wangtianshu/test_project/test.py
$ wakatime-cli --entity ssh://IP:PORT/home/wangtianshu/test_project/test.py --plugin emacs-wakatime/1.0.2

.wakatime.log:

{"caller":"github.com/wakatime/wakatime-cli/pkg/remote/remote.go:188","func":"Connect","level":"warning","message":"no host key found for IP. It will try to make an insecure connection","now":"2022-02-22T17:17:27+08:00","version":"v1.37.0"}
{"caller":"github.com/wakatime/wakatime-cli/pkg/remote/remote.go:88","func":"1","level":"error","message":"failed to download file to temporary folder: failed to connect to sftp host: failed to connecto to 'IP:PORT': ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain","now":"2022-02-22T17:17:27+08:00","version":"v1.37.0"}
{"caller":"github.com/wakatime/wakatime-cli/pkg/language/chroma.go:128","func":"selectByCustomizedPriority","level":"warning","message":"failed to load folder extensions: failed to read directory: open ssh://IP:PORT/home/wangtianshu/test_project/: no such file or directory","now":"2022-02-22T17:17:27+08:00","version":"v1.37.0"}

and it seems like wakatime doesn't obey .ssh/config:

HOST hostalias
  Hostname IP
  User wangtianshu
  Port PORT

and run wakatime-cli --entity ssh://hostalias/home/wangtianshu/test_project/test.py --plugin emacs-wakatime/1.0.2 got:

{"caller":"github.com/wakatime/wakatime-cli/pkg/remote/remote.go:188","func":"Connect","level":"warning","message":"no host key found for hostalias. It will try to make an insecure connection","now":"2022-02-22T17:20:03+08:00","version":"v1.37.0"}
{"caller":"github.com/wakatime/wakatime-cli/pkg/remote/remote.go:88","func":"1","level":"error","message":"failed to download file to temporary folder: failed to connect to sftp host: failed to connecto to 'hostalias:22': dial tcp: lookup jumpbox: no such host","now":"2022-02-22T17:20:03+08:00","version":"v1.37.0"}
{"caller":"github.com/wakatime/wakatime-cli/pkg/language/chroma.go:128","func":"selectByCustomizedPriority","level":"warning","message":"failed to load folder extensions: failed to read directory: open ssh://hostalias/home/wangtianshu/test_project/: no such file or directory","now":"2022-02-22T17:20:03+08:00","version":"v1.37.0"}

[gandarez]

It tries to get ssh public key from ~/.ssh/knwon_hosts. Could you please double check if you have added it before and should be there?

[tshu-w]

are you sure it is ~/.ssh/knwon_hosts?

The known_hosts File is a client file containing all remotely connected known hosts

[gandarez]

are you sure it is ~/.ssh/knwon_hosts?

The known_hosts File is a client file containing all remotely connected known hosts

yep, there's.

[tshu-w]

I also get error with username and password provide:

wakatime-cli --entity ssh://USER:PASS@IP:PORT/home/wangtianshu/test_project/test.py

[tshu-w]

are you sure it is ~/.ssh/knwon_hosts?

The known_hosts File is a client file containing all remotely connected known hosts

yep, there's.

I still got no host key found for IP even if I ssh user@ip -p port and press yes to add host key to ~/.ssh/knwon_hosts

I'm quite sure because the content in known_hosts seems be hashed but I think it's added as trashed and ssh, a new line was added.

[gandarez]

Could you please check if there's a line in this format IP <algorithm> <hash>?

example:

192.168.1.200 ssh-ed25519 <hash>

[tshu-w]

@gandarez Sorry for not being clear. I open a new issue with minimal setup #658