There's a 'download' button in the file explorer when connecting vscode to remote:

Currently to download a file from remote in an extension, a file server is needed on the remote. Making this download button an API would greatly simply such work in extensions.