Confirmed: explicit support for rclone is in production at rsync.net #3254
Comments
:-) Would you like rsync.net to be listed as a supported cloud provider on https://rclone.org/ ? This could link to the docs page for sftp or for a paragraph on how to set up rclone with rsync.net if you want. Would you consider a reciprocal link back to rclone.org? (Not a requirement for the above!)
How do you run rclone on an rsync.net account? Do you ssh in and run it, or run it via an ssh exec? Or via some kind of web user interface? I would say that for the business of transferring files about then I'll note that you should probably disable the |
Yes, thank you - that would be great - could you simply add "rsync.net" to your alphabetical list of providers on the rclone/github pages ? Would you like us to write some examples (or anything at all) for that page ? We are polishing an rclone-specific signup page on our end and it will certainly have prominent links back to the main project and github pages, etc. One very helpful hint that you might add to the SFTP docs page is changing this line: "Note that some SFTP servers will need the leading / - Synology is a good example of this." to: "Note that some SFTP servers will need the leading / - Synology is a good example of this. rsync.net, on the other hand, requires users to OMIT the leading /.
No web interface - our storage arrays run nothing but OpenSSH and have no ports open other than 22. So we simply created a rclone binary executable for FreeBSD and placed it into our (very stripped, secured) environment - you just run it over SSH without logging on interactively. Do you not have a free test account ? I think you should have one - please email info@rsync.net so we can give you one.
I believe this is already the case but thank you for emphasizing that. |
|
OK, here is the rclone-specific signup page at rsync.net: https://www.rsync.net/products/rclone.html and it does indeed have links back to both the rclone main site and the github repo. |
What I normally do is have a link to your main website plus a link to the docs on the relevant backend page. This can either go to the main page or to a paragraph of text like this one in the webdav backend. So if you've got some preferred text for that, that would be super! Some hints on how to set rclone up with sftp would be useful. If rsync.net has a set of fixed endpoints, these could appear in the examples for ease of use.
:-)
OK I've done that.
Oh I see, very nice!
I have done so.
When I get the free test account I'll do a quick security review for you (on the things built into rclone). How much effort was it patching rclone to remove the features you don't want? It would be possible to make build tags to make that easier so you could do |
Understood. For the link to the main website, use this rclone link: https://rsync.net/products/rclone.html For now, since we don't have anything written to hand to you for the rclone site itself, could the "doc" icon just point to the SFTP "doc page" that already exists ? It makes sense because it is a stock, standard SFTP implementation ... later we can make an actual rsync.net page with examples, etc., for your rclone.org page.
We use a stock, standard rclone binary - nothing custom about it. However we have a very special, custom built shell that has a commands whitelist in it - and those particular rclone --arguments are not in the whitelist, so they just fail. Custom built tags not necessary, but thank you :) |
Sure that is fine.
Ah ha! Do you restrict environment variables too? you can configure rclone with environment variables too... |
Yes. An rsync.net account is not a general purpose shell (although it appears to be, and you can run many different UNIX commands over ssh[1]) and the restricted shell we provide has env vars locked down. [1] https://www.rsync.net/resources/howto/remote_commands.html |
|
I think we can mark this as closed - I am announcing it more broadly here: https://twitter.com/rsyncnet/status/1143567793801113602 ... and would appreciate it if you could retweet. Other than that, thank you for your help and guidance - I really am excited that we have this in place and we are actively developing "recipes" for doing useful things with rclone inside an rsync.net cloud storage account (hence my question about gmail export in your forum...) Cheers! |
|
I've retweeted that :-) Thank you for the test account. I'm working on a patch to rclone so it supports the Also I'll get |
rsync.net uses the freebsd equivalent of sha1sum and md5sum so adapt to that.
rsync.net uses the freebsd equivalent of sha1sum and md5sum so adapt to that.
Before this change, rclone serve sftp operating with a new rclone after the md5sum/sha1sum detection was reworked to just run a plain `md5sum`/`sha1sum` command in 3ea8203 sftp: support md5/sha1 with rsync.net #3254 Failed to signal to the remote that md5sum/sha1sum wasn't supported as in 71e172a serve/sftp: support empty "md5sum" and "sha1sum" commands We unconditionally return good hashes even if the remote being served doesn't support the hash type in question. This fix checks the hash type is supported and returns an error MD5 hash not supported When the backend is first contacted this will cause the sftp backend to detect that the hash type isn't available. Unfortunately this may have cached the wrong state so editing or remaking the config may be necessary to fix it.
Before this change, rclone serve sftp operating with a new rclone after the md5sum/sha1sum detection was reworked to just run a plain `md5sum`/`sha1sum` command in 3ea8203 sftp: support md5/sha1 with rsync.net #3254 Failed to signal to the remote that md5sum/sha1sum wasn't supported as in 71e172a serve/sftp: support empty "md5sum" and "sha1sum" commands We unconditionally return good hashes even if the remote being served doesn't support the hash type in question. This fix checks the hash type is supported and returns an error MD5 hash not supported When the backend is first contacted this will cause the sftp backend to detect that the hash type isn't available. Unfortunately this may have cached the wrong state so editing or remaking the config may be necessary to fix it.
Before this change, rclone serve sftp operating with a new rclone after the md5sum/sha1sum detection was reworked to just run a plain `md5sum`/`sha1sum` command in 3ea8203 sftp: support md5/sha1 with rsync.net #3254 Failed to signal to the remote that md5sum/sha1sum wasn't supported as in 71e172a serve/sftp: support empty "md5sum" and "sha1sum" commands We unconditionally return good hashes even if the remote being served doesn't support the hash type in question. This fix checks the hash type is supported and returns an error MD5 hash not supported When the backend is first contacted this will cause the sftp backend to detect that the hash type isn't available. Unfortunately this may have cached the wrong state so editing or remaking the config may be necessary to fix it.
(I am referencing a note from nickcw on HN, as well as some old chatter here, in github issues...)
We (rsync.net) have deployed the rclone binary to all of our production servers - the rsync.net cloud storage platform can now be used as a target / source / etc. for rclone operations.
We operate a very stock/standard OpenSSH deployment and your transport is going to be over SSH. You have full control over your own .ssh directory, including authorized_keys, so you can lock access to 'rclone' or even a specific 'rclone --arguments'.
One caveat is that we have disallowed 'rclone serve' and 'rclone mount'. I would appreciate any comments from anyone here as to how significant those restrictions are ... 'rclone serve' is definitely out, but we're considering how we might (safely, securely) offer 'rclone mount' - it would be nice to know how common a use-case that is, or if most people are happy to 'rclone mount' from their local location.
AMA ...
The text was updated successfully, but these errors were encountered: