Install: Rclone

Rclone (by Nick Craig-Wood) is "rsync for the cloud". Basically, it is used to transfer data to or from a variety of supported cloud storage providers (eg Google Drive).

Rclone is used by Cloudplow and Backup to upload media and backup Cloudbox, respectively.

The guide below assumes you are using Google Drive.

If you would like to set this up with another cloud storage provider, but have not setup Rclone yet, then follow the guide below and replace all mentions of "Google Drive" with your preferred cloud storage provider's name. [Note that this does not mean that any arbitrary rclone backend will actually work in this context]

If you already have Rclone configured, you can jump directly to the relevant section.

New Rclone Setup

This guide goes through configuring rclone to connect to a Google Drive "My Drive". You can substitute a Shared Drive ["Shared Drives" and "Team Drives" are the same thing] if you are familiar with that process. Currently, that isn't documented here.

Before you start, you should install the current version of rclone on your local computer. You will need it to authenticate the rclone remote due to changes on the Google side.

1. Run the following command:

   rclone config
   

2. Type n for "New remote" and press Enter.

   

3. For "name", type in google and press Enter.

   

4. For "Type of storage", type in drive, or the corresponding number, and press Enter. Note that this list is constantly changing, will be much longer, and the numbers won't match what's shown here. Read what's on the screen.

   

5. Create a Google Drive API Client ID and Client Secret for Rclone.

   (If you know what you're doing and want to use a service account file to authenticate, you can skip creating the ID and Secret and leave the next two items blank)

6. For "Google Application Client ID", paste the Client ID from Step #5, and press Enter.

   

7. For "Google Application Client Secret", paste the Client Secret from Step #5, and press Enter.

   

8. For the "Scope that rclone should use when requesting access from drive", type in drive, or the corresponding number (i.e. 1), to select "Full access all files, excluding Application Data Folder", and press Enter.

   

9. For "ID of the root folder", leave blank and press Enter.

   

10. For "Service Account Credentials JSON file path", leave blank and press Enter.

    (If you know what you're doing and want to use a service account file to authenticate, enter the path to the service account JSON file here)

    

11. For "Edit advanced config", type n and press Enter.

    

12. For "Use auto config?", type n for "...remote or headless machine" and press Enter.

    

13. In the next section, follow the instructions as shown.

    For this to work, you will need rclone available on a machine that has  a web browser available.
    For more help and alternate methods see: https://rclone.org/remote_setup/
    Execute the following on the machine with the web browser (same rclone version recommended):
        rclone authorize "drive" "xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    Then paste the result.
    Enter a value.
    config_token>
    

14. If asked to login, use the Google Drive account you want to store your data in (see Prerequisites).

    

15. Give access by clicking "Allow".

    

16. You will now be given a token. Copy it.

    

17. Paste the token at the rclone prompt and press Enter.

    Enter a value.
    config_token>  PASTE_YOUR_TOKEN_HERE
    

18. For "Configure this as a team drive?", type n and press Enter.

    NOTE: This guide is documenting the standard case, which does not use Shared Drives. If you are familiar with setting up Shared Drives, you can do so here.

    

19. To confirm that the remote details look OK, type y and press Enter.

    

20. To exit, type q and press Enter.

    

Existing Rclone Setup

The default remote specified in settings.yml is google for Google Drive. If the Rclone remote in your config has the same name, then you are OK to skip this page and go on to the next.

If you are using Google Drive and the Rclone remote in your config has a different name, then you will need to either:

• Rename your current Rclone remote to the default one (i.e. google). Instructions for this are below.

  Or

• Edit the Rclone remote entry in settings.yml with yours.

If you prefer to use another cloud storage provider, you can add the name of the Rclone remote in to settings.yml.

Rename Existing Rclone Remote

To rename the Google Drive remote to google:

1. Find and edit your Rclone configuration file.

   nano $(rclone config file | tail -n 1)
   

2. Rename the Google Drive drive remote (name between the brackets) to google.

3. It will now look like this:

   [google]
   type = drive
   client_id = 1234567890123-mjffsmxvendscftuvnyngkhegapovgnv.apps.googleusercontent.com
   client_secret = klflzftkrwuwuedesxzewsfz
   token = {"access_token":"ya30.gelftvrymioiilvdtfegfvhfgallrhocewjckdnnvmxdjpjzbdhkmgulvqhgbafkdtpottzthhnyzysxwlpf-38ikRIxZvimyoxyKdse$
   

4. Save the file and exit: Ctrl + X Y Enter.

5. Copy the config file to ~/.config/rclone/rclone.conf (if it isn't there already):

   cp -n $(rclone config file | tail -n 1) ~/.config/rclone/rclone.conf
   

6. Give it the proper ownership and permissions. Replace user and group to match yours' (see here):

   sudo chown user:group ~/.config/rclone/rclone.conf
   sudo chmod 755 ~/.config/rclone/rclone.conf