Install: Rclone

Rclone (by Nick Craig-Wood) is "rsync for the cloud". Basically, it is used to transfer data to or from a variety of supported cloud storage providers (eg Google Drive).

Rclone is used by Cloudplow and Backup to upload media and backup Cloudbox, respectively.

The guide below assumes you are using Google Drive.

If you would like to set this up with another cloud storage provider, but have not setup Rclone yet, then follow the guide below and replace all mentions of "Google Drive" with your preferred cloud storage provider's name. [Note that this does not mean that any arbitrary rclone backend will actually work in this context]

If you already have Rclone configured, you can jump directly to the relevant section.

New Rclone Setup

This guide goes through configuring rclone to connect to a Google Drive "My Drive". You can substitute a Shared Drive ["Shared Drives" and "Team Drives" are the same thing] if you are familiar with that process. Currently, that isn't documented here.

Before you start, you should install the current version of rclone on your local computer. You will need it to authenticate the rclone remote due to changes on the Google side.

1. Run the following command:

   rclone config
   

2. Type n for "New remote" and press Enter.

   rclone config
   Current remotes:
   
   No remotes found make a new one
   n) New remote
   s) Set configuration password
   q) Quit config
   n/s/q> n
   

3. For "name", type in google and press Enter.

   

   n/s/q> n
   name> foobar
   

4. For "Type of storage", type in drive, or the corresponding number, and press Enter. Note that this list is constantly changing, will be much longer, and the numbers won't match what's shown here. Read what's on the screen.

   name> foobar
   Option Storage.
   Type of storage to configure.
   Choose a number from below, or type in your own value.
    1 / 1Fichier
      \ (fichier)
   ...
   16 / Google Cloud Storage (this is not Google Drive)
      \ (google cloud storage)
   17 / Google Drive
      \ (drive)
   ...
   46 / seafile
      \ (seafile)
   Storage> drive
   

5. Create a Google Drive API Client ID and Client Secret for Rclone.

   (If you know what you're doing and want to use a service account file to authenticate, you can skip creating the ID and Secret and leave the next two items blank)

6. For "Google Application Client ID", paste the Client ID from Step #5, and press Enter.

   

   Storage> drive
   Option client_id.
   Google Application Client Id
   Setting your own is recommended.
   See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
   If you leave this blank, it will use an internal key which is low performance.
   Enter a value. Press Enter to leave empty.
   client_id> BINGBANGBOING
   

7. For "Google Application Client Secret", paste the Client Secret from Step #5, and press Enter.

   client_id> BINGBANGBOING
   Option client_secret.
   OAuth Client Secret.
   Leave blank normally.
   Enter a value. Press Enter to leave empty.
   client_secret> FOOBARBAZ
   

8. For the "Scope that rclone should use when requesting access from drive", type in drive, or the corresponding number (i.e. 1), to select "Full access all files, excluding Application Data Folder", and press Enter.

   

   Option scope.
   Scope that rclone should use when requesting access from drive.
   Choose a number from below, or type in your own value.
   Press Enter to leave empty.
    1 / Full access all files, excluding Application Data Folder.
      \ (drive)
    2 / Read-only access to file metadata and file contents.
      \ (drive.readonly)
      / Access to files created by rclone only.
    3 | These are visible in the drive website.
      | File authorization is revoked when the user deauthorizes the app.
      \ (drive.file)
      / Allows read and write access to the Application Data folder.
    4 | This is not visible in the drive website.
      \ (drive.appfolder)
      / Allows read-only access to file metadata but
    5 | does not allow any access to read or download file content.
      \ (drive.metadata.readonly)
   scope> 1
   

9. For "ID of the root folder", leave blank and press Enter.

   

   Option root_folder_id.
   ID of the root folder.
   Leave blank normally.
   Fill in to access "Computers" folders (see docs), or for rclone to use
   a non root folder as its starting point.
   Enter a value. Press Enter to leave empty.
   root_folder_id>
   

10. For "Service Account Credentials JSON file path", leave blank and press Enter.

    (If you know what you're doing and want to use a service account file to authenticate, enter the path to the service account JSON file here)

    

    Option service_account_file.
    Service Account Credentials JSON file path.
    Leave blank normally.
    Needed only if you want use SA instead of interactive login.
    Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
    Enter a value. Press Enter to leave empty.
    service_account_file>
    

11. For "Edit advanced config", type n and press Enter.

    

12. For "Use auto config?", type n for "...remote or headless machine" and press Enter.

    

13. In the next section, follow the instructions as shown.

    For this to work, you will need rclone available on a machine that has  a web browser available.
    For more help and alternate methods see: https://rclone.org/remote_setup/
    Execute the following on the machine with the web browser (same rclone version recommended):
        rclone authorize "drive" "xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    Then paste the result.
    Enter a value.
    config_token>
    

14. If asked to login, use the Google Drive account you want to store your data in (see Prerequisites).

    

15. Give access by clicking "Allow".

    

16. The browser should report success.

    

17. And a token should show up in the terminal on your local computer:

    2022/05/09 22:56:09 NOTICE: Log in and authorize rclone for access
    2022/05/09 22:56:09 NOTICE: Waiting for code...
    2022/05/09 22:56:16 NOTICE: Got code
    Paste the following into your remote machine --->
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    ...
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    <---End paste
    

18. Paste the token at the rclone prompt and press Enter.

    Enter a value.
    config_token>  PASTE_YOUR_TOKEN_HERE
    

19. For "Configure this as a team drive?", type n and press Enter.

    NOTE: This guide is documenting the standard case, which does not use Shared Drives. If you are familiar with setting up Shared Drives, you can do so here.

    

20. To confirm that the remote details look OK, type y and press Enter.

    

21. To exit, type q and press Enter.

    

Existing Rclone Setup

The default remote specified in settings.yml is google for Google Drive. If the Rclone remote in your config has the same name, then you are OK to skip this page and go on to the next.

If you are using Google Drive and the Rclone remote in your config has a different name, then you will need to either:

• Rename your current Rclone remote to the default one (i.e. google). Instructions for this are below.

  Or

• Edit the Rclone remote entry in settings.yml with yours.

If you prefer to use another cloud storage provider, you can add the name of the Rclone remote in to settings.yml.

Rename Existing Rclone Remote

To rename the Google Drive remote to google:

1. Find and edit your Rclone configuration file.

   nano $(rclone config file | tail -n 1)
   

2. Rename the Google Drive drive remote (name between the brackets) to google.

3. It will now look like this:

   [google]
   type = drive
   client_id = 1234567890123-mjffsmxvendscftuvnyngkhegapovgnv.apps.googleusercontent.com
   client_secret = klflzftkrwuwuedesxzewsfz
   token = {"access_token":"ya30.gelftvrymioiilvdtfegfvhfgallrhocewjckdnnvmxdjpjzbdhkmgulvqhgbafkdtpottzthhnyzysxwlpf-38ikRIxZvimyoxyKdse$
   

4. Save the file and exit: Ctrl + X Y Enter.

5. Copy the config file to ~/.config/rclone/rclone.conf (if it isn't there already):

   cp -n $(rclone config file | tail -n 1) ~/.config/rclone/rclone.conf
   

6. Give it the proper ownership and permissions. Replace user and group to match yours' (see here):

   sudo chown user:group ~/.config/rclone/rclone.conf
   sudo chmod 755 ~/.config/rclone/rclone.conf