Skip to content

A lambda function to run `rclone sync` periodically

License

Notifications You must be signed in to change notification settings

badouralix/rclone-lambda-sync

Repository files navigation

Rclone Lambda Sync

Github Actions Lint

A lambda function to run rclone sync periodically.

Prerequisites

Install a lambda layer with rclone, such as rclone-lambda-layer.

Create a valid rclone config with two remotes source and destination. See https://rclone.org/docs/#configure for details.

rclone config create "source" ...
rclone config create "destination" ...

This creates a local file that looks like this :

$ cat ~/.config/rclone/rclone.conf
[source]
...

[destination]
...

Upload this config in the SSM parameter rclone-config, either using the console or the following one-liner :

aws ssm put-parameter --region eu-west-3 --name rclone-config --type SecureString --value file://$HOME/.config/rclone/rclone.conf --description "Entire rclone.conf for rclone-lambda"

AWS Systems Manager > Parameter Store > rclone-config > Overview

Also set DEPLOYMENT_BUCKET and EVENTS_FILE in a new .env local file.

Deploy

serverless deploy

Test

serverless invoke --function sync

Configuration

Environment Variable Description Default
RCLONE_CONFIG_SSM_NAME Name of the SSM parameter to fetch the config from. rclone-config
RCLONE_SYNC_CONTENT_DESTINATION Name of the sync destination in the format "dest:path". destination:/
RCLONE_SYNC_CONTENT_SOURCE Name of the sync source in the format "source:path". source:/
RCLONE_SYNC_DRY_RUN Do a trial run with no permanent changes. false
RCLONE_SYNC_EXTRA_FLAGS List of flags passed to rclone. See available flags in https://rclone.org/flags/. --exclude /Downloads/** --exclude /External/**

Schedules

By default, rclone-lambda-sync runs once a day around 00:00 UTC. See rclone_lambda_daily.yaml. This behavior can be customized by following these steps :

  1. Create a custom yaml file containing the desired schedules

    - schedule:
        name: rclone-lambda-daily-1
        rate: cron(0 1 * * ? *)
        input:
          RCLONE_CONFIG_SSM_NAME: rclone-config-1
    
    - schedule:
        name: rclone-lambda-daily-2
        rate: cron(0 2 * * ? *)
        input:
          RCLONE_CONFIG_SSM_NAME: rclone-config-2
  2. Change EVENTS_FILE value to the name of the custom yaml file in the .env local file

  3. Test with a slightly different command

    serverless invoke --function sync --data '{"RCLONE_CONFIG_SSM_NAME": "rclone-config-1"}'

Documentation

License

Unless explicitly stated to the contrary, all contents licensed under the MIT License.

About

A lambda function to run `rclone sync` periodically

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages