copy cmd. not recognising password2

[chymian]

version 16.01.

after manually initialization of repo 2 with the --copy-chunker-params, the following copy cmd of a repo does not work:

resticprofile --name=home copy 
2022/03/04 09:25:03 profile 'home': initializing repository (if not existing)
2022/03/04 09:25:03 starting command: /usr/bin/restic init --option rclone.program=/usr/bin/rclone --option rclone.args=serve\ restic\ --stdio\ --transfers\ 10\ --verbose --password-file key --repo /dpool/backup/restic/sagar/home   
2022/03/04 09:25:03 profile 'home': starting 'copy'
2022/03/04 09:25:03 starting command: /usr/bin/restic copy --option rclone.program=/usr/bin/rclone --option rclone.args=serve\ restic\ --stdio\ --transfers\ 10\ --verbose --password-file key --password-file2 key --repo /dpool/backup/restic/sagar/home --repo2 rclone:s3:/sagar/restic//home                                                            
repository 92c570cd opened successfully, password is correct
rclone: 2022/03/04 09:25:10 NOTICE: S3 bucket sagar path restic//home: Odd name received "restic/home/keys/8457008b6c6e88ad02793634a8a0c3b6afc5f9c94324e75d72a370ae0a82b1f1"
Fatal: wrong password or no key found
2022/03/04 09:25:10 starting 'run-after-fail' profile command 1/1
2022/03/04 09:25:10 copy on profile 'home': exit status 1

Manually resorting the parameters to the following works:

$ restic  --option rclone.program=/usr/bin/rclone --option rclone.args="serve restic --stdio --transfers 10 --verbose" --password-file key --repo /dpool/backup/restic/sagar/home copy --repo2 rclone:s3:/sagar/restic/home  --password-file2 key
repository 92c570cd opened successfully, password is correct
repository 79e3bbb9 opened successfully, password is correct
…

another question

is there a way to automatically let resticprofile initialize the copy target-repo with the --copy-chunkers-params flag?

[creativeprojects]

after manually initialization of repo 2 with the --copy-chunker-params, the following copy cmd of a repo does not work:

I can't replicate it with my setup: I'll try with rclone as it looks like it could be the position of the options that makes it fail.

is there a way to automatically let resticprofile initialize the copy target-repo with the --copy-chunkers-params flag?

I didn't know about that feature, I just read it (I'm not using the copy command myself).
I'll update resticprofile to accommodate these new features from restic 0.12

[chymian]

While you are there, I realized that when a repo2/pw2 is defined, it's is given to the cmd-line on other cmds, like backup, snapshots, … which can not process it.
that means, one can only define these params at exactly the profile-section one need it, but not in a general-variable section like that:

[repos]
inherit = "base"
repository = "{{ $repo_local }}/{{ .Profile.Name }}"
password-file = "key"
repository2 = "{{ $repo_s3 }}/{{ .Profile.Name }}"
password-file2 = "key"
option = [
    "rclone.program=/usr/bin/rclone",
    "rclone.args=serve restic --stdio --transfers 10 --verbose"
]

resticprofile --name=root --trace snapshots
2022/03/05 07:35:33 profile 'root': initializing repository (if not existing)
2022/03/05 07:35:33 starting command: /usr/bin/restic init --option rclone.program=/usr/bin/rclone --option rclone.args=serve\ restic\ --stdio\ --transfers\ 10\ --verbose --password-file key --password-file2 key --repo /dpool/backup/restic/sagar/root --repository2 rclone:s3:/sagar/restic//root
2022/03/05 07:35:33 profile 'root': starting 'snapshots'
2022/03/05 07:35:33 starting command: /usr/bin/restic snapshots --option rclone.program=/usr/bin/rclone --option rclone.args=serve\ restic\ --stdio\ --transfers\ 10\ --verbose --password-file key --password-file2 key --repo /dpool/backup/restic/sagar/root --repository2 rclone:s3:/sagar/restic//root
unknown flag: --password-file2
2022/03/05 07:35:33 starting 'run-after-fail' profile command 1/1
Unit resticprofile-snapshots@profile-root.service could not be found.
2022/03/05 07:35:33 snapshots on profile 'root': exit status 1

[chymian]

at the moment it is only possible to have one inherit statement per section. is there a reason for that? IMHO it would give the modularity of the config a greater flexilibity, if one could use more of them.

[chymian]

I have another example for you. it seems to be the format of the string, in which rclone.args are passed:

2022/03/05 08:38:27 profile 's3-root': starting 'snapshots'
2022/03/05 08:38:27 starting command: /usr/bin/restic snapshots --option rclone.program=/usr/bin/rclone --option rclone.args=serve\ restic\ --stdio\ --transfers\ 10\ --verbose --password-file key --repo rclone:s3:/sagar/restic//root
rclone: 2022/03/05 08:38:33 NOTICE: S3 bucket sagar path restic//root: Odd name received "restic/root/keys/89f5f189a135c05f511e8c6f9854f54372c2dbba978864961fb3065e9275f39b"
Fatal: wrong password or no key found
2022/03/05 08:38:33 starting 'run-after-fail' profile command 1/1
Unit resticprofile-snapshots@profile-s3-root.service could not be found.
2022/03/05 08:38:33 snapshots on profile 's3-root': exit status 1

manually reformating the rclone.args to:

$ restic snapshots --option rclone.program=/usr/bin/rclone --option rclone.args="serve restic --stdio --transfers 10 --verbose" --password-file key --repo rclone:s3:/sagar/restic/root
repository 0adaccb7 opened successfully, password is correct
ID        Time                 Host        Tags        Paths
----------------------------------------------------------------
610a1ae4  2022-02-28 20:31:48  sagar       root        /
…

[creativeprojects]

While you are there, I realized that when a repo2/pw2 is defined, it's is given to the cmd-line on other cmds, like backup, snapshots, … which can not process it.

Yes, this is by design. You can add whatever flag you want at any level. If it's not handled by resticprofile it's going to be passed to the restic command line as is, even if it's not a real restic flag.

With that system I don't need to keep track of every single parameter accepted by restic. In theory I could load the restic package and load all the flags from there but I haven't come to that yet.

[creativeprojects]

is there a way to automatically let resticprofile initialize the copy target-repo with the --copy-chunkers-params flag?

Sorry it took me some time to finalise it, but it now works. The flag is called initialize-copy-chunker-params (prefixed with initialise because it's not a flag of the copy command)

It will be available in version 0.18.0, or right now if compiling and installing from source.

Here's an example of the configuration file:

profile:
    initialize: false
    repository: "/backup/original"
    password-file: key
    copy:
        initialize: true
        initialize-copy-chunker-params: true
        repository: "/backup/copy"
        password-file: other_key

and the full documentation, when built.