Feature request - make it possible to include another config file in rclone.conf

[tddschn]

What is your current rclone version (output from rclone version)?

rclone v1.51.0
- os/arch: darwin/amd64
- go version: go1.13.7

What problem are you are trying to solve?

I want to share part of the config file between my 2 machines, and I guess it would be really helpful if I could just add this line include path/to/shared/config in rclone.conf.

[dragetd]

This sounds like a nice idea.

Tho there are different ways of approaching this.

1. A new 'include' directive in the config. Which might complicate the parsing and not play along nicely with the config format, I am not sure.
   On the other hand, this would allow some really powerful and dynamic setups.

2. Fixed code that accepts a directory and parses all .conf files inside. The default lookup for the config could then include no only rclone/rclone.conf but also rclone/rclone.conf.d/*.
   One could point --rclone-config on the commandline to a directory.

3. rclone could accept multiple --rclone-config parameters, joining the configs together. This would work for your usecase, if you invoked it accordingly each time.

I think I would prefer option 1.

[ncw]

Of those I'd probably prefer option 1 too.

The rclone config is in ini style.

Is there a standard for includes in ini style config? I didn't see one in the wikipedia page about ini files

This isn't straight forward alas, remember rclone writes tokens etc back to the config file so it would have to have the concept of multiple config files. Or maybe it should only write back to the last config file and merge the keys from the other config files (this is a request I've had before). Giving rclone the concept of multiple config files would make all 3 options reasonably easy.

[ivandeex]

@dragetd @ncw
Implementing include directive involves a method to merge two ini files. What would be the result if included file contained sections, esp. overlapping with including parent? Who wins for each section? Options:

1. Included file
2. The file that includes
3. The First section in the order of parsing (after substituting the directive by the text)
4. The Last

[tddschn]

I guess option 3 or 4 would make more sence?

[ivandeex]

@onedr0p
Probably the include directive could allow either file path or `https://hostorip/path1 URL as a parameter. Would it suite your case #3655 (comment) ? (onedr0p/home-ops#59).

However I'm biased against URLs in config. It will make startup long and fragile. Detecting inclusion loops becomes a headache. Let alone security implications.

Just including an external file can solve your case. You will run a helper that templates out a snippet before running rclone and add include /path/to/snippets.d/*.conf in ~/.rclone.conf or ~/.config/rclone/rclone.conf.

@ncw ?

[ivandeex]

Idea: the include directive syntax could be extended to allow for patterns, like include /path/to/snippets.d/* or include /etc/rclone.d/*.conf.

The behavior can be tuned too: if a single file is absent then abort, but if pattern returns empty set then continue silently.

To avoid injections, rclone must check permissions and ownership of included files, especially via pattern!

[ivandeex]

Another syntax idea stolen from sshd: replace ~/ prefix in the parameter by the environment HOME, XDG_HOME_DIR or whatever they use on Windows.

[raphaelcohn]

I'd really appreciate this; personally, I don't have the need for includes, but I do have the need to split config into that which is checked into source control and that which is not; it's also a common need to be able to provide different Linux perms to different files (eg a backup admins groups vs a local admins group, where backup admins are a superset group). I need to be able to accomplish two scenarios:-

• Partly define a backend, so the first config file defines most settings and the latter just adds sensitive or scenario specific details
• Define a backend in one file and another backend in another file

I wrapper rclone with a shell wrapper in most setups, so the ability to pass multiple --rclone-config flags would make this a snap. This would also be very much in the spirit of Debian's run-parts approach to splitting up config, allowing multiple packages to install separate config files (or even the user).

Include syntax is a way into madness, with checks for closed loops, recursion depth, how to manage relative paths (relative to file's parent folder, relative to pwd, relative to first config file's parent folder), etc. It's doable, of course, and it's powerful, but most scenarios can be simply solved by other means.

If we're going down the includes root, I would prefer something simpler than INI - it's not particularly friendly to generation and manipulation using standard POSIX tooling. I suspect that ship's sailed, though.

[rileyrg]

Just a +1. I would like to github part of my rclone.conf but not all - it would include the non githubbed "sensitive" eg gdrive section with secrets.