Skip to content

[WIP] Move config file handling to external packages#3582

Closed
fionera wants to merge 2 commits intorclone:masterfrom
fionera:rework-config-package
Closed

[WIP] Move config file handling to external packages#3582
fionera wants to merge 2 commits intorclone:masterfrom
fionera:rework-config-package

Conversation

@fionera
Copy link
Contributor

@fionera fionera commented Oct 1, 2019

What is the purpose of this change?

Replace the current goconfig handling with an abstraction to allow use of other config types. I only did Search and Replace for the tests, so I dont know if anything works yet. Also I dont know if this is even a good idea. I use viper for a fork I maintain to have a map[string][]string type for remotes and more flexible handling of these, so I thought maybe upstream it

Was the change discussed in an issue or in the forum before?

I think I mentioned it somewhere 😄

Checklist

  • I have read the contribution guidelines.
  • I have added tests for all changes in this PR if appropriate.
  • I have added documentation for the changes if appropriate.
  • All commit messages are in house style.
  • I'm done, this Pull Request is ready for review :-)

return s.GetStringDefault(name, "")
}

func (s *section) GetStringDefault(name string, default_ string) string {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; method parameter default_ should be default (from golint)

)

const (
RemotesPrefix = "remotes"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exported const RemotesPrefix should have comment (or a comment on this block) or be unexported (from golint)

func (c *viperConfig) GetRemotes() []string {
var remotes []string
remoteEntries := viper.GetStringMap(RemotesPrefix)
for key, _ := range remoteEntries {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should omit 2nd value from range; this loop is equivalent to for key := range ... (from golint)

return s.v.GetString(GetConfigKey(s.basePath, name))
}

func (s *section) GetStringDefault(name string, default_ string) string {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; method parameter default_ should be default (from golint)

func (s *section) GetStringDefault(name string, default_ string) string {
if s.v.IsSet(GetConfigKey(s.basePath, name)) {
return s.v.GetString(GetConfigKey(s.basePath, name))
} else {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if block ends with a return statement, so drop this else and outdent its block (from golint)

"context"
"flag"
"fmt"
"github.com/rclone/rclone/fs/config/provider/goconfig"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File is not goimports-ed (from goimports)

@ncw
Copy link
Member

ncw commented Oct 1, 2019

Replace the current goconfig handling with an abstraction to allow use of other config types. I only did Search and Replace for the tests, so I dont know if anything works yet. Also I dont know if this is even a good idea.

The config system was designed to be replaceable by setting these global variables

rclone/fs/config.go

Lines 16 to 28 in 77a520c

// Read a value from the config file
//
// This is a function pointer to decouple the config
// implementation from the fs
ConfigFileGet = func(section, key string) (string, bool) { return "", false }
// Set a value into the config file and persist it
//
// This is a function pointer to decouple the config
// implementation from the fs
ConfigFileSet = func(section, key, value string) (err error) {
return errors.New("no config file set handler")
}

Now whether that is enough configuration I'm not sure since I don't think anyone used that feature...

That doesn't work with rclone config but should be enough for library use of rclone.

I think your patch goes one step further so rclone config would work with viper - is that right?

I use viper for a fork I maintain to have a map[string][]string type for remotes and more flexible handling of these, so I thought maybe upstream it

I thought about using viper at some point but decided it would be too difficult to retro-fit. I don't really know how it works though - how are you using it?

@fionera
Copy link
Contributor Author

fionera commented Oct 1, 2019

The config system was designed to be replaceable by setting these global variables

yes I saw that but since I did not wanted to use a string as storage, it wasnt enought. I wanted to use configs like this: https://i.imgur.com/QxaJpAt.png

I think your patch goes one step further so rclone config would work with viper - is that right?

Yes I completely moved the file handling to a different package and made it switchable. The access is now over interfaces which also makes mocking far easier.

I don't really know how it works though - how are you using it?

It is basicly a wrapper around yaml, toml etc.

@ncw
Copy link
Member

ncw commented Oct 1, 2019

yes I saw that but since I did not wanted to use a string as storage, it wasn't enough.

I'm trying to understand in what way wasn't it enough? What you mean by "use a string as storage"?

@fionera
Copy link
Contributor Author

fionera commented Oct 1, 2019

I needed a map value and since I hate using CLIs I wanted to be able to still write the config manually. In the screenshot is an example of this

@fionera
Copy link
Contributor Author

fionera commented Oct 15, 2019

@ncw Would it be better to split this into multiple smaller PRs? Like moving the Config System itself into a different file, abstracting it more etc.?

@ncw
Copy link
Member

ncw commented Oct 18, 2019

@ncw Would it be better to split this into multiple smaller PRs? Like moving the Config System itself into a different file, abstracting it more etc.?

Sorry I've been a bit distracted trying to firm up the 1.50 release!

Yes smaller PRs would be much easier to review.

Maybe first step you could post a Config interface {} type which we could discuss?

@fionera
Copy link
Contributor Author

fionera commented Nov 8, 2019

Maybe first step you could post a Config interface {} type which we could discuss?

Sure thing :) Will close this and open an Issue

@fionera fionera closed this Nov 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants