Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[m3query] Allow for optional override config file. #1934

Merged
merged 3 commits into from Sep 11, 2019

Conversation

pavelnikolov
Copy link
Contributor

@pavelnikolov pavelnikolov commented Sep 9, 2019

What this PR does / why we need it:

Fixes #1921

TL;DR Some of the config is static and can be stored in git. However, there are parts of the config that are generated dynamically like, for example, the RPC remote addresses, which are generated using service-discovery in our case. This PR allows for multiple config files.

Special notes for your reviewer:

This is a draft proposal for additional configuration of m3query. This is a WIP and chances are that this will break unit tests of components. Before I invest more time in this I'd like to know if you agree with this approach. Ideally, I'd prefer allowing for multiple -f arguments, like this:

./m3query -f common.yaml -f production.yaml -f remotes.yaml

Does this PR introduce a user-facing and/or backwards incompatible change?:

NONE

- allow multiple config files

Does this PR require updating code package or user-facing documentation?:

YES

- allow override config file

@CLAassistant
Copy link

CLAassistant commented Sep 9, 2019

CLA assistant check
All committers have signed the CLA.

@pavelnikolov pavelnikolov force-pushed the allow-multiple-config-files branch 2 times, most recently from 244a146 to e8f1ffc Compare September 9, 2019 03:22
Copy link
Collaborator

@arnikola arnikola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general, a couple of nits mostly about naming of the xconfig utility

src/x/config/flag_test.go Show resolved Hide resolved
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

package config_test
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meganit: we typically don't use the _test package name convention

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was intentional because I am not using any private members of the package. If anything internal changes, the tests remain the same - only test the public API of a package. There should be very few reasons not to use _test, IMHO.

src/x/config/flag_test.go Show resolved Hide resolved
// Then it can be invoked like this:
// ./app -f file1.yaml -f file2.yaml -f valueN.yaml
// Finally, when the flags are parsed, the variable contains all the values.
type StringSlice []string
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make this something more contextual, like FlagSlice? StringSlice is pretty generic and I think introducing flag-specific context is fine since this is built to implement flag.Value

Maybe something like MultiFlagAccumulator or something to convey that it accumulates duplicate values?


import "fmt"

// StringSlice represents a slice of strings. When used as a flag variable,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Mind adding var _ flag.Value = &StringSlice{} for a little extra context?


func main() {
flag.Var(&configFiles, "f", "configuration file(s)")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about pulling all this configFiles stuff into xconfig.StringSlice, something like

xconfig.ParseConfigFilenames(flag string, description string) ([]string, bool)

That could handle the validation and make it a little easier to use? Don't mind either option much though, so up to you here 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd personally leave it like this for now.

@arnikola arnikola added the ci Triggers CI (useful for external contributors) label Sep 10, 2019
@schallert schallert removed the ci Triggers CI (useful for external contributors) label Sep 10, 2019
@arnikola arnikola added the ci Triggers CI (useful for external contributors) label Sep 10, 2019
@schallert schallert removed the ci Triggers CI (useful for external contributors) label Sep 10, 2019
@arnikola arnikola added the ci Triggers CI (useful for external contributors) label Sep 10, 2019
@schallert schallert removed the ci Triggers CI (useful for external contributors) label Sep 10, 2019
@arnikola arnikola added the ci Triggers CI (useful for external contributors) label Sep 10, 2019
@schallert schallert removed the ci Triggers CI (useful for external contributors) label Sep 10, 2019
@arnikola arnikola added the ci Triggers CI (useful for external contributors) label Sep 11, 2019
@schallert schallert removed the ci Triggers CI (useful for external contributors) label Sep 11, 2019
@arnikola arnikola merged commit db4dc03 into m3db:master Sep 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Additional m3query remotes configuration
4 participants