New config layout #414

prarit · 2020-09-02T17:54:57Z

WIP: New config layout

codecov · 2020-09-02T19:00:10Z

Codecov Report

Merging #414 into master will decrease coverage by 1.15%.
The diff coverage is 22.35%.

@@            Coverage Diff             @@
##           master     #414      +/-   ##
==========================================
- Coverage   63.33%   62.17%   -1.16%     
==========================================
  Files          50       50              
  Lines        3060     3141      +81     
==========================================
+ Hits         1938     1953      +15     
- Misses        972     1034      +62     
- Partials      150      154       +4

Impacted Files	Coverage Δ
internal/config/config.go	`28.00% <18.51%> (-17.46%)`	⬇️
cmd/issue_show.go	`92.00% <100.00%> (ø)`
cmd/mr_show.go	`81.11% <100.00%> (ø)`
cmd/snippet_browse.go	`81.48% <0.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eae04b7...6d8163f. Read the comment docs.

main.go

prarit · 2020-09-03T00:11:14Z

I see that using viper.WriteConfig() here don't replace all options not viper.Set()'ed like it did when using HCL format. Was it also an issue with the format?

Oh, that's a bug then. If viper.WriteConfig() doesn't write out all the config then yes, I have to use viper.WriteConfigAs(). I'll double check the behaviour.

zaquestion · 2020-09-03T05:18:25Z

I don't know why codecov is commenting every 5 lines on the stuff that isn't covered by tests ... Super noisy, sorry folks I try to find out how to disable that (unless it's just me?)

zaquestion

This is looking great already, I should have time tomorrow to weigh in more and catch up on the other stuff. Thanks for taking this on, huge improvement, and I never expected we'd migrate the original configs so that's a nice bonus

zaquestion · 2020-09-03T05:23:07Z

internal/config/config.go

+		fmt.Println("Warning: Could not delete old config file", oldconfig)
+	}
+
+	// HACK


fwiw, it doesn't surprise me at all that we had to do this. Should be fine since lab configs have remained simple, essentially for this reason too.

I don't know why codecov is commenting every 5 lines on the stuff that isn't covered by tests ... Super noisy, sorry folks I try to find out how to disable that (unless it's just me?)

Hmm ... I'm by no means an expert in codecov. We do have a few engineers who are familiar with it. Let me see if I can ping someone to take a look.

bmeneg · 2020-09-03T13:17:13Z

Nicely done @prarit !
I've tested all cases I could think of in this new push you did and things seem to be working pretty well.

Switch to using TOML format for the config files as the first step of rehauling the Config setup. TOML provides better viper support and is closer to the standard git config file format. HCL format is broken in viper and requires a small hack to convert double square brackets to single square brackets in the viper TOML file format. The test code extensively uses HCL and employs several viper HCL workarounds. The test code has been modified to remove HCL code and those workarounds. Before this change the ~/.config/lab.hcl contains "core" = { "host" = "https://gitlab.com" "token" = "abcdef12345" "user" = "prarit" } After this change the ~/.config/lab/lab.toml contains [core] host = "https://gitlab.com" token = "abcdef12345" user = "prarit" Also move the new config files into their own directories in ~/.config and the local .git. Convert config files from HCL to TOML and move them into their own ~/.config/lab and .git/lab/ directories. Signed-off-by: Prarit Bhargava <prarit@redhat.com>

The existing config code uses a lot of workarounds for viper's issues with the HCL format. With the move TOML format for the config files, a lot of this code can be eliminated by using standard viper calls. Rework the config code to remove workarounds and reorganize the config function. Suggested-by: Zaq? Wiedmann <zaquestion@gmail.com> Signed-off-by: Prarit Bhargava <prarit@redaht.com>

Move the config initialization code from main.go to internal/config/config.go. Suggested-by: Bruno Meneguele <bmeneg@redhat.com> Signed-off-by: Prarit Bhargava <prarit@redhat.com>

As noted in issue zaquestion#152 ("Accept self signed certificates"), the certificate code is no longer working. This is because the certificate code was executed before the configuration was loaded. The code is need of a massive clean up, and with the transition of the config files to TOML format, the code to get the certificate can easily be incorporated into the main configuration code. Fix and clean up the certifcate code. Signed-off-by: Prarit Bhargava <prarit@redhat.com>

prarit · 2020-09-04T00:14:18Z

@zaquestion, I've dropped WIP from the description and the patches. This code is ready for review IMO.

zaquestion · 2020-09-04T17:52:15Z

I'll be doing a deep dive over the weekend, thanks @prarit !

bmeneg

AFAICT the changes are working as expected and the code is pretty sane and clean.
Thanks @prarit .

zaquestion · 2020-09-05T22:32:38Z

Looking solid, I took it for a spin locally and saw my configs convert seamlessly. I added a simple test for the convert func, after the tests pass lets move forward and bring this in.

note: confpath convert was reordered to happen first to mitigate a very unlikely edge case where lab is run in `.config` causing the converted toml file to land in `.config` instead of `.config/lab/`

zaquestion · 2020-09-05T22:47:56Z

@prarit I'll let you do the honors 👍

When running lab in GitLab CI, lab will attempt to autoconfigure to use the CI credentials. While this is a useful feature for consumers of lab, it's working against us here. I've used the lab env vars which currently take precedence over the CI config, but when #414 comes in, I believe it'll break again, so we'll need to revisit then. The ssh config make it so lab can clone repos over ssh, as that key is attached to the `lab-testing` user

prarit changed the title ~~New config layout~~ WIP: New config layout Sep 2, 2020

prarit force-pushed the new_config_layout branch 8 times, most recently from f92f33f to 7ac7b53 Compare September 2, 2020 18:44

prarit mentioned this pull request Sep 2, 2020

Rehaul Configuration #407

Closed

prarit force-pushed the new_config_layout branch from 7708c93 to d0bb006 Compare September 2, 2020 20:09

bmeneg reviewed Sep 2, 2020

View reviewed changes

main.go Outdated Show resolved Hide resolved

main.go Outdated Show resolved Hide resolved

prarit force-pushed the new_config_layout branch 2 times, most recently from 21c1169 to 873fec5 Compare September 3, 2020 01:01

zaquestion reviewed Sep 3, 2020

View reviewed changes

prarit changed the title ~~WIP: New config layout~~ WIP: New config layout (SAVE YOUR ~/.config/lab.hcl BEFORE RUNNING THIS CODE) Sep 3, 2020

prarit force-pushed the new_config_layout branch 4 times, most recently from 3dd3552 to 7aeeb5a Compare September 3, 2020 17:15

prarit mentioned this pull request Sep 4, 2020

git-like config: Allow .git/lab/lab config to override ~/.config/lab/lab #415

Open

prarit added 4 commits September 3, 2020 20:10

config: Move config init code

8dda6ff

Move the config initialization code from main.go to internal/config/config.go. Suggested-by: Bruno Meneguele <bmeneg@redhat.com> Signed-off-by: Prarit Bhargava <prarit@redhat.com>

prarit changed the title ~~WIP: New config layout (SAVE YOUR ~/.config/lab.hcl BEFORE RUNNING THIS CODE)~~ New config layout Sep 4, 2020

prarit force-pushed the new_config_layout branch from 7aeeb5a to 51d9b7d Compare September 4, 2020 00:13

prarit linked an issue Sep 4, 2020 that may be closed by this pull request

Rehaul Configuration #407

Closed

prarit mentioned this pull request Sep 4, 2020

Read token from other sources (like secret/password manager) #382

Closed

bmeneg approved these changes Sep 4, 2020

View reviewed changes

zaquestion approved these changes Sep 5, 2020

View reviewed changes

config: add Convert test

6d8163f

note: confpath convert was reordered to happen first to mitigate a very unlikely edge case where lab is run in `.config` causing the converted toml file to land in `.config` instead of `.config/lab/`

zaquestion force-pushed the new_config_layout branch from 5b2f9c1 to 6d8163f Compare September 5, 2020 22:42

zaquestion added this to the 0.18.0 milestone Sep 5, 2020

prarit merged commit 084623b into zaquestion:master Sep 6, 2020

prarit deleted the new_config_layout branch January 21, 2021 19:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New config layout #414

New config layout #414

prarit commented Sep 2, 2020

codecov bot commented Sep 2, 2020 •

edited

prarit commented Sep 3, 2020

zaquestion commented Sep 3, 2020

zaquestion left a comment •

edited

zaquestion Sep 3, 2020

prarit Sep 3, 2020

bmeneg commented Sep 3, 2020 •

edited

prarit commented Sep 4, 2020

zaquestion commented Sep 4, 2020

bmeneg left a comment

zaquestion commented Sep 5, 2020

zaquestion commented Sep 5, 2020

New config layout #414

New config layout #414

Conversation

prarit commented Sep 2, 2020

codecov bot commented Sep 2, 2020 • edited

Codecov Report

prarit commented Sep 3, 2020

zaquestion commented Sep 3, 2020

zaquestion left a comment • edited

Choose a reason for hiding this comment

zaquestion Sep 3, 2020

Choose a reason for hiding this comment

prarit Sep 3, 2020

Choose a reason for hiding this comment

bmeneg commented Sep 3, 2020 • edited

prarit commented Sep 4, 2020

zaquestion commented Sep 4, 2020

bmeneg left a comment

Choose a reason for hiding this comment

zaquestion commented Sep 5, 2020

zaquestion commented Sep 5, 2020

codecov bot commented Sep 2, 2020 •

edited

zaquestion left a comment •

edited

bmeneg commented Sep 3, 2020 •

edited