Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: rewrite and persist config file #2153

Merged
merged 5 commits into from
Feb 26, 2020

Conversation

rleungx
Copy link
Member

@rleungx rleungx commented Feb 25, 2020

What problem does this PR solve?

After we enable the dynamic config, we will rewrite the config file and persist it once the config has been updated. Related to tikv/tikv#6684

What is changed and how it works?

The PR adds a new function named RewriteFile to do it.

Check List

Tests

  • Manual test

Signed-off-by: Ryan Leung <rleungx@gmail.com>
Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

server/config/config.go Outdated Show resolved Hide resolved
return nil
}
var buf bytes.Buffer
if err := toml.NewEncoder(&buf).Encode(*new); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it also writes out hidden configurations.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

slice := make([]interface{}, 0)
for _, str := range strSlice {
slice = append(slice, str)
// TODO: handle slice better
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you explain how it is not good enough now?

// TODO: handle slice better
if item, ok := updateItem.(string); ok {
strSlice := strings.Split(item, ",")
slice := make([]interface{}, 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var slice []interface{} or make([]interface{}, 0, len(strSlice))

Signed-off-by: Ryan Leung <rleungx@gmail.com>
return err
}
// 0644 represent the permission of file which is "-rw-r--r--" here.
return ioutil.WriteFile(filePath, buf.Bytes(), 0644)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid corrupted content during lost power, we should:

  1. write the content to temp file
  2. sync the temp file
  3. rename the temp file to the config file.

Signed-off-by: Ryan Leung <rleungx@gmail.com>
Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

dir := filepath.Dir(filePath)
tmpfile := filepath.Join(dir, "tmp_pd.toml")

f, err := os.Create(tmpfile)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default permission is 0666, should we change it to 0644?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's unnecessary?

@codecov-io
Copy link

Codecov Report

Merging #2153 into master will decrease coverage by 0.03%.
The diff coverage is 57.69%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2153      +/-   ##
==========================================
- Coverage   76.03%   75.99%   -0.04%     
==========================================
  Files         195      195              
  Lines       20376    20405      +29     
==========================================
+ Hits        15493    15507      +14     
- Misses       3695     3710      +15     
  Partials     1188     1188
Impacted Files Coverage Δ
server/api/config.go 56.75% <0%> (-0.98%) ⬇️
server/config/config.go 85.21% <33.33%> (-0.72%) ⬇️
server/server.go 79.56% <66.66%> (+0.03%) ⬆️
server/config_manager/config_manager.go 76.63% <75%> (-0.38%) ⬇️
server/kv/etcd_kv.go 75.32% <0%> (-9.1%) ⬇️
server/region_syncer/client.go 78.44% <0%> (-7.76%) ⬇️
client/base_client.go 84.82% <0%> (-2.07%) ⬇️
pkg/btree/btree.go 86.84% <0%> (-0.81%) ⬇️
server/tso/tso.go 80.29% <0%> (-0.73%) ⬇️
server/cluster/cluster.go 81.37% <0%> (-0.23%) ⬇️
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b82b56...2ddbdd9. Read the comment docs.

@rleungx
Copy link
Member Author

rleungx commented Feb 26, 2020

/merge

@sre-bot sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 26, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Feb 26, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Feb 26, 2020

@rleungx merge failed.

@rleungx
Copy link
Member Author

rleungx commented Feb 26, 2020

/run-all-tests

1 similar comment
@rleungx
Copy link
Member Author

rleungx commented Feb 26, 2020

/run-all-tests

@rleungx
Copy link
Member Author

rleungx commented Feb 26, 2020

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Feb 26, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Feb 26, 2020

@rleungx merge failed.

@rleungx
Copy link
Member Author

rleungx commented Feb 26, 2020

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Feb 26, 2020

/run-all-tests

@sre-bot sre-bot merged commit 05d6873 into tikv:master Feb 26, 2020
@tszCat
Copy link

tszCat commented May 8, 2020

will the comments in the old config file get persisted in the updated config file?

@rleungx
Copy link
Member Author

rleungx commented May 8, 2020

This feature has been removed, it won't rewrite the config file now

@tszCat
Copy link

tszCat commented May 8, 2020

This feature has been removed, it won't rewrite the config file now

Oh😱...
But I still wonder that is there any solutions to rewrite config file(TOML) while its comments get persisted too?

@rleungx
Copy link
Member Author

rleungx commented May 8, 2020

AFAIK, If you use BurntSushi/toml, there are issues that have some solution to do it, but they are not the official way, otherwise, you need to implement it by yourself.

@tszCat
Copy link

tszCat commented May 8, 2020

AFAIK, If you use BurntSushi/toml, there are issues that have some solution to do it, but they are not the official way, otherwise, you need to implement it by yourself.

got it, thx.
I've looked up issuses on encoding comments on BurntSushi's repo, but unfortunately I didn't get solution I expected. Maybe I should implement it by myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/config Configuration logic. status/can-merge Indicates a PR has been approved by a committer.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants