Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add decode kvsdb tool #2059

Merged
merged 7 commits into from Jun 26, 2023
Merged

add decode kvsdb tool #2059

merged 7 commits into from Jun 26, 2023

Conversation

datelier
Copy link
Contributor

Description:

add decode kvsdb tool.

run example:

go run main.go --format XXX --path XXX --file XXX --timestamp-file XXX

Related Issue:

Versions:

  • Go Version: 1.20.3
  • Docker Version: 20.10.8
  • Kubernetes Version: 1.22.0
  • NGT Version: 2.0.11

Checklist:

Special notes for your reviewer:

@datelier datelier requested review from a team, kmrmt and hlts2 and removed request for a team June 13, 2023 06:35
@vdaas-ci
Copy link
Collaborator

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 💌 /changelog - replace the PR body by changelog details
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • /rebase - rebase main
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

@cloudflare-pages
Copy link

cloudflare-pages bot commented Jun 13, 2023

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 5e12307
Status: ✅  Deploy successful!
Preview URL: https://14d1b97b.vald.pages.dev
Branch Preview URL: https://feature-tools-kvsdb.vald.pages.dev

View logs

kvsdb.Set(k, id, 0)
}
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be better to add header.

Comment on lines 48 to 65
f, _ = file.Open(
file.Join(*path, *kvsFileName),
os.O_RDONLY|os.O_SYNC,
fs.ModePerm,
)
_ = gob.NewDecoder(f).Decode(&m)

// timestamp
mt := make(map[string]int64)
gob.Register(map[string]int64{})
var ft *os.File
defer ft.Close()
ft, _ = file.Open(
file.Join(*path, *kvsTimestampFileName),
os.O_RDONLY|os.O_SYNC,
fs.ModePerm,
)
_ = gob.NewDecoder(ft).Decode(&mt)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should stop this program like below when it returns errors because there can be the case such as no permission.
And I think you should consider logging levels.
This suggestion is just idea.

Suggested change
f, _ = file.Open(
file.Join(*path, *kvsFileName),
os.O_RDONLY|os.O_SYNC,
fs.ModePerm,
)
_ = gob.NewDecoder(f).Decode(&m)
// timestamp
mt := make(map[string]int64)
gob.Register(map[string]int64{})
var ft *os.File
defer ft.Close()
ft, _ = file.Open(
file.Join(*path, *kvsTimestampFileName),
os.O_RDONLY|os.O_SYNC,
fs.ModePerm,
)
_ = gob.NewDecoder(ft).Decode(&mt)
var err error
f, err = file.Open(
file.Join(*path, *kvsFileName),
os.O_RDONLY|os.O_SYNC,
fs.ModePerm,
)
if err != nil {
log.Fatal(err)
}
if err = gob.NewDecoder(f).Decode(&m); err != nil {
log.Fatal(err)
}
// timestamp
mt := make(map[string]int64)
gob.Register(map[string]int64{})
var ft *os.File
defer ft.Close()
ft, err = file.Open(
file.Join(*path, *kvsTimestampFileName),
os.O_RDONLY|os.O_SYNC,
fs.ModePerm,
)
if err != nil {
log.Fatal(err)
}
if err = gob.NewDecoder(ft).Decode(&mt); err != nil {
log.Fatal(err)
}

s = append(s, []string{k, strconv.FormatUint(uint64(id), 10), "0"})
}
if len(s)*int(unsafe.Sizeof("")) > 4e+6 {
w.WriteAll(s)
Copy link
Contributor

@kmrmt kmrmt Jun 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

Suggested change
w.WriteAll(s)
if err = w.WriteAll(s); err != nil {
log.Warn(err)
}

s = nil
}
}
w.WriteAll(s)
Copy link
Contributor

@kmrmt kmrmt Jun 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

Suggested change
w.WriteAll(s)
if err = w.WriteAll(s); err != nil {
log.Fatal(err)
}

@github-actions github-actions bot added size/L and removed size/M labels Jun 20, 2023
Copy link
Contributor

@kmrmt kmrmt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@hlts2 hlts2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hlts2 hlts2 merged commit 5810429 into main Jun 26, 2023
22 of 23 checks passed
@hlts2 hlts2 deleted the feature/tools/kvsdb branch June 26, 2023 04:59
@hlts2 hlts2 mentioned this pull request Sep 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants