Skip to content

etcdctl: fix json output #19469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 1, 2025
Merged

Conversation

kstrifonoff
Copy link
Contributor

@kstrifonoff kstrifonoff commented Feb 23, 2025

Fix #19262

Example output of etcdctl member add test-node --learner=true --peer-urls=http://127.0.0.1:22380 --write-out json --hex command:

Before:

{
  "header": {
    "cluster_id": 14841639068965178418,
    "member_id": 10276657743932975437,
    "raft_term": 2
  },
  "member": {
    "ID": 17436648322565278587,
    "peerURLs": ["http://127.0.0.1:22380"],
    "isLearner": true
  },
  "members": [
    {
      "ID": 10276657743932975437,
      "name": "default",
      "peerURLs": ["http://localhost:2380"],
      "clientURLs": ["http://localhost:2379"]
    },
    {
      "ID": 17436648322565278587,
      "peerURLs": ["http://127.0.0.1:22380"],
      "isLearner": true
    }
  ]
}

After

{
  "header": {
    "cluster_id": "cdf818194e3a8c32",
    "member_id": "8e9e05c52164694d",
    "raft_term": 2
  },
  "member": {
    "ID": "5bdeb7bdb679126b",
    "peerURLs": ["http://127.0.0.1:22380"],
    "isLearner": true
  },
  "members": [
    {
      "ID": "5bdeb7bdb679126b",
      "peerURLs": ["http://127.0.0.1:22380"],
      "isLearner": true
    },
    {
      "ID": "8e9e05c52164694d",
      "name": "default",
      "peerURLs": ["http://localhost:2380"],
      "clientURLs": ["http://localhost:2379"]
    }
  ]
}

@k8s-ci-robot
Copy link

Hi @kstrifonoff. Thanks for your PR.

I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@kstrifonoff
Copy link
Contributor Author

/cc @ivanvc

@k8s-ci-robot k8s-ci-robot requested a review from ivanvc February 23, 2025 15:10
@ivanvc
Copy link
Member

ivanvc commented Feb 25, 2025

Hi @kstrifonoff, thanks for the pull request. I'm respectful to the original contributor who asked to work on this. Our guideline is to re-assign after a week of no answer when asking a contributor for updates.

A couple of suggestions by quickly looking at the code.

  1. The imports contain a lint error. Running make verify-lint locally will show you the error.
  2. The code doesn't have unit tests for this, but there's an e2e test, TestCtlV3MemberListWithHex. I think the test should pass, but adding your new cases would be good.
  3. I suggest reviewing the code from printKV. I think it would be better to follow that implementation approach, i.e., there would be less repetition with the if conditions.

Thanks again for your pull request :)

@kstrifonoff
Copy link
Contributor Author

Hi @ivanvc ,

I’ve made some changes to my initial suggestions based on your feedback.

Now, the “JSON with Hex” implementation is just a slight extension of the default “printJSON” implementation. I think it should probably be moved into the “utils” package.

Here are a few things I noticed:

  1. The implementation assumes marshalling back and forth the default JSON string, which can be inefficient, but I assume it’s still okay for the CLI.
  2. Also, the current approach makes it easy to add “json with hex” output for all the rest of commands that support JSON output.
  3. I’m not sure what kind of end-to-end test should be done here. What should it cover? It’s also not clear to me what the current end-to-end test actually checks.

@kstrifonoff kstrifonoff marked this pull request as ready for review March 6, 2025 20:07
@ivanvc
Copy link
Member

ivanvc commented Mar 7, 2025

/ok-to-test

@ivanvc
Copy link
Member

ivanvc commented Mar 7, 2025

I’m not sure what kind of end-to-end test should be done here. What should it cover? It’s also not clear to me what the current end-to-end test actually checks.

The e2e tests already exist, my suggestion is to expand them to cover the new cases you added:

func memberListWithHexTest(cx ctlCtx) {
resp, err := getMemberList(cx, false)
if err != nil {
cx.t.Fatalf("getMemberList error (%v)", err)
}
cmdArgs := append(cx.PrefixArgs(), "--write-out", "json", "--hex", "member", "list")
proc, err := e2e.SpawnCmd(cmdArgs, cx.envMap)
if err != nil {
cx.t.Fatalf("memberListWithHexTest error (%v)", err)
}
var txt string
txt, err = proc.Expect("members")
if err != nil {
cx.t.Fatalf("memberListWithHexTest error (%v)", err)
}
if err = proc.Close(); err != nil {
cx.t.Fatalf("memberListWithHexTest error (%v)", err)
}
hexResp := etcdserverpb.MemberListResponse{}
dec := json.NewDecoder(strings.NewReader(txt))
if err := dec.Decode(&hexResp); errors.Is(err, io.EOF) {
cx.t.Fatalf("memberListWithHexTest error (%v)", err)
}
num := len(resp.Members)
hexNum := len(hexResp.Members)
if num != hexNum {
cx.t.Fatalf("member number,expected %d,got %d", num, hexNum)
}
if num == 0 {
cx.t.Fatal("member number is 0")
}
if resp.Header.RaftTerm != hexResp.Header.RaftTerm {
cx.t.Fatalf("Unexpected raft_term, expected %d, got %d", resp.Header.RaftTerm, hexResp.Header.RaftTerm)
}
for i := 0; i < num; i++ {
if resp.Members[i].Name != hexResp.Members[i].Name {
cx.t.Fatalf("Unexpected member name,expected %v, got %v", resp.Members[i].Name, hexResp.Members[i].Name)
}
if !reflect.DeepEqual(resp.Members[i].PeerURLs, hexResp.Members[i].PeerURLs) {
cx.t.Fatalf("Unexpected member peerURLs, expected %v, got %v", resp.Members[i].PeerURLs, hexResp.Members[i].PeerURLs)
}
if !reflect.DeepEqual(resp.Members[i].ClientURLs, hexResp.Members[i].ClientURLs) {
cx.t.Fatalf("Unexpected member clientURLs, expected %v, got %v", resp.Members[i].ClientURLs, hexResp.Members[i].ClientURLs)
}
}
}

Copy link

codecov bot commented Mar 7, 2025

Codecov Report

Attention: Patch coverage is 95.74468% with 2 lines in your changes missing coverage. Please review.

Project coverage is 69.22%. Comparing base (d744c19) to head (b42b5f9).
Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
etcdctl/ctlv3/command/printer_json.go 95.74% 2 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
etcdctl/ctlv3/command/printer_json.go 46.60% <95.74%> (+46.60%) ⬆️

... and 20 files with indirect coverage changes

@@            Coverage Diff             @@
##             main   #19469      +/-   ##
==========================================
- Coverage   69.28%   69.22%   -0.06%     
==========================================
  Files         414      414              
  Lines       34408    34455      +47     
==========================================
+ Hits        23840    23853      +13     
- Misses       9180     9203      +23     
- Partials     1388     1399      +11     

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d744c19...b42b5f9. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kstrifonoff kstrifonoff force-pushed the fix-hex-json-printer branch from 59ffb52 to d744c19 Compare June 29, 2025 04:56
@kstrifonoff kstrifonoff reopened this Jun 29, 2025
@kstrifonoff kstrifonoff marked this pull request as ready for review June 29, 2025 04:59
@kstrifonoff kstrifonoff force-pushed the fix-hex-json-printer branch 3 times, most recently from 2e1a2cc to 17b58b6 Compare June 29, 2025 05:28
minor refactoring
add unit tests

Signed-off-by: Kirill Trifonov <kstrifonoff@gmail.com>
@kstrifonoff kstrifonoff force-pushed the fix-hex-json-printer branch from 17b58b6 to b42b5f9 Compare June 29, 2025 05:44
@kstrifonoff
Copy link
Contributor Author

/retest

1 similar comment
@kstrifonoff
Copy link
Contributor Author

/retest

@k8s-ci-robot
Copy link

k8s-ci-robot commented Jun 29, 2025

@kstrifonoff: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci-etcd-robustness-release36-amd64 311626f link true /test ci-etcd-robustness-release36-amd64

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@kstrifonoff
Copy link
Contributor Author

/retest

@kstrifonoff kstrifonoff requested a review from ahrtr June 29, 2025 21:34
Copy link
Member

@ahrtr ahrtr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM & thx

@k8s-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahrtr, fuweid, kstrifonoff

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Elbehery
Copy link
Member

Elbehery commented Jul 1, 2025

@ivanvc did @kstrifonoff address your review ?

@ahrtr ahrtr merged commit 0199cca into etcd-io:main Jul 1, 2025
31 checks passed
@kstrifonoff kstrifonoff deleted the fix-hex-json-printer branch July 1, 2025 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

etcdctl member add -wjson --hex=true does not output member IDs in hex
6 participants