Add Truncindex for container, sandbox and image #235

yanxuean · 2017-09-11T11:11:05Z

Signed-off-by: yanxuean yan.xuean@zte.com.cn

yanxuean · 2017-09-15T01:49:09Z

Really bumpy. It is passed finally.

Random-Liu · 2017-09-15T22:17:17Z

Will probably review this a bit later. Will review other PRs first.
And please rebase to get rid of the commit after your config PR gets merged. :)
Thanks!

Random-Liu · 2017-10-09T17:17:20Z

@yanxuean I'll take a look at this today.

yanxuean · 2017-10-10T02:23:55Z

@Random-Liu Tks

Random-Liu

LGTM overall with several nits.

Random-Liu · 2017-10-11T03:59:25Z

pkg/server/container_attach.go

@@ -38,7 +38,7 @@ func (c *criContainerdService) Attach(ctx context.Context, r *runtime.AttachRequ

 	cntr, err := c.containerStore.Get(r.GetContainerId())
 	if err != nil {
-		return nil, fmt.Errorf("failed to find container in store: %v", err)
+		return nil, fmt.Errorf("failed to find container %q in store: %v", r.GetContainerId(), err)


I intentionally didn't added the id, because the CRI caller already knows the id, and usually print out the id in its error message.

I think it's fine to not add the id.

I don't understand. Why add it at below?

func (c *criContainerdService) Attach(ctx context.Context, r *runtime.AttachRequest) (retRes *runtime.AttachResponse, retErr error) { glog.V(2).Infof("Attach for %q with tty %v and stdin %v", r.GetContainerId(), r.GetTty(), r.GetStdin()) defer func() { if retErr == nil { glog.V(2).Infof("Attach for %q returns URL %q", r.GetContainerId(), retRes.Url) } }()

@yanxuean That's log. For log we should include the id, so that we could see what container is throwing this error in log. :)

Any way, I just feel like, on kubelet side, when there is an error, we'll see the id in error 2 or more times in the error message, which is a bit redundant to me. So I kind of prefer not including container id in our error message if caller already knows it.

I see. Thanks

Random-Liu · 2017-10-11T04:10:11Z

pkg/store/container/container_test.go

 		container, err := NewContainer(
-			metadatas[id],
+			v,


nit: Can we still use metadatas[id]?

So that in the future, if this becomes a pointer, this still won't have problem

Random-Liu · 2017-10-11T04:17:53Z

pkg/store/image/image_test.go

+		assert.Contains(got.RepoDigests, oldRepoDigest, newRepoDigest)
+
+		t.Logf("should not be able to add duplicated repo tags/digests")
+		assert.Equal(nil, s.Add(newImg))


assert.NoError.

Random-Liu · 2017-10-11T04:27:42Z

pkg/server/image_pull.go

@@ -142,7 +142,9 @@ func (c *criContainerdService) PullImage(ctx context.Context, r *runtime.PullIma
 		img.RepoTags = []string{repoTag}
 	}

-	c.imageStore.Add(img)
+	if err := c.imageStore.Add(img); err != nil {
+		return nil, fmt.Errorf("failed to add image %q into store: %v", img.ID, err)


I'm fine with this ID here, because caller doesn't know it.

Random-Liu · 2017-10-11T04:29:02Z

pkg/store/container/container.go

@@ -112,6 +116,10 @@ func (s *Store) Add(c Container) error {
 	if _, ok := s.containers[c.ID]; ok {
 		return store.ErrAlreadyExist
 	}
+


nit: unnecessary empty line.

Random-Liu · 2017-10-11T04:39:45Z

pkg/store/image/image_test.go

+		newImg := v
+		newImg.RepoTags = []string{newRepoTag}
+		newImg.RepoDigests = []string{newRepoDigest}
+		s.Add(newImg)


assert.NoError.

Random-Liu · 2017-10-11T04:40:15Z

pkg/store/sandbox/sandbox.go

@@ -58,6 +62,11 @@ func (s *Store) Add(sb Sandbox) error {
 	if _, ok := s.sandboxes[sb.ID]; ok {
 		return store.ErrAlreadyExist
 	}
+


nit: empty line not needed and the following one.

done
Are there rules? @Random-Liu

I was told that we should avoid unnecessary empty lines to make the code short and easy to read. So that we could have more code in one screen and easier for reader to understand.

Some convention inside Google. :)

Not from google but i hate empty lines as well ;)

@crosbymichael OK. So everyone hates empty lines not only Google. :)

The rules are subjective:)

Random-Liu · 2017-10-11T04:40:21Z

pkg/store/sandbox/sandbox.go

+	if err != nil {
+		return
+	}
+	_ = s.idIndex.Delete(id)


Random-Liu · 2017-10-11T04:40:32Z

pkg/store/sandbox/sandbox_test.go

-		sandboxes[id] = Sandbox{Metadata: metadatas[id]}
+	//ids := []string{"1", "2", "3"}
+	for id, v := range metadatas {
+		sandboxes[id] = Sandbox{Metadata: v}


Random-Liu · 2017-10-11T04:40:45Z

pkg/store/sandbox/sandbox_test.go

 	for id, sb := range sandboxes {
-		got, err := s.Get(id)
+		got, err := s.Get(genTruncIndex(id))
+


why empty line?

done.
I promise not for KPI:)

yanxuean · 2017-10-11T09:58:38Z

Do we add truncindex for the filter of Stat function?

yanxuean · 2017-10-11T10:21:11Z

refer to #293. ContainerStats

mikebrow

/LGTM very nice

Random-Liu

LGTM with last comment. It's my fault, sorry for the trouble.

Random-Liu · 2017-10-11T17:54:22Z

pkg/store/sandbox/sandbox.go

@@ -88,5 +102,7 @@ func (s *Store) List() []Sandbox {
 func (s *Store) Delete(id string) {
 	s.lock.Lock()
 	defer s.lock.Unlock()
+	id, _ = s.idIndex.Get(id) // nolint: errcheck


One last comment. It's my fault.
I didn't know that idIndex.Delete doesn't handle truncated index.
If so, I'm fine with your original code, the new one seems fragile. Sorry for the trouble... :(

Both make sence. When i write it at first I just do it like this. We do best effort to clear ID even through there are some abnormal case in idindex. Because it is regular ID not truncated index in most case. We only use truncindex with crictl.Do you think about it?

@yanxuean If there is an error, the returned id will be empty string, I feel like we should just return as what you did before. Please add comment to explain why we return there.

As for the Delete, I think it's fine to just call s.idIndex.Delete(id), we don't need _ =` in front.

Random-Liu · 2017-10-11T18:01:05Z

@yanxuean Could you add a simple integration test for this? Just create a single container pod, and get pod/container/image with truncated index.

Integration test is here https://github.com/kubernetes-incubator/cri-containerd/tree/master/integration.
I'm fine with adding that in another PR.

yanxuean · 2017-10-11T23:48:19Z

OK, I will add a intergration test.

yanxuean · 2017-10-12T01:52:06Z

@Random-Liu @crosbymichael @mikebrow Thank for review

fix containerd#222 Signed-off-by: yanxuean <yan.xuean@zte.com.cn>

k8s-ci-robot added cncf-cla: yes size/L labels Sep 11, 2017

yanxuean force-pushed the truncindex branch 4 times, most recently from c5d7128 to 220f353 Compare September 12, 2017 00:30

k8s-ci-robot added size/XXL and removed size/L labels Sep 12, 2017

yanxuean force-pushed the truncindex branch 4 times, most recently from 1cec173 to 71e9e79 Compare September 15, 2017 01:10

yanxuean force-pushed the truncindex branch 4 times, most recently from 797ec70 to e9a73a1 Compare September 19, 2017 00:57

Random-Liu mentioned this pull request Sep 21, 2017

CRI-Containerd Missing Pieces #62

Closed

42 tasks

yanxuean force-pushed the truncindex branch 3 times, most recently from 204f8e6 to 341c141 Compare September 27, 2017 15:58

Random-Liu added this to the v1.0.0-alpha.1 milestone Sep 28, 2017

Random-Liu reviewed Oct 11, 2017

View reviewed changes

Random-Liu self-assigned this Oct 11, 2017

yanxuean force-pushed the truncindex branch from 341c141 to 67d5ae6 Compare October 11, 2017 12:53

mikebrow approved these changes Oct 11, 2017

View reviewed changes

Random-Liu reviewed Oct 11, 2017

View reviewed changes

yanxuean force-pushed the truncindex branch from 67d5ae6 to 2bf9ba7 Compare October 12, 2017 01:14

yanxuean force-pushed the truncindex branch 2 times, most recently from 51c2f5f to 5ee3423 Compare October 12, 2017 02:30

add truncindex

5ee3423

fix containerd#222 Signed-off-by: yanxuean <yan.xuean@zte.com.cn>

Random-Liu added the lgtm label Oct 12, 2017

Random-Liu merged commit 61d598d into containerd:master Oct 12, 2017

yanxuean deleted the truncindex branch October 12, 2017 05:53

Add Truncindex for container, sandbox and image #235

Add Truncindex for container, sandbox and image #235

Conversation

yanxuean commented Sep 11, 2017

yanxuean commented Sep 15, 2017

Random-Liu commented Sep 15, 2017 • edited

Random-Liu commented Oct 9, 2017

yanxuean commented Oct 10, 2017

Random-Liu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yanxuean Oct 11, 2017 • edited

Choose a reason for hiding this comment

yanxuean commented Oct 11, 2017

yanxuean commented Oct 11, 2017

mikebrow left a comment

Choose a reason for hiding this comment

Random-Liu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Random-Liu commented Oct 11, 2017 • edited

yanxuean commented Oct 11, 2017

yanxuean commented Oct 12, 2017

Random-Liu commented Sep 15, 2017 •

edited

yanxuean Oct 11, 2017 •

edited

Random-Liu commented Oct 11, 2017 •

edited