Fake docker-client assigns random IPs to containers

[shyamjvs]

Fixes #59823

/cc @wojtek-t @Random-Liu

[k8s-ci-robot]

@shyamjvs: Adding do-not-merge/release-note-label-needed because the release note process has not been followed.

One of the following labels is required "release-note", "release-note-action-required", or "release-note-none".
Please see: https://git.k8s.io/community/contributors/devel/pull-requests.md#write-release-notes-if-needed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[shyamjvs]

/test pull-kubernetes-kubemark-e2e-gce-big

To verify if we stop seeing those 422s for PUT endpoints calls.

[wojtek-t]

@shyamjvs - two things:

1. it seems it didn't really help
2. bazel failure is a real one

[shyamjvs]

I'll look into the bazel failure (which I guess is because of unit test failing).
And re "it didn't really help", it seems like there is a reduced no. of 422s (as compared to before when all updates starting form the 2nd one were returning 422s) - so it did help by some amount iiuc.

Let's wait for your run with debug statements to know more.

[wojtek-t]

I'll look into the bazel failure (which I guess is because of unit test failing).

Yes - 2.3.4.5 is hardcoded somewhere.

And re "it didn't really help", it seems like there is a reduced no. of 422s (as compared to before when all updates starting form the 2nd one were returning 422s) - so it did help by some amount iiuc.

Agree - it helped somewhat, but I wouldn't expect any "BadRequest" errors coming from our components...

[wojtek-t]

So what is happening is this:

AAA (Endpoints load-small-971-svc) subsets[0].addresses[0].ip: Invalid value: "127.68.255.188": may not be in the loopback range (127       .0.0.0/8)

You need to generate addresse that will pass this validation:
https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/core/validation/validation.go#L4760

[wojtek-t]

In particular, what you seem to break is this function:
https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/core/validation/validation.go#L4782

[shyamjvs]

Thanks a lot for looking into it @wojtek-t. I'll fix it by making the IP address always start with 10 (i.e 10.x.x.x) which should avoid all those failure conditions in that function.

[shyamjvs]

Fixed the unit test (hopefully) by using a deterministic random seed.
Also creating valid random IPs in the docker mock now.

@wojtek-t PTAL

[shyamjvs]

/test pull-kubernetes-kubemark-e2e-gce-big

[wojtek-t]

I verified the impact on: #59807
In short:

• no more 422 errors
• 2x+ more memory allocations (~100GB coming from serialization)
  Both are expected and so this issues seems to be fixed now.

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: shyamjvs, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these OWNERS Files:

• pkg/kubelet/OWNERS [wojtek-t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-github-robot]

/test all [submit-queue is verifying that this PR is safe to merge]

[shyamjvs]

Thanks @wojtek-t. To add to it, the aggregate.Errors() are no longer seen in the memory profile (so this change pretty much seems to reduce all of that alloc space). Ref - https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/59807/pull-kubernetes-kubemark-e2e-gce-big/16/artifacts/profiles/ApiserverMemoryProfile_load.pdf

[k8s-ci-robot]

@shyamjvs: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-kubemark-e2e-gce-big	b54aa14e7a50acf58a9dc097ca8cf1c91be2bc31	link	/test pull-kubernetes-kubemark-e2e-gce-big

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-github-robot]

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.