Use sync.map to scale equiv class cache better

[resouer]

What this PR does / why we need it:

Change the current lock in first level ecache into sync.Map, which is known for scaling better than sync. Mutex on machines with >8 CPUs

ref: https://golang.org/pkg/sync/#Map

And the code is much cleaner in this way.

5k Nodes, 10k Pods benchmark with ecache enabled in 64 cores VM:

// before
BenchmarkScheduling/5000Nodes/0Pods-64             10000          17550089 ns/op

// after
BenchmarkScheduling/5000Nodes/0Pods-64             10000          16975098 ns/op

Comparing to current implementation, the improvement after this change is noticeable, and the test is stable in 8, 16, 64 cores VM.

Special notes for your reviewer:

Release note:

Use sync.map to scale ecache better

[resouer]

/assign @bsalamat

xref: #65714 (comment)

[misterikkit]

/lgtm

[misterikkit]

I think return new(Cache) is more canonical.

[resouer]

Done! Thanks!

[misterikkit]

/lgtm

[bsalamat]

/lgtm

[bsalamat]

If you look at the sync.Map documentation, it reads:

"The Map type is optimized for two common use cases: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate Mutex or RWMutex."

eCache does not follow any of these patterns. The second pattern is closer to what eCache does, but even that is not completely applicable to eCache. We have informer event handlers which may write/delete eCache entries in parallel to other go-routines that run our predicate functions. So, the sets of entries written/deleted in various goroutines are not disjoint.
The documentation also states:
"The Map type is specialized. Most code should use a plain Go map instead, with separate locking or coordination, for better type safety and to make it easier to maintain other invariants along with the map content."

These are the reasons that I am a bit unsure about using sync.Map here. Also, the performance improvement does not seem to be large. Those said, I don't have any serious objection against it.

[resouer]

Thanks Bobby! Actually, that's also the reason I did not use sync.Map at the beginning :D

While since lock contention is visibly reduced during benchmark CPU profiling:

https://github.com/resouer/temp/blob/master/torch_lock_true.5000.svg
https://github.com/resouer/temp/blob/master/torch_lock_true.map.5000.svg

and as the Cache logic is very simple, I guess we can keep the change as is.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bsalamat, misterikkit, resouer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/scheduler/OWNERS [bsalamat]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-github-robot]

/test all [submit-queue is verifying that this PR is safe to merge]

[k8s-github-robot]

Automatic merge from submit-queue (batch tested with PRs 66862, 67618). If you want to cherry-pick this change to another branch, please follow the instructions here.

[k8s-ci-robot]

@resouer: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-kubemark-e2e-gce-big	17d0190	link	/test pull-kubernetes-kubemark-e2e-gce-big

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.