Fix scheduler race

[bprashanth]

The scheduler counts pods by combining the pods in the scheduled and assumed pod stores. The listing needs a to be synchronized with the informer.

/ref #10720
@davidopp @gmarek @lavalamp

[wojtek-t]

Good catch - LGTM.

[k8s-bot]

GCE e2e build/test passed for commit 0728c08.

• Build Log
• Test Artifacts
• Internal Jenkins Results

[gmarek]

Thanks @bprashanth great catch. I missed this, as nothing else was locked in this file:/

[thockin]

Does this need to cherry-pick to release-1.0 ?

@zmerlynn FYI

[thockin]

Waiting for OK tag before merge

[gmarek]

It's a (very) probable cause of max-pods e2e test failure. Without fixing this it's possible that scheduler will overallocate Nodes (it happens in the test). Locking of modeller is distributed, which makes changes around LockedAction slightly tricky though...

[bprashanth]

Shippable looks like a unittest flake (https://app.shippable.com/builds/55a3cd28a4a0be0e007c1f22), test-go passes locally so I kicked it.

Risk: low-medium, since it just adds a lock around a non blocking operation.
Value: @gmarek is right.

overcommit nodes are notoriously hard to debug and the kubelet won't reject these pods either, so I think we should consider either this or #11161.

@davidopp to make the call about cherry-pick

[davidopp]

LGTM

Thanks for finding and fixing!

I'll add this to my list of things to cherry-pick.

[lavalamp]

Good find!

[rrati]

@davidopp Did this get merged to 1.0?