New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace individual endpoint_cnt read from store with 1 bulk read #1632

Merged
merged 2 commits into from Feb 2, 2017

Conversation

Projects
None yet
2 participants
@mavenugo
Contributor

mavenugo commented Feb 2, 2017

getNetworksFromStore reads networks and endpoint_cnt from the kvstores.
endpoint_cnt especially is read in a for-loop for each network and that
causes a lot of stress in poorly performing KV-Stores.
This fix eases the load on the kvstore by fetching all the endpoint_cnt
in a single read and the operation is performed on it.

Signed-off-by: Madhu Venugopal madhu@docker.com

@@ -152,21 +153,24 @@ func (c *controller) getNetworksFromStore() ([]*network, error) {
continue
}
kvep, err := store.Map(datastore.Key(epCntKeyPrefix), &endpointCnt{})

This comment has been minimized.

@aboch

aboch Feb 2, 2017

Contributor

Looking at the diffs, I am realizing there was no need for getNetworks() to reconstruct the epCount for each pulled network from store. Whatever code that makes a decision about the epCount, it does pull the latest epCount from store itself. There is no need to retrieve and construct the internal epCounts in the networks retrieved as list.

My advice is to simply remove the existing code which pulls the epCount in the loop below.
It is not needed.

@aboch

aboch Feb 2, 2017

Contributor

Looking at the diffs, I am realizing there was no need for getNetworks() to reconstruct the epCount for each pulled network from store. Whatever code that makes a decision about the epCount, it does pull the latest epCount from store itself. There is no need to retrieve and construct the internal epCounts in the networks retrieved as list.

My advice is to simply remove the existing code which pulls the epCount in the loop below.
It is not needed.

This comment has been minimized.

@aboch

aboch Feb 2, 2017

Contributor

Goal now is to improve the performance of current code in respect to the getNetworksFromStore().
GivengetNetworksFromStore() retrieves the list of networks in a batch, it makes a lot of sense to retrieve the epCnt as a batch along it.

We can address the code removal I was suggesting in a subsequent PR outside of the release cycle.

@aboch

aboch Feb 2, 2017

Contributor

Goal now is to improve the performance of current code in respect to the getNetworksFromStore().
GivengetNetworksFromStore() retrieves the list of networks in a batch, it makes a lot of sense to retrieve the epCnt as a batch along it.

We can address the code removal I was suggesting in a subsequent PR outside of the release cycle.

Show outdated Hide outdated endpoint_info.go

mavenugo added some commits Feb 2, 2017

replace individual endpoint_cnt read from store with 1 bulk read
getNetworksFromStore reads networks and endpoint_cnt from the kvstores.
endpoint_cnt especially is read in a for-loop for each network and that
causes a lot of stress in poorly performing KV-Stores.
This fix eases the load on the kvstore by fetching all the endpoint_cnt
in a single read and the operation is performed on it.

Signed-off-by: Madhu Venugopal <madhu@docker.com>
Trust the endpoint state if we have a valid sandbox-id
Signed-off-by: Madhu Venugopal <madhu@docker.com>
@aboch

This comment has been minimized.

Show comment
Hide comment
@aboch

aboch Feb 2, 2017

Contributor

LGTM

Contributor

aboch commented Feb 2, 2017

LGTM

@aboch aboch merged commit 802895f into docker:master Feb 2, 2017

2 checks passed

ci/circleci Your tests passed on CircleCI!
Details
dco-signed All commits are signed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment