Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test: Unify the randomization logic for number of shards and replicas #7522

Closed
wants to merge 1 commit into from

Conversation

Projects
None yet
3 participants
@javanna
Copy link
Member

commented Aug 30, 2014

We currently have two ways to randomize the number of shards and replicas: random index template, that stays the same for all indices created under the same scope, and the overridable indexSettings method, called by createIndex and prepareCreate which uses a different value for each new index.

Now that the randomIndexTemplate method is not static anymore, we can easily apply the same logic in both cases. Especially for number of replicas, we used to have slightly different behaviours, where more than one replicas were only rarely used through random index template, which gets now applied to the indexSettings method too (might speed up the tests a bit).

Side note: randomIndexTemplate had its own logic which didn't depend on numberOfReplicas or maximumNumberOfReplicas, which was causing bw comp tests failures since in some cases too many copies of the data are requested, which cannot be allocated to older nodes, and the write consistency quorum cannot be met, thus indexing times out.

[TEST] Unify the randomization logic for number of shards and replicas
We currently have two ways to randomize the number of shards and replicas: random index template, that stays the same for all indices created under the same scope, and the overridable `indexSettings` method, called by `createIndex` and `prepareCreate` which returns different values each time it gets called.

Now that the `randomIndexTemplate` method is not static anymore, we can easily apply the same logic to both. Especially for number of replicas, we used to have slightly different behaviours, where more than one replicas were only rarely used through random index template, which gets now applied to the `indexSettings` method too (might speed up the tests a bit).

Side note: `randomIndexTemplate` had its own logic which didn't depend on `numberOfReplicas` or `maximumNumberOfReplicas`, which was causing bw comp tests failures since in some cases too many copies of the data are requested, whcih cannot be allocated to older nodes, and the write consistency quorum cannot be met, thus indexing times out.

@javanna javanna added test labels Aug 30, 2014

@javanna javanna self-assigned this Aug 30, 2014

@javanna javanna added the review label Aug 30, 2014

@s1monw

This comment has been minimized.

Copy link
Contributor

commented Aug 31, 2014

LGTM

@s1monw s1monw removed the review label Aug 31, 2014

@javanna javanna closed this in ab57d4a Sep 1, 2014

javanna added a commit that referenced this pull request Sep 1, 2014

[TEST] Unify the randomization logic for number of shards and replicas
We currently have two ways to randomize the number of shards and replicas: random index template, that stays the same for all indices created under the same scope, and the overridable `indexSettings` method, called by `createIndex` and `prepareCreate` which returns different values each time.

Now that the `randomIndexTemplate` method is not static anymore, we can easily apply the same logic to both. Especially for number of replicas, we used to have slightly different behaviours, where more than one replicas were only rarely used through random index template, which gets now applied to the `indexSettings` method too (might speed up the tests a bit)

Side note: `randomIndexTemplate` had its own logic which didn't depend on `numberOfReplicas` or `maximumNumberOfReplicas`, which was causing bw comp tests failures since in some cases too many copies of the data are requested, which cannot be allocated to older nodes, and the write consistency quorum cannot be met, thus indexing times out.

Closes #7522

javanna added a commit that referenced this pull request Sep 8, 2014

[TEST] Unify the randomization logic for number of shards and replicas
We currently have two ways to randomize the number of shards and replicas: random index template, that stays the same for all indices created under the same scope, and the overridable `indexSettings` method, called by `createIndex` and `prepareCreate` which returns different values each time.

Now that the `randomIndexTemplate` method is not static anymore, we can easily apply the same logic to both. Especially for number of replicas, we used to have slightly different behaviours, where more than one replicas were only rarely used through random index template, which gets now applied to the `indexSettings` method too (might speed up the tests a bit)

Side note: `randomIndexTemplate` had its own logic which didn't depend on `numberOfReplicas` or `maximumNumberOfReplicas`, which was causing bw comp tests failures since in some cases too many copies of the data are requested, which cannot be allocated to older nodes, and the write consistency quorum cannot be met, thus indexing times out.

Closes #7522

@clintongormley clintongormley changed the title [TEST] Unify the randomization logic for number of shards and replicas Test: Unify the randomization logic for number of shards and replicas Sep 8, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.