Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IndexShardRoutingTable's shard randomization to not throw out-of-bounds exceptions. #5561

Closed
wants to merge 2 commits into from

Conversation

jpountz
Copy link
Contributor

@jpountz jpountz commented Mar 26, 2014

I initially wanted to make the diff minimal but this ended up being quite complicated
so I finally refactored a bit the way shards are randomized. Yet, it uses the same logic:

  • rotations to shuffle shards,
  • an AtomicInteger to generate the distances to use for the rotations.

Close #5559

final List<Object> rotated = CollectionUtils.rotate(list, size);
assertEquals(rotated.size(), list.size());
assertEquals(Iterables.size(rotated), list.size());
assertEquals(new HashSet<>(rotated), new HashSet<>(list));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm I gues we should also test that the contents are the same here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grr nevermind I didn't see the last line pfff...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one think that I think we should test is that the roation here is actually stable. It if you iterate the same list twice it has the same order?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean checking that rotating twice the same list gives the same output?

assertEquals(CollectionUtils.rotate(list, size), CollectionUtils.rotate(list, size));

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or N times

@s1monw
Copy link
Contributor

s1monw commented Mar 27, 2014

this looks great - I left some commetns

@jpountz
Copy link
Contributor Author

jpountz commented Mar 27, 2014

@s1monw I pushed a new commit

@s1monw
Copy link
Contributor

s1monw commented Mar 27, 2014

LGTM

@jpountz
Copy link
Contributor Author

jpountz commented Mar 27, 2014

Thanks Simon.

@jpountz jpountz closed this Mar 27, 2014
@jpountz jpountz deleted the fix/IndexShardRoutingTable branch March 27, 2014 13:49
jpountz added a commit to jpountz/elasticsearch that referenced this pull request May 5, 2014
Change elastic#5561 introduced a potential bug in that iterations that are performed
on a thread are might not be visible to other threads due to the removal of the
`volatile` keyword.

Close elastic#6039
jpountz added a commit that referenced this pull request May 5, 2014
Change #5561 introduced a potential bug in that iterations that are performed
on a thread are might not be visible to other threads due to the removal of the
`volatile` keyword.

Close #6039
jpountz added a commit that referenced this pull request May 5, 2014
Change #5561 introduced a potential bug in that iterations that are performed
on a thread are might not be visible to other threads due to the removal of the
`volatile` keyword.

Close #6039
jpountz added a commit that referenced this pull request May 5, 2014
Change #5561 introduced a potential bug in that iterations that are performed
on a thread are might not be visible to other threads due to the removal of the
`volatile` keyword.

Close #6039
@clintongormley clintongormley added :Search/Search Search-related issues that do not fall into other categories v0.90.14 v1.0.3 v2.0.0-beta1 v1.2.0 v1.1.1 >bug labels Jun 7, 2015
mute pushed a commit to mute/elasticsearch that referenced this pull request Jul 29, 2015
Change elastic#5561 introduced a potential bug in that iterations that are performed
on a thread are might not be visible to other threads due to the removal of the
`volatile` keyword.

Close elastic#6039
mute pushed a commit to mute/elasticsearch that referenced this pull request Jul 29, 2015
Change elastic#5561 introduced a potential bug in that iterations that are performed
on a thread are might not be visible to other threads due to the removal of the
`volatile` keyword.

Close elastic#6039
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Search Search-related issues that do not fall into other categories v0.90.14 v1.0.3 v1.1.1 v1.2.0 v2.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

IndexShardRoutingTable might barf if it has handled lots of searches...
3 participants