Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RecoveryState clean up #9811

Closed
wants to merge 9 commits into from
Closed

Conversation

bleskes
Copy link
Contributor

@bleskes bleskes commented Feb 23, 2015

To support the _recovery API, the recovery process keeps track of current progress in a class called RecoveryState. This class currently have some issues, mostly around concurrency (see #6644 ). This PR cleans it up as well as other issues around it:

  • Make the Index subsection API cleaner:
    • remove redundant information - all calculation is done based on the underlying file map
    • clearer definition of what is what: total files, vs reused files (local files that match the source) vs recovered files (copied over). % based progress is reported based on recovered files only.
    • cleaned up json response to match other API (sadly this breaks the structure). We now properly report human values for dates and other units.
    • Add more robust unit testing
  • Detail flag was passed along as state (it's now a ToXContent param)
  • State lookup during reporting is now always done via the IndexShard , no more fall backs to many other classes.
  • Cleanup APIs around time and move the little computations to the state class as opposed to doing them out of the API

I also improved error messages out of the REST testing infra for things I run into.

Given the BWC nature of the change I'm on the fence whether this should go into 1.4.X - it does fix a concurrency issue and makes things consistent where they weren't.

Closes #6644

@s1monw
Copy link
Contributor

s1monw commented Feb 23, 2015

oooh nice I will review...

}
if (peerRecoveryState != null) {
RecoveryState recoveryState = indexShard.recoveryState();
if (recoveryState == null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

weird... this if has an empty body?

@s1monw
Copy link
Contributor

s1monw commented Feb 24, 2015

left some comments.... I think this should not go into 1.4 since it's pretty big though...

@bleskes
Copy link
Contributor Author

bleskes commented Feb 24, 2015

@s1monw I pushed another round, with more simplifications.

@s1monw
Copy link
Contributor

s1monw commented Feb 25, 2015

I had one minor comments - LGTM otherwise

@s1monw s1monw removed the review label Feb 25, 2015
bleskes added a commit to bleskes/elasticsearch that referenced this pull request Feb 25, 2015
To support the `_recovery` API, the recovery process keeps track of current progress in a class called RecoveryState. This class currently have some issues, mostly around concurrency (see elastic#6644 ). This PR cleans it up as well as other issues around it:

- Make the Index subsection API cleaner:
    - remove redundant information - all calculation is done based on the underlying file map
    - clearer definition of what is what: total files, vs reused files (local files that match the source) vs recovered files (copied over). % based progress is reported based on recovered files only.
    - cleaned up json response to match other API (sadly this breaks the structure). We now properly report human values for dates and other units.
    - Add more robust unit testing
- Detail flag was passed along as state (it's now a ToXContent param)
- State lookup during reporting is now always done via the IndexShard , no more fall backs to many other classes.
- Cleanup APIs around time and move the little computations to the state class as opposed to doing them out of the API

I also improved error messages out of the REST testing infra for things I run into.

Closes elastic#6644
Closes elastic#9811
@bleskes bleskes closed this in 3e32dd9 Feb 25, 2015
@bleskes bleskes deleted the recovery_state_cleanup branch February 25, 2015 16:56
@clintongormley clintongormley added the :Distributed/Recovery Anything around constructing a new shard, either from a local or a remote source. label Mar 19, 2015
@clintongormley clintongormley changed the title Recovery: RecoveryState clean up RecoveryState clean up Jun 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>breaking >bug :Distributed/Recovery Anything around constructing a new shard, either from a local or a remote source. v1.5.0 v2.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants