Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow to trim all ops above a certain seq# with a term lower than X #30176

Merged
merged 54 commits into from
Jun 8, 2018
Merged

Allow to trim all ops above a certain seq# with a term lower than X #30176

merged 54 commits into from
Jun 8, 2018

Commits on Apr 30, 2018

  1. Allow to trim all ops above a certain seq# with a term lower than X 

    Relates to elastic#10708
    @vladimirdolzhenko
    vladimirdolzhenko committed Apr 30, 2018
    Configuration menu
    Copy the full SHA
    d3d210a View commit details
    Browse the repository at this point in the history

Commits on May 4, 2018

  1. Allow to trim all ops above a certain seq# with a term lower than X 

    Relates to elastic#10708
    Vladimir Dolzhenko committed May 4, 2018
    Configuration menu
    Copy the full SHA
    e622794 View commit details
    Browse the repository at this point in the history

  2. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … fixes on PR#2

Relates to elastic#10708
    Vladimir Dolzhenko committed May 4, 2018
    Configuration menu
    Copy the full SHA
    586908c View commit details
    Browse the repository at this point in the history

  3. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … fixes on PR#2

Relates to elastic#10708
    Vladimir Dolzhenko committed May 4, 2018
    Configuration menu
    Copy the full SHA
    92addd1 View commit details
    Browse the repository at this point in the history

  4. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … fixes on PR elastic#3

Relates to elastic#10708
    Vladimir Dolzhenko committed May 4, 2018
    Configuration menu
    Copy the full SHA
    2a9c58c View commit details
    Browse the repository at this point in the history

  5. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … fixes on PR elastic#4

Relates to elastic#10708
    Vladimir Dolzhenko committed May 4, 2018
    Configuration menu
    Copy the full SHA
    1e67a8a View commit details
    Browse the repository at this point in the history

  6. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … fixes on PR elastic#5

Relates to elastic#10708
    Vladimir Dolzhenko committed May 4, 2018
    Configuration menu
    Copy the full SHA
    d454a30 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2018

  1. merged from master

    Vladimir Dolzhenko committed May 7, 2018
    Configuration menu
    Copy the full SHA
    92d2d68 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2018

  1. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … fixes for the case of no entities

Relates to elastic#10708
    Vladimir Dolzhenko committed May 8, 2018
    Configuration menu
    Copy the full SHA
    13ae045 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2018

  1. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … fixes for PR#6

Relates to elastic#10708
    Vladimir Dolzhenko committed May 10, 2018
    Configuration menu
    Copy the full SHA
    762f298 View commit details
    Browse the repository at this point in the history

  2. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … fixes for PR#6

Relates to elastic#10708
    Vladimir Dolzhenko committed May 10, 2018
    Configuration menu
    Copy the full SHA
    3896280 View commit details
    Browse the repository at this point in the history

Commits on May 11, 2018

  1. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

added TODO note for test regarding Lucene rollback
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    2c13980 View commit details
    Browse the repository at this point in the history

  2. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

dropped extra debug info of replicas seq# stats
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    49167c8 View commit details
    Browse the repository at this point in the history

  3. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

check source of a doc after trimming
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    eaef2e4 View commit details
    Browse the repository at this point in the history

  4. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

minTranslogGeneration has NOT to refer to SeqNumbers
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    ef7811d View commit details
    Browse the repository at this point in the history

  5. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

add check that translog is not corrupted after any IO exception
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    928ac1e View commit details
    Browse the repository at this point in the history

  6. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

don't randomize max attempts to get into random IO exception
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    fd6e16e View commit details
    Browse the repository at this point in the history

  7. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

moved ensureOpen check and term check from TranslogReader to Translog
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    65604be View commit details
    Browse the repository at this point in the history

  8. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

dropped redundant belowTerm paremter from reader.closeIntoTrimmedReader
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    c215e6c View commit details
    Browse the repository at this point in the history

  9. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

added test case with more attempts - random roll generation, checks for primary terms
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    7c33612 View commit details
    Browse the repository at this point in the history

  10. Allow to trim all ops above a certain seq# with a term lower than X -… 

    … Relates to elastic#10708

fix test to pass codestyle
    Vladimir Dolzhenko committed May 11, 2018
    Configuration menu
    Copy the full SHA
    6a41d7b View commit details
    Browse the repository at this point in the history

Commits on May 16, 2018

  1. move translog checks under the lock, prevent readers being double closed

    Vladimir Dolzhenko committed May 16, 2018
    Configuration menu
    Copy the full SHA
    1afd5be View commit details
    Browse the repository at this point in the history

  2. we intend to break translog - make chance of failure proportional to … 

    …number of attempt, use normal translog to verify translog ops
    Vladimir Dolzhenko committed May 16, 2018
    Configuration menu
    Copy the full SHA
    96ff7aa View commit details
    Browse the repository at this point in the history

  3. some test code cleanup - create and add operations to translog in the… 

    … same place, handle expected IOException testRandomExceptionsOnTrimOperations more clearly
    Vladimir Dolzhenko committed May 16, 2018
    Configuration menu
    Copy the full SHA
    42aec35 View commit details
    Browse the repository at this point in the history

  4. added testSyncerSendsMaxSeqNo

    Vladimir Dolzhenko committed May 16, 2018
    Configuration menu
    Copy the full SHA
    bdbcffa View commit details
    Browse the repository at this point in the history

  5. extend test to check that trimAboveSeqNo goes only in the 0th request

    Vladimir Dolzhenko committed May 16, 2018
    Configuration menu
    Copy the full SHA
    51cf6d6 View commit details
    Browse the repository at this point in the history

  6. add assertion on assumption that translog.current has no any op seq# … 

    …> trimmedMaxSeqNo, extended test with duplicates with the same (seq#, term)
    Vladimir Dolzhenko committed May 16, 2018
    Configuration menu
    Copy the full SHA
    963f7b6 View commit details
    Browse the repository at this point in the history

  7. fix broken test

    Vladimir Dolzhenko committed May 16, 2018
    Configuration menu
    Copy the full SHA
    dbb9b13 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2018

  1. rebuild assertion check on assumption that translog.current has no an… 

    …y op seq# > trimmedMaxSeqNo
    Vladimir Dolzhenko committed May 17, 2018
    Configuration menu
    Copy the full SHA
    edc4fe6 View commit details
    Browse the repository at this point in the history

  2. extend test with duplicates <same seq#, lower primaryTerm>

    Vladimir Dolzhenko committed May 17, 2018
    Configuration menu
    Copy the full SHA
    2f99f3b View commit details
    Browse the repository at this point in the history

Commits on May 20, 2018

  1. fixed assertNoSeqAbove - op.primaryTerm < belowTerm; small cosmetic f… 

    …ixes - IAE message, comment for the reason of assertNoSeqAbove
    Vladimir Dolzhenko committed May 20, 2018
    Configuration menu
    Copy the full SHA
    dac5945 View commit details
    Browse the repository at this point in the history

  2. dropped testSyncerSendsMaxSeqNo as all tested functionality is covere… 

    …d in testSyncerSendsOffCorrectDocuments
    Vladimir Dolzhenko committed May 20, 2018
    Configuration menu
    Copy the full SHA
    cd19157 View commit details
    Browse the repository at this point in the history

  3. assertNoSeqAbove has to take into account seq > aboveSeqNo rather >=

    Vladimir Dolzhenko committed May 20, 2018
    Configuration menu
    Copy the full SHA
    4060cd7 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2018

  1. dropped unnecessary seq# check in assertNoSeqAbove

    Vladimir Dolzhenko committed May 21, 2018
    Configuration menu
    Copy the full SHA
    8075a7d View commit details
    Browse the repository at this point in the history

  2. use the most pessimistic fail rate approach in testRandomExceptionsOn… 

    …TrimOperations
    Vladimir Dolzhenko committed May 21, 2018
    Configuration menu
    Copy the full SHA
    1299112 View commit details
    Browse the repository at this point in the history

  3. simplified testSnapshotTrimmedOperations

    Vladimir Dolzhenko committed May 21, 2018
    Configuration menu
    Copy the full SHA
    f7d8f52 View commit details
    Browse the repository at this point in the history

  4. fixing testSnapshotTrimmedOperations: higher term with the same seq# … 

    …has to win
    Vladimir Dolzhenko committed May 21, 2018
    Configuration menu
    Copy the full SHA
    d4fb32b View commit details
    Browse the repository at this point in the history

Commits on May 22, 2018

  1. fixing testSnapshotTrimmedOperations: duplicates could have only forw… 

    …ard-or-the-same-as-last-one primaryTerms
    Vladimir Dolzhenko committed May 22, 2018
    Configuration menu
    Copy the full SHA
    5c565c4 View commit details
    Browse the repository at this point in the history

  2. Revert "rebuild assertion check on assumption that translog.current h… 

    …as no any op seq# > trimmedMaxSeqNo" - assertNoSeqAbove relies in-memory map of current operations
    Vladimir Dolzhenko committed May 22, 2018
    Configuration menu
    Copy the full SHA
    5a88b65 View commit details
    Browse the repository at this point in the history

  3. addressing case of trimming trimmed translog

    Vladimir Dolzhenko committed May 22, 2018
    Configuration menu
    Copy the full SHA
    4b6da39 View commit details
    Browse the repository at this point in the history

  4. fixed testSnapshotCurrentHasUnexpectedOperationsForTrimmedOperations:… 

    … don't create new tranlog that shares files with original one
    Vladimir Dolzhenko committed May 22, 2018
    Configuration menu
    Copy the full SHA
    108f8af View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'remotes/origin/master' into trim_translog

    Vladimir Dolzhenko committed May 22, 2018
    Configuration menu
    Copy the full SHA
    cbba0b1 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2018

  1. aboveSeqNo has to be aboveOrEqSeqNo to be able to trim op those are s… 

    …trictly less than aboveOrEqSeqNo; addressed code style issues in tests
    Vladimir Dolzhenko committed May 23, 2018
    Configuration menu
    Copy the full SHA
    cdbb0a1 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2018

  1. reverted back to aboveSeqNo, make NO_OPS_PERFORMED valid seq#

    Vladimir Dolzhenko committed May 24, 2018
    Configuration menu
    Copy the full SHA
    147e709 View commit details
    Browse the repository at this point in the history

  2. simplified testSnapshotTrimmedOperations and dropped testSnapshotTrim… 

    …TrimmedOperations as testSnapshotTrimmedOperations includes this behaviour
    Vladimir Dolzhenko committed May 24, 2018
    Configuration menu
    Copy the full SHA
    dc9c2dd View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'remotes/origin/master' into trim_translog

    Vladimir Dolzhenko committed May 24, 2018
    Configuration menu
    Copy the full SHA
    7a0750d View commit details
    Browse the repository at this point in the history

Commits on May 25, 2018

  1. Merge remote-tracking branch 'remotes/origin/master' into trim_translog

    Vladimir Dolzhenko committed May 25, 2018
    Configuration menu
    Copy the full SHA
    9cb43c7 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2018

  1. streamification; InMemoryTranslog does not delegate anything to translog

    Vladimir Dolzhenko committed May 30, 2018
    Configuration menu
    Copy the full SHA
    99ae2df View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2018

  1. Merge remote-tracking branch 'remotes/origin/master' into trim_translog

    Vladimir Dolzhenko committed Jun 2, 2018
    Configuration menu
    Copy the full SHA
    ba339d9 View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2018

  1. keep operationsList and currentOperations apart

    Vladimir Dolzhenko committed Jun 5, 2018
    Configuration menu
    Copy the full SHA
    0f9b206 View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2018

  1. drop assumption on not trimming current in InMemoryTranslog

    Vladimir Dolzhenko committed Jun 6, 2018
    Configuration menu
    Copy the full SHA
    9d74767 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'remotes/origin/master' into trim_translog

    Vladimir Dolzhenko committed Jun 6, 2018
    Configuration menu
    Copy the full SHA
    546c890 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2018

  1. Merge remote-tracking branch 'remotes/origin/master' into trim_translog

    Vladimir Dolzhenko committed Jun 7, 2018
    Configuration menu
    Copy the full SHA
    db4105f View commit details
    Browse the repository at this point in the history

  2. InMemoryTranslog simplification

    Vladimir Dolzhenko committed Jun 7, 2018
    Configuration menu
    Copy the full SHA
    88f702b View commit details
    Browse the repository at this point in the history