Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make sorted recovery write to RFiles #2117

Merged
merged 29 commits into from Jun 21, 2021
Merged

Make sorted recovery write to RFiles #2117

merged 29 commits into from Jun 21, 2021
+436 −109

Commits on Jun 17, 2021

  1. Change sorted WALs to write to RFiles 

    Create LogFileKey.toKey() and LogFileValue.toValue() methods

Make RecoveryLogsIterator iterate over Key Value

Add fromKey and fromValue methods for parsing from Key

Rewrite LogSorter writeBuffer method and make static to test
    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    8a52be2 View commit details
    Browse the repository at this point in the history

  2. Improve writeBuffer method

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    5d1920a View commit details
    Browse the repository at this point in the history

  3. Revert timeout increase

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    e546835 View commit details
    Browse the repository at this point in the history

  4. Add log path to exception

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    aad476a View commit details
    Browse the repository at this point in the history

  5. Rework RecoveryLogsIterator and SortedLogRecoveryTest for multipart logs

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    a1ba8f3 View commit details
    Browse the repository at this point in the history

  6. Only check first key once, before setting scan range

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    b69ebee View commit details
    Browse the repository at this point in the history

  7. Clean up SortedLogRecoveryTest

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    42410a5 View commit details
    Browse the repository at this point in the history

  8. Create scanner per WAL part for better memory usage

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    84785a6 View commit details
    Browse the repository at this point in the history

  9. Make logging consistent with old recovery

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    ef1d2c5 View commit details
    Browse the repository at this point in the history

  10. Refactor to pass server context to provide configuration

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    91fbd0b View commit details
    Browse the repository at this point in the history

  11. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Keith Turner <kturner@apache.org>
    @milleruntime @keith-turner
    milleruntime and keith-turner committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    4afab5e View commit details
    Browse the repository at this point in the history

  12. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Keith Turner <kturner@apache.org>
    @milleruntime @keith-turner
    milleruntime and keith-turner committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    7c0e9f2 View commit details
    Browse the repository at this point in the history

  13. Use KeyBuilder to store qualifier bytes so I don't need base64

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    26abf73 View commit details
    Browse the repository at this point in the history

  14. Use dummy filename for start and end keys

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    ec1eaae View commit details
    Browse the repository at this point in the history

  15. Format the row using bytes

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    57b6357 View commit details
    Browse the repository at this point in the history

  16. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Keith Turner <kturner@apache.org>
    @milleruntime @keith-turner
    milleruntime and keith-turner committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    e0e0201 View commit details
    Browse the repository at this point in the history

  17. Fixes

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    b55e14c View commit details
    Browse the repository at this point in the history

  18. Updates from PR

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    231397d View commit details
    Browse the repository at this point in the history

  19. Remove setting extent in min and max keys

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    d688522 View commit details
    Browse the repository at this point in the history

  20. Some refactoring from PR

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    ff53d8a View commit details
    Browse the repository at this point in the history

  21. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Keith Turner <kturner@apache.org>
    @milleruntime @keith-turner
    milleruntime and keith-turner committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    0609f37 View commit details
    Browse the repository at this point in the history

  22. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Keith Turner <kturner@apache.org>
    @milleruntime @keith-turner
    milleruntime and keith-turner committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    05014a3 View commit details
    Browse the repository at this point in the history

  23. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Keith Turner <kturner@apache.org>
    @milleruntime @keith-turner
    milleruntime and keith-turner committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    f47faec View commit details
    Browse the repository at this point in the history

  24. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Keith Turner <kturner@apache.org>
    @milleruntime @keith-turner
    milleruntime and keith-turner committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    2f40e3e View commit details
    Browse the repository at this point in the history

  25. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Keith Turner <kturner@apache.org>
    @milleruntime @keith-turner
    milleruntime and keith-turner committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    7a147ae View commit details
    Browse the repository at this point in the history

  26. Drop colFamToFetch

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    e831472 View commit details
    Browse the repository at this point in the history

  27. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileValue.java

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
    @milleruntime @ctubbsii
    milleruntime and ctubbsii committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    352af97 View commit details
    Browse the repository at this point in the history

  28. Update server/tserver/src/main/java/org/apache/accumulo/tserver/logge… 

    …r/LogFileKey.java

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
    @milleruntime @ctubbsii
    milleruntime and ctubbsii committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    6cd0ef8 View commit details
    Browse the repository at this point in the history

  29. CR updates

    @milleruntime
    milleruntime committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    6fc3582 View commit details
    Browse the repository at this point in the history