Skip to content
This repository
branch: release-1.3.1

Mar 25, 2013

  1. abh1nay

    Prepare to release version release-1.3.1

    abh1nay authored
  2. abh1nay

    fixed run-class.sh to take absolute paths

    abh1nay authored
  3. abh1nay

    Updated release number

    abh1nay authored
  4. abh1nay

    Updated release notes

    abh1nay authored
  5. abh1nay

    last commit did not pick up the actual fix for the test case-- checki…

    …ng ti in
    abh1nay authored
  6. abh1nay

    Minor changes to test case# Please enter the commit message for your …

    …changes. Lines starting
    abh1nay authored

Mar 22, 2013

  1. abh1nay

    Added test case to test runtime exceptions

    Cleaned up the try catch  logic
    abh1nay authored
  2. abh1nay

    Adding test cases whch simulate intermittent exceptions etc

    in the hdfsfetcher and simulate retry logic
    this ensures checksum calculation is robust
    abh1nay authored
  3. Adding an extra catch block for Exception and Throwable types. This i…

    …s used to catch the ClassNotFound exceptions
    Chinmay Soman authored abh1nay committed
  4. Using a per file checksum generator in the file copy in HdfsFetcher. …

    …This is used to handle the case where we might retry the copy in case of a Filesystem (hdfs) error.
    Chinmay Soman authored abh1nay committed
  5. Jay Wylie

    Reduce chatter in logs while a node is unavailable (from INFO to DEBU…

    …G level output).
    jayjwylie authored
  6. Jay Wylie

    Fix to log4j set up in bin/run-class.sh that should correctly configu…

    …re log4j.
    jayjwylie authored

Mar 20, 2013

  1. Jay Wylie

    Added pending release notes for this merge to master.

    jayjwylie authored
  2. Jay Wylie

    TODOs for refactoring and copyright header updates.

    jayjwylie authored
  3. Jay Wylie

    Addressed all code review comments for KeySampler and KeyVersionFetch…

    …er. Renamed many classes and methods related to FetchStreamRequestHandler.
    
    - All sub-classes of FetchStreamRequestHandler have been renamed to have a more consistent nomenclature.
    - Did some further refactoring in the FullScan* classes to move more work from leaf classes to FullScanFetchRequestHandler.java
    - moved scan accounting to overall bae class
    - Added getNodesPartitionIdForKey method to StoreInstance to help with some fetch logic
    jayjwylie authored
  4. Jay Wylie

    Interim commit to rename a bunch of files.

    jayjwylie authored
  5. Jay Wylie

    Addressed review feedback and TODOs for KeyVersionSamplerCLI (and ren…

    …amed it to KeyVersionFetcherCLI).
    
    - mostly usability changes about command line options...
    - one copyright fix
    jayjwylie authored
  6. Jay Wylie

    Addressed all review feedback and TODOs for KeySamplerCLI

    KeySamplerCLI
    - added options: --store-names, --partition-ids, --keys-per-second-limit, and --progress-period-ops
    - got rid of unnecessary (and weird) retry loop. Can add seomthing like that later if needed.
    - pass all partitions to fetcher now instead of one-at-a-time
    
    Also did cosmetic fixes for KeyVersionSamplerCLI and Entropy.java
    jayjwylie authored
  7. Jay Wylie

    Correctness fixes and significant refactoring of Fetch*StreamRequestH…

    …andlers. Expanded AdminFetchTest.
    
    Added more common helper methods to common base class of all fetchers FetchStreamRequestHandler.
    
    Added abstract base classes for partition-based fetching and non-partition-based fetching:
    - FetchPartitionStreamRequestHandler (partition-based)
    - FetchItemsStreamRequestHandler (non-partition-based)
    
    Refactored some code up to abstract base classes and made implementations as similar as possible (without heroic efforts) across all fetchers:
    - FetchEntriesStreamRequestHandler
    - FetchKeysStreamRequestHandler
    - FetchPartitionEntriesStreamRequestHandler
    - FetchPartitionKeysStreamRequestHandler
    
    Significant better test coverage in AdminFetchTest
    - tests fetching keys as well as fetching entries
    - tests partition-aware and non-partition-aware servers
    - tests per-partition limits on entries/keys fetched
    
    All of this clean up and additional testing lead to minor correctness fixes.
    
    Minor other clean ups of comments, override annotations, and fixes for KeySamplerCLI.
    jayjwylie authored
  8. Jay Wylie

    change maxRecords to recordsPerPartition in fetch API and protobuf

    These are cosmetic changes. The client-side and server-side code does not properly do recordsPerPartition yet.
    
    Added a few TODOs in the code too.
    jayjwylie authored
  9. Jay Wylie

    remove skipRecords from fetching API and protobuf

    AFAIK skipRecords was never used. By inspection, the code that would have been exercised if it had been used has never been correct. Removing skipRecords from the code base.
    
    Also:
    - Added a number of TODOs to the code from the reviews
    - Changed some variable names
    jayjwylie authored
  10. Jay Wylie

    Minor fixes for tests that broke due to changes elsewhere in the code.

    jayjwylie authored
  11. Jay Wylie

    Minor fix for change to AdminClient

    src/java/voldemort/client/protocol/admin/AdminClient.java
    - do not close down AdminStoreClient from queryKeys
    
    test/unit/voldemort/client/AdminServiceBasicTest.java
    - added some additional checks to test to confirm (non)existence of exceptions&values
    jayjwylie authored
  12. Jay Wylie

    Many minor tweaks to ConsistencyFix code and related files to address…

    … minor review feedback.
    
    build.xml
    - fixed commenting out of 'protobuff' target
    
    src/java/voldemort/client/protocol/admin/AdminClient.java
    - add ClientConfig to constructor. This is needed for AdminStoreClient creation. It is confusing that we need both an AdminClientConfig and ClientConfig, but that is because the *ClientConfig code is so clumsy.
    - changed ".stop()" methods to ".close()" to be consistent with other interfaces.
    
    et cetera
    - Updated all copyright notices that have changed on this branch since December. This touched a ton of files...
    - annotated some TODOs with "(refactor)" to make refactoring todos easier to find.
    jayjwylie authored
  13. Jay Wylie

    Added unit tests for ConsistencyFix, ConsistencyFixWorker, and QueryK…

    …eyResult.
    
    Many other fixes and cleanup:
    
    src/java/voldemort/utils/ConsistencyFix.java
    - tweak many variable names
    - add close method to stop adminClient
    - broke out BadKey to wrap a key with its string representation st failed fixes of badkey's can be dumped in full to file to be retried (without any additional effort)
    - marked 'parseVersion' as deprecated since, if we do this again, we should dump bytes not strings
    - track obsolete version exceptions and various statuses in Stats
    
    src/java/voldemort/utils/ConsistencyFixCLI.java
    - clean up of arguments, variable names, etc.
    - cleanly close down fixer...
    
    src/java/voldemort/utils/ConsistencyFixWorker.java
    - more logger.trace output
    - minor cleanup
    
    test/common/voldemort/TestUtils.java
    - added getVersioned() helper method
    
    test/common/voldemort/config/stores.xml
    - added consistency-fix store
    
    test/unit/voldemort/store/routed/ReadRepairerTest.java
    - marked all tests as @Test
    
    test/unit/voldemort/utils/ConsistencyCheckTest.java
    - update copyright notice
    jayjwylie authored
  14. Jay Wylie

    Actually adding files KeySamplerCLI and KeyVersionSamplerCLI.

    jayjwylie authored
  15. Jay Wylie

    Documented correct method of compiling protobuffs by hand.

    jayjwylie authored
  16. Jay Wylie

    Added KeySampler and KeyVersionSampler tools as a first step towards …

    …replacing "entropy" tool. Added another argument to bulk fetch operations that specifies maxRecords so that server can fetch a subset of a partition.
    
    src/java/voldemort/utils/KeySamplerCLI.java
    - Samples keys from a cluster
    
    src/java/voldemort/utils/KeyVersionSamplerCLI.java
    - Given file that lists keys per store, samples versions from each "responsible node" for that key
    
    src/java/voldemort/client/protocol/admin/AdminClient.java
    - passed maxRecords through
    - TODO for future clean up of some types
    
    src/java/voldemort/client/protocol/pb/VAdminProto.java
    - auto generated!
    
    src/java/voldemort/server/protocol/admin/AdminServiceRequestHandler.java
    - white space
    
    src/java/voldemort/server/protocol/admin/FetchStreamRequestHandler.java
    src/java/voldemort/server/protocol/admin/FetchEntriesStreamRequestHandler.java
    src/java/voldemort/server/protocol/admin/FetchKeysStreamRequestHandler.java
    - handle maxRecords
    
    src/java/voldemort/server/protocol/admin/FetchPartitionKeysStreamRequestHandler.java
    src/java/voldemort/server/protocol/admin/FetchPartitionEntriesStreamRequestHandler.java
    - handle maxRecords
    - fixed usage of skipRecords
    
    src/java/voldemort/utils/Entropy.java
    - added maxRecords
    
    src/proto/voldemort-admin.proto
    - added mac_records to protobuff definition
    
    test/unit/voldemort/client/AdminFetchTest.java
    - added maxRecords field to test
    jayjwylie authored
  17. Jay Wylie

    Made rebalance --show-plan slightly more verbose and added yet anothe…

    …r analysis for cluster balance ("zone primary").
    
    src/java/voldemort/client/rebalance/RebalancePartitionsInfo.java
    - print out hostname within plan to make it easier to read (rather than having to lookup node ID)
    
    src/java/voldemort/utils/ClusterInstance.java
    - calculate "zone primary" balance to understand which hosted partitions act as pseudo-master when zoned routing is used.
    jayjwylie authored
  18. Jay Wylie

    Review and cleanup of consistency checker.

    - added required argument for an output file name for bad keys
    - changed Reporter to print out 'just the key' to the output file; it
      outputs more info at DEBUG level in general.
    - removed 'quiet' option
    - throw exceptions:
      - if # partitions differ across clusters
      - if replication factor is hinky
      - if isExpired encounters unknown type
    - main catches exceptions and fails fast
    - changed system.out debugging to logger.trace
    jayjwylie authored
  19. Jay Wylie

    Added more info to cluster dump to track which nodes host 'hot' parti…

    …tions.
    jayjwylie authored
  20. Jay Wylie

    Tweaked Rebalancer --output-dir again to better name interim metadata…

    … files for each batch.
    jayjwylie authored
  21. Jay Wylie

    Changed Rebalancer --output-dir option to append numbers to each .xml…

    … file it outputs so that we have access to interim cluster configs.
    jayjwylie authored
  22. Zhongjie Wu

    Refactored Consistency Check

    zhongjiewu authored jayjwylie committed
  23. Jay Wylie

    Added server-put tracking to progress bar.

    jayjwylie authored
Something went wrong with that request. Please try again.