Skip to content
This repository
branch: bz1160-eunit-t…

Feb 09, 2012

  1. Jon Meredith

    On failed merge, print failed file and contine with the rest.

    jonmeredith authored

Feb 08, 2012

  1. Dave Smith

    Cleanup error handling when merge fails; track merged/input files sep…

    …arately and only delete those files that merged completely.
    dizzyd authored jonmeredith committed
  2. Daniel Reverri

    Test case for merge failing if a bad CRC is found.

    It is unclear what Bitcask *should* do in this scenario. One suggestion is to
    move on to the next file in the merge process, move the corrupted file into a
    "corrupted" directory, and complete the merge.
    dreverri authored jonmeredith committed
  3. Jon Meredith

    Merge branch 'az1077-corrupted-hintfile-recovery'

    jonmeredith authored
  4. Jon Meredith

    Merge branch 'jdm-fix-put-at-merge-time'

    jonmeredith authored

Feb 06, 2012

  1. Jon Meredith

    Added require_hint_crc entry to bitcask.app.src and enabled for eqc t…

    …est.
    
    Without the setting EQC finds and edge case where it truncates
    the hint file on a hint record boundary.  required_hint_crc will
    be set to true by default in a future release.
    jonmeredith authored
  2. Jon Meredith

    Removed original dead hintfile creation code.

    In the distant past casks were written, then reread and a hint file
    was written for them.  Now hints are written concurrently with the
    main cask file so all of this code is dead.
    jonmeredith authored
  3. Jon Meredith

    Added env var to require strict hintfile checking.

    Otherwise it is possible to truncate at the end of a hintfile
    record and be treated as a hint file without CRCs.
    jonmeredith authored
  4. Jon Meredith

    Updated truncated_datafile_test for hintfile changes.

    With the new 'recovery' option for keyfolding, the cask files
    are scanned after the hint file read discovers an object beyond
    the end of the cask file.
    jonmeredith authored
  5. Jon Meredith

    Fixed truncated_merge_test.

    The truncated hint files are now detected so only the two
    data file corruptions are bad.
    jonmeredith authored
  6. Jon Meredith

    Added check for hintfile CRC on startup.

    jonmeredith authored
  7. Jon Meredith

    Fixed typo in badcrc error return. Found a new counterexample.

    ```
    [{set,{var,1},{call,bitcask_qc_fsm,set_keys,[[<<"hØv">>]]}},
     {set,{var,2},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,none}]]}},
     {set,{var,10},{call,bitcask,delete,[{var,2},<<"hØv">>]}},
     {set,{var,30},{call,bitcask,close,[{var,2}]}},
     {set,{var,45},{call,bitcask_qc_fsm,corrupt_hint,[0,37]}},
     {set,{var,47},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,none}]]}},
     {set,{var,48},{call,bitcask,put,[{var,47},<<"hØv">>,<<>>]}}]
    {exception,{'EXIT',{{badmatch,already_exists},
                        [{bitcask,put,3},
                         {eqc_statem,f515_0,5},
                         {eqc_statem,f507_0,5},
                         {eqc_statem,run_commands,2},
                         {eqc_fsm,run_commands,2},
                         {bitcask_qc_fsm,'-prop_bitcask/0-fun-0-',1},
                         {eqc,'-f777_0/2-fun-4-',3},
                         {eqc_gen,'-f321_0/2-fun-0-',5}]}}} /= ok
    false
    ```
    jonmeredith authored
  8. Jon Meredith

    Added hintfile corruption test and found counterexample.

    ```
    12> eqc:check(bitcask_qc_fsm:prop_bitcask(), eqc:current_counterexample()).
    Corrupting from 112 to <<143>> at 23 size 42
    
    =ERROR REPORT==== 2-Feb-2012::10:11:15 ===
    Hintfile '/tmp/bitcask.qc/1.bitcask.hint' has bad CRC 486109250 expected 838574031
    Failed!
    [{set,{var,1},{call,bitcask_qc_fsm,set_keys,[[<<"±#ª:×p">>]]}},
     {set,{var,2},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,o_sync}]]}},
     {set,{var,8},{call,bitcask,put,[{var,2},<<"±#ª:×p">>,<<>>]}},
     {set,{var,34},{call,bitcask,close,[{var,2}]}},
     {set,{var,39},{call,bitcask_qc_fsm,corrupt_hint,[0,18]}},
     {set,{var,42},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,none}]]}},
     {set,{var,43},{call,bitcask,get,[{var,42},<<"±#ª:×p">>]}}]
    {postcondition,{expected,<<>>,got,not_found}} /= ok
    false
    ```
    jonmeredith authored
  9. Jon Meredith

    Add CRC to hintfile and check during startup.

    Calculate CRC of hintfile as it is written and write to
    special backward compatible trailer record at the end of
    the hints file.
    
    Changed the hint folding code to return {error, blah} more
    reliably and added a new fold_keys variant that will
    switch to scanning the cask files if there were any issues
    with the hint file.
    jonmeredith authored
  10. Jon Meredith

    Added hintfile truncation to bitcask_qc_fsm

    jonmeredith authored
  11. Jon Meredith

    Removed commented out code.

    Cleanups from review - thanks Zman.
    jonmeredith authored
  12. Andrew Thompson

    Merge pull request #33 from basho/adt-fix-eqc-includes

    Don't use include_lib for local include files.
    Vagabond authored
  13. Andrew Thompson

    Don't use include_lib for local include files.

    This breaks the EQC tests when the toplevel dir is not called 'bitcask'.
    Vagabond authored
  14. Jon Meredith

    Changed put to wrap write file and retry on already_exists.

    Resolves issue discovered by bitcask_qc_fsm EQC test.  If
    the merge rewrites keys during the same second as an incoming
    put, the incoming was failing with already_exists when writing
    to the keydir.
    
    Now, bitcask will roll over to a new file and reattempt the
    put once as the put should succeed now the writer process
    has a greater file_id than the merge process.
    jonmeredith authored
  15. Jon Meredith

    Changed the way keys are generated in bitcask_qc_fsm to overlap more.

    Have counterexample
    
    32> eqc:check(bitcask_qc_fsm:prop_bitcask(), eqc:current_counterexample()).
    Failed!
    [{set,{var,1},{call,bitcask_qc_fsm,set_keys,[[<<"\r'Æb¾Û">>]]}},
     {set,{var,3},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,none}]]}},
     {set,{var,28},{call,bitcask,put,[{var,3},<<"\r'Æb¾Û">>,<<>>]}},
     {set,{var,31},{call,bitcask,close,[{var,3}]}},
     {set,{var,32},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,o_sync}]]}},
     {set,{var,46},{call,bitcask,put,[{var,32},<<"k">>,<<>>]}},
     {set,{var,47},{call,bitcask,merge,["/tmp/bitcask.qc"]}},
     {set,{var,48},{call,bitcask,put,[{var,32},<<"\r'Æb¾Û">>,<<>>]}}]
    {exception,{'EXIT',{{badmatch,already_exists},
                        [{bitcask,put,3},
                         {eqc_statem,f515_0,5},
                         {eqc_statem,f507_0,5},
                         {eqc_statem,run_commands,2},
                         {eqc_fsm,run_commands,2},
                         {bitcask_qc_fsm,'-prop_bitcask/0-fun-0-',1},
                         {eqc,'-f777_0/2-fun-4-',3},
                         {eqc_gen,'-f321_0/2-fun-0-',5}]}}} /= ok
    false
    jonmeredith authored
  16. Jon Meredith

    Merge remote-tracking branch 'hdima/read_check_crc_first'

    jonmeredith authored

Feb 04, 2012

  1. Dmitry Vasiliev

    On read CRC must be checked first to eliminate possible 'badmatch' er…

    …ror.
    
    See #30 for details.
    hdima authored

Jan 12, 2012

  1. Jared Morrow

    Bump to version 1.4.1

    jaredmorrow authored

Jan 11, 2012

  1. Jon Meredith

    Merge pull request #29 from basho/jdm-fix-osx32-bit-off_t

    Decode off_t argument more carefully for pread/pwrite calls.
    jonmeredith authored
  2. Jon Meredith

    Decode off_t argument more carefully for pread/pwrite calls.

    On some platforms sizeof(off_t)==8, sizeof(unsigned long) == 4
    so the pos argument for read and write is half garbage.
    jonmeredith authored

Dec 28, 2011

  1. Dave Smith

    Merge pull request #27 from hdima/fold_loop_fixes

    Properly handle EOF case in bitcask_fileops:fold_loop()
    dizzyd authored
  2. Dave Smith

    Merge pull request #28 from hdima/gitignore_fixes

    Added 'ebin' to the list of ignored files
    dizzyd authored
  3. Dmitry Vasiliev

    Added 'ebin' to the list of ignored files

    hdima authored
  4. Dmitry Vasiliev

    Properly handle EOF case in bitcask_fileops:fold_loop()

    hdima authored

Dec 27, 2011

  1. Dave Smith

    Bumping to 1.4.0 in prep to merge w/ master

    dizzyd authored
  2. Dave Smith

    Fix bug where writing file was considered for merging; add clarifying…

    … comments to small_file_threshold
    
    Yay for code review!
    dizzyd authored
  3. Dave Smith

    Remove expired entries from keydir on read

    dizzyd authored
  4. Dave Smith

    Add QC test for validating new needs_merge

    dizzyd authored
  5. Dave Smith

    Add ability to close current writing file for testing; rework merging

    to provide info when selecting files to merge.
    
    Adding {log_needs_merge, true} to the .app will enable this logging.
    dizzyd authored
  6. Dave Smith

    Add dependency on meck for testing expiration

    dizzyd authored
Something went wrong with that request. Please try again.