Skip to content
Commits on Feb 9, 2012
  1. @jonmeredith
Commits on Feb 8, 2012
  1. @dizzyd @jonmeredith

    Cleanup error handling when merge fails; track merged/input files sep…

    …arately and only delete those files that merged completely.
    dizzyd committed with jonmeredith Jan 16, 2012
  2. @dreverri @jonmeredith

    Test case for merge failing if a bad CRC is found.

    It is unclear what Bitcask *should* do in this scenario. One suggestion is to
    move on to the next file in the merge process, move the corrupted file into a
    "corrupted" directory, and complete the merge.
    dreverri committed with jonmeredith Jan 16, 2012
  3. @jonmeredith
  4. @jonmeredith
Commits on Feb 6, 2012
  1. @jonmeredith

    Added require_hint_crc entry to bitcask.app.src and enabled for eqc t…

    …est.
    
    Without the setting EQC finds and edge case where it truncates
    the hint file on a hint record boundary.  required_hint_crc will
    be set to true by default in a future release.
    jonmeredith committed Feb 6, 2012
  2. @jonmeredith

    Removed original dead hintfile creation code.

    In the distant past casks were written, then reread and a hint file
    was written for them.  Now hints are written concurrently with the
    main cask file so all of this code is dead.
    jonmeredith committed Feb 6, 2012
  3. @jonmeredith

    Added env var to require strict hintfile checking.

    Otherwise it is possible to truncate at the end of a hintfile
    record and be treated as a hint file without CRCs.
    jonmeredith committed Feb 6, 2012
  4. @jonmeredith

    Updated truncated_datafile_test for hintfile changes.

    With the new 'recovery' option for keyfolding, the cask files
    are scanned after the hint file read discovers an object beyond
    the end of the cask file.
    jonmeredith committed Feb 5, 2012
  5. @jonmeredith

    Fixed truncated_merge_test.

    The truncated hint files are now detected so only the two
    data file corruptions are bad.
    jonmeredith committed Feb 5, 2012
  6. @jonmeredith
  7. @jonmeredith

    Fixed typo in badcrc error return. Found a new counterexample.

    ```
    [{set,{var,1},{call,bitcask_qc_fsm,set_keys,[[<<"hØv">>]]}},
     {set,{var,2},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,none}]]}},
     {set,{var,10},{call,bitcask,delete,[{var,2},<<"hØv">>]}},
     {set,{var,30},{call,bitcask,close,[{var,2}]}},
     {set,{var,45},{call,bitcask_qc_fsm,corrupt_hint,[0,37]}},
     {set,{var,47},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,none}]]}},
     {set,{var,48},{call,bitcask,put,[{var,47},<<"hØv">>,<<>>]}}]
    {exception,{'EXIT',{{badmatch,already_exists},
                        [{bitcask,put,3},
                         {eqc_statem,f515_0,5},
                         {eqc_statem,f507_0,5},
                         {eqc_statem,run_commands,2},
                         {eqc_fsm,run_commands,2},
                         {bitcask_qc_fsm,'-prop_bitcask/0-fun-0-',1},
                         {eqc,'-f777_0/2-fun-4-',3},
                         {eqc_gen,'-f321_0/2-fun-0-',5}]}}} /= ok
    false
    ```
    jonmeredith committed Feb 2, 2012
  8. @jonmeredith

    Added hintfile corruption test and found counterexample.

    ```
    12> eqc:check(bitcask_qc_fsm:prop_bitcask(), eqc:current_counterexample()).
    Corrupting from 112 to <<143>> at 23 size 42
    
    =ERROR REPORT==== 2-Feb-2012::10:11:15 ===
    Hintfile '/tmp/bitcask.qc/1.bitcask.hint' has bad CRC 486109250 expected 838574031
    Failed!
    [{set,{var,1},{call,bitcask_qc_fsm,set_keys,[[<<"±#ª:×p">>]]}},
     {set,{var,2},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,o_sync}]]}},
     {set,{var,8},{call,bitcask,put,[{var,2},<<"±#ª:×p">>,<<>>]}},
     {set,{var,34},{call,bitcask,close,[{var,2}]}},
     {set,{var,39},{call,bitcask_qc_fsm,corrupt_hint,[0,18]}},
     {set,{var,42},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,none}]]}},
     {set,{var,43},{call,bitcask,get,[{var,42},<<"±#ª:×p">>]}}]
    {postcondition,{expected,<<>>,got,not_found}} /= ok
    false
    ```
    jonmeredith committed Feb 2, 2012
  9. @jonmeredith

    Add CRC to hintfile and check during startup.

    Calculate CRC of hintfile as it is written and write to
    special backward compatible trailer record at the end of
    the hints file.
    
    Changed the hint folding code to return {error, blah} more
    reliably and added a new fold_keys variant that will
    switch to scanning the cask files if there were any issues
    with the hint file.
    jonmeredith committed Feb 1, 2012
  10. @jonmeredith
  11. @jonmeredith

    Removed commented out code.

    Cleanups from review - thanks Zman.
    jonmeredith committed Feb 6, 2012
  12. @Vagabond

    Merge pull request #33 from basho/adt-fix-eqc-includes

    Don't use include_lib for local include files.
    Vagabond committed Feb 6, 2012
  13. @Vagabond

    Don't use include_lib for local include files.

    This breaks the EQC tests when the toplevel dir is not called 'bitcask'.
    Vagabond committed Feb 6, 2012
  14. @jonmeredith

    Changed put to wrap write file and retry on already_exists.

    Resolves issue discovered by bitcask_qc_fsm EQC test.  If
    the merge rewrites keys during the same second as an incoming
    put, the incoming was failing with already_exists when writing
    to the keydir.
    
    Now, bitcask will roll over to a new file and reattempt the
    put once as the put should succeed now the writer process
    has a greater file_id than the merge process.
    jonmeredith committed Feb 6, 2012
  15. @jonmeredith

    Changed the way keys are generated in bitcask_qc_fsm to overlap more.

    Have counterexample
    
    32> eqc:check(bitcask_qc_fsm:prop_bitcask(), eqc:current_counterexample()).
    Failed!
    [{set,{var,1},{call,bitcask_qc_fsm,set_keys,[[<<"\r'Æb¾Û">>]]}},
     {set,{var,3},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,none}]]}},
     {set,{var,28},{call,bitcask,put,[{var,3},<<"\r'Æb¾Û">>,<<>>]}},
     {set,{var,31},{call,bitcask,close,[{var,3}]}},
     {set,{var,32},
          {call,bitcask,open,
                ["/tmp/bitcask.qc",
                 [read_write,{open_timeout,0},{sync_strategy,o_sync}]]}},
     {set,{var,46},{call,bitcask,put,[{var,32},<<"k">>,<<>>]}},
     {set,{var,47},{call,bitcask,merge,["/tmp/bitcask.qc"]}},
     {set,{var,48},{call,bitcask,put,[{var,32},<<"\r'Æb¾Û">>,<<>>]}}]
    {exception,{'EXIT',{{badmatch,already_exists},
                        [{bitcask,put,3},
                         {eqc_statem,f515_0,5},
                         {eqc_statem,f507_0,5},
                         {eqc_statem,run_commands,2},
                         {eqc_fsm,run_commands,2},
                         {bitcask_qc_fsm,'-prop_bitcask/0-fun-0-',1},
                         {eqc,'-f777_0/2-fun-4-',3},
                         {eqc_gen,'-f321_0/2-fun-0-',5}]}}} /= ok
    false
    jonmeredith committed Jan 31, 2012
  16. @jonmeredith
Commits on Feb 4, 2012
  1. @hdima

    On read CRC must be checked first to eliminate possible 'badmatch' er…

    …ror.
    
    See #30 for details.
    hdima committed Feb 4, 2012
Commits on Jan 12, 2012
  1. @jaredmorrow

    Bump to version 1.4.1

    jaredmorrow committed Jan 11, 2012
Commits on Jan 11, 2012
  1. @jonmeredith

    Merge pull request #29 from basho/jdm-fix-osx32-bit-off_t

    Decode off_t argument more carefully for pread/pwrite calls.
    jonmeredith committed Jan 11, 2012
  2. @jonmeredith

    Decode off_t argument more carefully for pread/pwrite calls.

    On some platforms sizeof(off_t)==8, sizeof(unsigned long) == 4
    so the pos argument for read and write is half garbage.
    jonmeredith committed Jan 11, 2012
Commits on Dec 28, 2011
  1. @dizzyd

    Merge pull request #27 from hdima/fold_loop_fixes

    Properly handle EOF case in bitcask_fileops:fold_loop()
    dizzyd committed Dec 28, 2011
  2. @dizzyd

    Merge pull request #28 from hdima/gitignore_fixes

    Added 'ebin' to the list of ignored files
    dizzyd committed Dec 28, 2011
  3. @hdima
  4. @hdima
Commits on Dec 27, 2011
  1. @dizzyd
  2. @dizzyd

    Fix bug where writing file was considered for merging; add clarifying…

    … comments to small_file_threshold
    
    Yay for code review!
    dizzyd committed Dec 21, 2011
  3. @dizzyd
  4. @dizzyd
  5. @dizzyd

    Add ability to close current writing file for testing; rework merging

    to provide info when selecting files to merge.
    
    Adding {log_needs_merge, true} to the .app will enable this logging.
    dizzyd committed Dec 21, 2011
  6. @dizzyd
Something went wrong with that request. Please try again.