Permalink
Commits on Dec 25, 2013
  1. @slfritchie
Commits on Dec 24, 2013
  1. @slfritchie
  2. @slfritchie
  3. @slfritchie
Commits on Dec 23, 2013
  1. @slfritchie

    Fix merge race when keydir_put specifies old fileid & offset races wi…

    …th delete
    
    Scenario, with 3 writers, 2 & 3 are racing:
    
    * Writer 1: Put K, write @ {file 1, offset 63}
    * Writer 2: Delete operation starts ... but there is no write to disk yet
    * Writer 3: Merge scans file 1, sees K @ {1,30} -> not out of date ->
      but there is no write to disk yet
    * Writer 2: writes a tombstone @ {3,48}
    * Writer 2: Keydir conditional delete @ old location @ {1,63} is ok
    * Writer 2: keydir delete returns from NIF-land
    * Writer 3: merge copies data from {1, 63} -> {4, 42}
    * Writer 3: keydir put {4, 42} conditional on {1,63} succeeds due to
      incorrect conditional validation: the record is gone, but bug
      permits put to return 'ok'.
    slfritchie committed Dec 23, 2013
  2. @slfritchie
Commits on Dec 21, 2013
  1. @slfritchie
  2. @slfritchie

    Add UNIX epoch time to put_int NIF call, to avoid C use of time(3).

    I hope this will eliminate a nasty source of nondeterminism during
    PULSE testing.
    slfritchie committed Dec 21, 2013
  3. @slfritchie

    Omnibus bugfixing, including:

    * Add 'already_exists' return to bitcask_nifs_keydir_remove(): we need
    it to signal races with merge, alas.
    
    * Add state to #filestate to be able to 'undo' last update to both a
    data file and its hint file.  This probably means that we're going
    to have to play some games with merge file naming, TBD, stay tuned.
    
    * For bitcask:delete(), make the keydir delete conditional: if it fails,
    redo the entire thing again.
    
    * inner_merge_write() can have a race that, if a partial merge happens
    at the proper time after, we see an old value reappearing.  Fix by
    checking the return value of the keydir put, and if 'already_exists',
    then undo the write.
    
    * When do_put() has a race and gets 'already_exists' from the keydir,
    undo the write before retrying.  If this key is deleted sometime later,
    and then a partial merge happens after that, we might see this value
    reappear after the merge is done.
    
    * Add file_truncate() to bitcask_file.erl.  TODO: do the same for the
    NIF style I/O.
    
    * Better robustness (I hope) to EUnit tests in bitcask_merge_delete.erl
    slfritchie committed Dec 21, 2013
Commits on Dec 18, 2013
  1. @slfritchie
  2. @slfritchie

    Fix bug introduced by almost-bugfix in commit 1a9c99 that was suppose…

    …d to be a partial fix for GH #82
    slfritchie committed Dec 18, 2013
  3. @slfritchie

    Fix iterator freeze bug demonstrated by last commit: remove kh_put_wi…

    …ll_resize() predicate test for creating a keydir->pending
    slfritchie committed Dec 18, 2013
  4. @slfritchie
  5. @slfritchie

    WIP: new test new_20131217_c_test_() is broken, need to experiment wi…

    …th bitcask:delete & NIF usage before continuing
    slfritchie committed Dec 18, 2013
Commits on Dec 17, 2013
  1. @slfritchie

    FML: fix tombstone merging bug when K & K's tombstone are in same fileid

    Originally found with bitcask_pulse, I deconstructed the test case to
    help understand what was happening: the new EUnit test is
    new_20131217_a_test_.
    
    As a result of the puts, key #13 is written 3x to fileid #1 (normal,
    tombstone, normal) and 1x to fileid #2 (normal @ the very beginning
    of the file).  The merge creates fileid #3 and copies only the
    tombstone (the normal entry isn't copied because it is out-of-date).
    Before the close, the internal keydir contains the correct info
    about key #13, but after the close and re-open, we see key #13's
    entries: normal (and most recent) in fileid 32, and tombstone in
    fileid #3, oops.
    
    The fix is to remove all of the merge input fileids from the set of fileids
    that will survive/exist after the merge is finished.
    slfritchie committed Dec 17, 2013
  2. @slfritchie

    Fix remaining bug when bitcask scan/fold order was changed to oldest-…

    …>youngest
    
    The NIF change fixes a long-standing latent bug: when put'ing a key
    that does not exist, if there's a race with a merge, keydir_put_int()
    would return 'ok' (error) rather than 'already_exists' (correct).  The
    'already_exists' return value is a signal to the read-write owner of
    the bitcask that the current append file must be closed and a new one
    opened (with a larger fileid than any merge).
    
    The tombstone change adds a new tombstone data format.  Old tombstones
    will be handled correctly.  New tombstones for any key K contain
    the fileid & offset of the key that it is deleting.  If the fileid
    F still exists, then the tombstone will always be merged forward.
    If the fileid F does not exist, then merging forward is not
    necessary. When F was merged, the on-disk representation of key K
    not be merged forward: K does not exist in the keydir (because it
    was deleted by this tombstone), or it was replaced by a newer put.
    slfritchie committed Dec 17, 2013
Commits on Dec 15, 2013
  1. @slfritchie
  2. @slfritchie
  3. @slfritchie
  4. @slfritchie
  5. @slfritchie
  6. @slfritchie

    Add regression test for GH #82

    slfritchie committed Dec 15, 2013
  7. @slfritchie
  8. @slfritchie

    Correct usage of ?SHRINK()

    slfritchie committed Dec 15, 2013
Commits on Dec 13, 2013
  1. @slfritchie

    Fix (I hope) PULSE model nondeterminism

    I should have checked this list of side-effect functions
    much earlier, but ... EBRAINTOOSMALL.  I hope that these
    changes will help avoid the evil nondeterminism that I've
    seen with shrinking PULSE model counterexamples.
    slfritchie committed Dec 13, 2013
Commits on Dec 12, 2013
  1. @slfritchie

    Merge pull request #124 from basho/feature/pulse-makefile-target

    Add a Makefile target for pulse tests
    slfritchie committed Dec 12, 2013
  2. @slfritchie
  3. @slfritchie

    Merge pull request #123 from basho/refactor/fileops-fold-keys

    Refactor bitcask_fileops:fold_keys function
    slfritchie committed Dec 12, 2013
Commits on Dec 11, 2013
  1. @joedevivo

    Merge pull request #125 from basho/jd-cuttlefish-api

    Updated schema for new cuttlefish api
    joedevivo committed Dec 11, 2013
  2. Updated schema for new cuttlefish api

    Joe DeVivo committed Dec 11, 2013
Commits on Dec 10, 2013
  1. @kellymclaughlin
  2. @kellymclaughlin

    Add a Makefile target for pulse tests

    Add a Makefile target to run the pulse tests. The goal is to make it
    as simple as possible to run these tests in order to increase the
    likelihood they are run during the normal course of code reviews and
    not simply forgotten. The pulse test modules are moved up directly
    into the test directory to allow rebar to find them and the test
    module is renamed to bitcask_pulse for easier identification.  Also a
    new version of rebar that can properly execute the new Makefile target
    is added.
    kellymclaughlin committed Nov 22, 2013
Commits on Dec 2, 2013
  1. @slfritchie
  2. @slfritchie
Commits on Nov 27, 2013
  1. @slfritchie