Filename from max timestamp (resubmitted) #19

Closed
wants to merge 2 commits into
from

Conversation

Projects
None yet
1 participant
@jonmeredith
Contributor

jonmeredith commented Jul 8, 2011

Changed next data file name (timestamp) to be greater than any current fileid

If muliple data files are created in a second, previously bitcask would
just increment the timestamp based filename by one and try and create again.
With very small file sizes the next active filename could get many seconds
ahead of the current time. If merge ran and removed files then it would create
new files with timestamps less than the current output file confusing the merge
logic.

(Resubmitted with bctt enhancements committed first)

jonmeredith added some commits Jul 11, 2011

Improved merging in bctt.
The writer thread now calls needs_merge (which closes open file handles
from the old merge) and kicks the merge process using the filenames
returned by merge.

Added support for merge parameters.

Added extra docs in the header comment.
Changed next data file name (timestamp) to be greater than any curren…
…t fileid.

If multiple data files are created in a second, previously bitcask would
just increment the timestamp based filename by one and try and create again.
With very small file sizes the next active filename could get many seconds
ahead of the current time.  If merge ran and removed files then it would create
new files with timestamps less than the current output file confusing the merge
logic.
@jonmeredith

This comment has been minimized.

Show comment
Hide comment
@jonmeredith

jonmeredith Jul 11, 2011

Contributor

Third times the charm - bctt was broken. Resubmitting.

Contributor

jonmeredith commented Jul 11, 2011

Third times the charm - bctt was broken. Resubmitting.

slfritchie added a commit that referenced this pull request Nov 27, 2013

Work-around model limitation/bug in PULSE test, bitcask_eqc.erl
In the C65 counterexample below, the {var,2} worker proc tries to
get key #19 and allegedly fails.  The last step of the main proc is
a put loop of keys #1 - #34.  The {var,2} worker is racing with
both the main proc and also with the {var,7} worker that is
doing a fold.  The fold happens first, which freezes the keydir
when the next put happens.  Then the {var,2} worker does its get
and finds (correctly) `not_found` ... but the model is buggy and
can't predict the correct answer.

I'm not good enough at the temporal logic to fix the model
The Correct Way.  So I've patched it up so that any 'bad' thing
detected by the main/1st proc is truly bad.  But if a 'bad' thing
happens in a forked worker, then if we know that there's a fold
happening at the same time, we assume that the model's predicted
answer is bad and thus we *ignore* the 'bad' thing.

    191> C65 = binary_to_term(element(2,file:read_file("zzz.c65@develop@commit=a2d75.bin"))).
    [[{set,{var,2},
           {call,bitcask_eqc,fork,
                 [[{init,{state,undefined,false,false,[]}},
                   {set,{not_var,8},{not_call,bitcask_eqc,bc_open,[false]}}]]}},
      {set,{var,4},
           {call,bitcask_eqc,fork,
                 [[{init,{state,undefined,false,false,[]}},
                   {set,{not_var,2},{not_call,bitcask_eqc,bc_open,[false]}},
                   {set,{not_var,3},
                        {not_call,bitcask_eqc,bc_close,[{not_var,2}]}},
                   {set,{not_var,9},{not_call,bitcask_eqc,bc_open,[false]}},
                   {set,{not_var,10},
                        {not_call,bitcask_eqc,get,[{not_var,9},19]}}]]}},
      {set,{var,6},
           {call,bitcask_eqc,fork,
                 [[{init,{state,undefined,false,false,[]}},
                   {set,{not_var,3},{not_call,bitcask_eqc,bc_open,[false]}}]]}},
      {set,{var,7},
           {call,bitcask_eqc,fork,
                 [[{init,{state,undefined,false,false,[]}},
                   {set,{not_var,5},{not_call,bitcask_eqc,bc_open,[false]}},
                   {set,{not_var,7},
                        {not_call,bitcask_eqc,fold,[{not_var,5}]}}]]}},
      {set,{var,12},
           {call,bitcask_eqc,fork,
                 [[{init,{state,undefined,false,false,[]}},
                   {set,{not_var,3},{not_call,bitcask_eqc,bc_open,[false]}},
                   {set,{not_var,8},
                        {not_call,bitcask_eqc,fold,[{not_var,3}]}}]]}},
      {set,{var,17},{call,bitcask_eqc,bc_open,[true]}},
      {set,{var,28},{call,bitcask_eqc,put,[{var,17},1,<<>>]}},
      {set,{var,29},{call,bitcask_eqc,needs_merge,[{var,17}]}},
      {set,{var,30},{call,bitcask_eqc,fork_merge,[{var,17}]}},
      {set,{var,32},
           {call,bitcask_eqc,puts,[{var,17},{1,34},<<>>]}}],
     {77863,46676,48146},
     [{events,[]}]]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment