Permalink
Branch: develop
Commits on Aug 14, 2013
  1. Merge pull request #27 from basho/cv-3717-patch

    rzezeski committed Aug 14, 2013
    Cv 3717 patch
  2. Use the `git` vsn in .app file

    rzezeski committed Aug 14, 2013
    Stop relying on human to bump the version number.  It should be unique
    for every commit.
  3. Remove drop call from EQC test

    rzezeski committed Aug 14, 2013
    The drop call causes occasional errors with the EQC test.  I'm
    removing it because the EQC test does not model how Riak Search uses
    the drop call.  Which is only done after a handoff has finished.  It
    would be nice to find and fix the race condition that exists between
    drop and the other operations but for now I just want a consistent
    test.
  4. Only compact if there are segments to compact

    rzezeski committed Aug 14, 2013
    Don't needlessly write a new segment if there were no segments to
    compact in the first place.  This was an error caught by the EQC
    tests.
Commits on Aug 13, 2013
  1. Don't compact on empty segment list

    rzezeski committed Aug 13, 2013
    If there are no segments there is nothing to compact.  This issue was
    hit during EQC test and could happen if user invoked manual compaction
    before any segments existed.
  2. Add verbose eunit

    rzezeski committed Aug 13, 2013
  3. Merge branch 'master' into cv-3717-patch

    rzezeski committed Aug 13, 2013
    Conflicts:
    	src/merge_index.app.src
  4. Bump version

    rzezeski committed Aug 13, 2013
Commits on Jul 3, 2013
  1. Merge pull request #30 from basho/rz-no-delayed-write

    rzezeski committed Jul 3, 2013
    Remove the delayed_write option
  2. Remove the delayed_write option

    rzezeski committed Jul 3, 2013
    Don't use the delayed_write option when opening a file.  This option
    is known to cause corruption issues in Erlang 15x.  It is fixed in
    16B01 but currently Riak must run on 15B01, thus the VM cannot simply
    be upgraded.
    
    This patch is being made in direct relation to a customer having
    buffer corruption issues when the machine is under heavy load.  I was
    able to reproduce easily using an underpowered centos VM running on
    virtual box.
Commits on Mar 29, 2013
  1. Keep mtime in seg record, simplify, add test

    rzezeski committed Mar 29, 2013
    * Keep the mtime in the segment record.  Segments are immutable files
      and this makes it easier to write tests.
    
    * Allow only 2 time scalars, minutes and hours.  Seconds is too fine
      and days can easily be done my multiplying hours.  Less combinations
      means less to break.
    
    * Add test to verify the average + stale check works.
    
    * Set default to 1 hour to amortize the cost rather than build up
      large segments.
    
    * Do strict checking of the app-env value to prevent user typos from
      crashing merge index.
Commits on Dec 28, 2012
  1. Allow runtime change of the segments-to-merge calculation

    rzezeski committed Dec 28, 2012
    The default calculation which determines the segments to merge is
    easily skewed since it relies on the mean average.  This especially
    hurts given automatic compaction is driven solely by buffer rollovers,
    thus guarenteeing skew (because the new segments will always be much
    smaller than old segments).
    
    This change will allow a user to change the calculating function at
    runtime.  This could be useful in production, allowing the ability to
    swap in more agressive algorithms temporarily to help free up disk
    space and memory.
Commits on Sep 4, 2012
  1. Merge branch '1.2'

    rzezeski committed Sep 4, 2012
  2. Merge branch 'rz-fix-iter-1.2' into 1.2

    rzezeski committed Sep 4, 2012
  3. Fix iterator API

    rzezeski committed Aug 31, 2012
    The iterator API will timeout any time there is a segment larger than
    the `segment_full_read_size`.  In this scenario the segment file is
    opened by the `mi_server` process but then reading is attempted by the
    spawned iterator process.  Since the segment file is opened in raw mode
    this will fail.  The port message will go to the `mi_server` causing
    an `Unexpected info...` msg and the iteraor will hit the
    `lookup_timeout`.
    
    Many thanks to Arnaud Wetzel who discovered this bug and took time to
    explain it to me.
  4. Add failing iterator test

    rzezeski committed Aug 31, 2012
    Add the iterator command to the EQC test to prove that it is broken.
Commits on Jun 19, 2012
Commits on Jun 13, 2012
  1. Add back pressure to iterator API

    rzezeski committed Jun 13, 2012
    In order to prevent slower consumers from piling up result chunks add
    back pressure to the iterator API.  This is important because this API
    is used for handoff & repair both of which have back pressure and will
    potentially not be able to consume data as fast as the local iterator
    can produce it.
    
    Currently after every 4 result chunks sent the MI server iterator will
    wait until it sees a continue msg.  The number 4 was chosen
    arbitrarily and the reason for not applying back pressure on a
    per-chunk basis is to avoid some latency/overhead by reducing the
    number of msgs that need to be sent.
Commits on Jun 12, 2012
  1. Factor out buffer/segment locking code

    rzezeski committed Jun 12, 2012
    There are multiple places where the buffers and segments need to be
    locked.  Factor this code into a shared function.
  2. Add iterator API to iterate entire index in non-blocking fashion

    rzezeski committed May 16, 2012
    This is used by async folds in order to allow repair without blocking
    primary vnodes.
  3. Remove trailing whitespace

    rzezeski committed May 16, 2012
Commits on Sep 1, 2011
  1. Merge pull request #13 from basho/az608-use-less-memory

    rzezeski committed Sep 1, 2011
    More sane buffer conversion with better memory usage
  2. Make converter worker transient so it restarts itself

    rzezeski committed Sep 1, 2011
    Get rid of my whacky link/trap exit/restart code and let the supervisor do that
    since that's what it's meant for.
Commits on Aug 31, 2011
  1. Merge pull request #16 from basho/az629-use-lager

    rzezeski committed Aug 31, 2011
    Convert to lager
Commits on Aug 30, 2011
  1. Convert to lager

    rzezeski committed Aug 30, 2011
    az629
    
    Convert all logging to use lager. Also resurrected some logging that I removed
    in commit `7c6d74f76b00abb8f5c7e46d999c75416967222d` as it is potentially
    useful.
Commits on Aug 29, 2011
  1. Merge pull request #14 from rampage/patch-1

    rzezeski committed Aug 29, 2011
    Updated to the correct usage of the API
  2. More sane buffer conversion with better memory usage

    rzezeski committed Aug 26, 2011
    az608 bz1065
    
    Convert the awkward buffer->segment process to a more OTP compliant
    simple_one_for_one supervisor.  To keep the previous behavior of only one
    buffer at a time per mi_server I made use of a queue in the mi_server to
    keep track of pending buffer conversions.
    
    This reduces memory usage as it now completely avoids long-lived buffer
    conversion processes.
Commits on Aug 24, 2011
  1. Merge pull request #12 from basho/az606-general-cleanup

    rzezeski committed Aug 24, 2011
    General cleanup - add API to `mi_server`
Commits on Aug 19, 2011
  1. General cleanup - add API to `mi_server`

    rzezeski committed Aug 19, 2011
    az606 bz502
    
    1. Add full API to `mi_server` so as not to make direct gen_server calls
    in `merge_index`.
    
    2. Alphabetize API in `mi_server`.
    
    3. Label module sections and move functions to correct locations in
    `mi_server`.
    
    4. Remove unused elements from state record