Permalink
Switch branches/tags
vector-clock-fixes v1.4.0 release-1.10.24-cutoff release-1.10.23-cutoff release-1.10.22-cutoff release-1.10.21-cutoff release-1.10.20-cutoff release-1.10.19.1-cutoff release-1.10.19-cutoff release-1.10.18-cutoff release-1.10.17-cutoff release-1.10.16-cutoff release-1.10.15-cutoff release-1.10.14-cutoff release-1.10.13-cutoff release-1.10.12-cutoff release-1.10.11-cutoff release-1.10.10-cutoff release-1.10.9-cutoff release-1.10.8-cutoff release-1.10.7-cutoff release-1.10.6-cutoff release-1.10.5-cutoff release-1.10.4-cutoff release-1.10.3-cutoff release-1.10.2-cutoff release-1.10.1-cutoff release-1.10.0-cutoff release-1.9.22-cutoff release-1.9.21-cutoff release-1.9.20-cutoff release-1.9.19-cutoff release-1.9.18-cutoff release-1.9.17-cutoff release-1.9.16-cutoff release-1.9.15-cutoff release-1.9.14-cutoff release-1.9.13-cutoff release-1.9.12-cutoff release-1.9.11-cutoff release-1.9.10-cutoff release-1.9.9-cutoff release-1.9.8-cutoff release-1.9.7-cutoff release-1.9.6-cutoff release-1.9.5-cutoff release-1.9.4-cutoff release-1.9.3-cutoff release-1.9.2-cutoff release-1.9.1-cutoff release-1.9.0-cutoff release-1.8.16-cutoff release-1.8.15-cutoff release-1.8.14-cutoff release-1.8.13-cutoff release-1.8.12-cutoff release-1.8.11-cutoff release-1.8.10-cutoff release-1.8.9-cutoff release-1.8.8-cutoff release-1.8.5-cutoff release-1.8.4-cutoff release-1.8.3-cutoff release-1.8.1-cutoff release-1.7.3-cutoff release-1.7.2-cutoff release-1.7.1-cutoff release-1.7.0-cutoff release-1.6.8-cutoff release-1.6.6-cutoff release-1.6.4 release-1.6.4-cutoff release-1.6.3-cutoff release-1.6.2-cutoff release-1.6.1-cutoff release-1.6.0-cutoff release-1.5.9-cutoff release-1.5.8-cutoff release-1.5.7-cutoff release-1.5.4-cutoff release-1.5.2-cutoff release-1.5.1-cutoff release-1.3.0-cutoff before-replicatype-was-removed before-donorbased-was-removed
Nothing to show
Commits on Oct 11, 2012
  1. Added comment to explain why NPEs can end up in the log during shutdo…

    …wn if there async requests are queued up.
    jayjwylie committed Oct 11, 2012
  2. Changed both serial (sync) operations and parallel (async) operations…

    … to deduct the elapsed checkout time from the operation (routing) timeout for specific requests.
    jayjwylie committed Oct 11, 2012
Commits on Oct 10, 2012
Commits on Oct 9, 2012
  1. Changes to make tight timing tests for QueuedKeyedResourcePool and Sl…

    …owStorageEngine less sensitive on slower machines.
    jayjwylie committed Oct 9, 2012
  2. Switched SlowStorageEngine to take a StorageEngine<K,V,T> in the cons…

    …tructor to be more flexible.
    jayjwylie committed Oct 9, 2012
  3. Fixed E2E non blocking checkout test to actually check for non-blocki…

    …ng checkouts. (Addresses review feedback from Chinmay Soman.)
    jayjwylie committed Oct 8, 2012
  4. synchronize the reset of a specific keyed pool to avoid invoking dest…

    …royResource on the same resource multiple times.
    jayjwylie committed Oct 8, 2012
  5. bumped up test timeouts since Hudson seems slower than local machine …

    …for contention experiments.
    jayjwylie committed Oct 8, 2012
  6. Copyright statement cleanup.

    jayjwylie committed Oct 5, 2012
  7. Minor changes to deal with remaining TODOs in this change. I still be…

    …lieve there are some ugly code paths that fire off too many exceptions when we tear down a connection. Hopefully, the connection re-write that is starting off will clean up these ugly code paths.
    jayjwylie committed Oct 5, 2012
  8. Added Jmx interfaces for all queue stats we now track. Updated Client…

    …SocketStatsTest as well.
    
    Added a big TODO expressing concern over how statistics are tracked with suggestions for improvements.
    jayjwylie committed Oct 4, 2012
  9. Implementations of various async vs sync queueing policies for socket…

    … checkout. Two commented out policies are included in this commit.
    
    src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorPool.java
    - a few TODOs to be investigated before completing work on async checkouts
    
    /src/java/voldemort/utils/pool/AsyncResourceRequest.java
    - a couple helper methods for implementing/debugging queueing policies
    
    src/java/voldemort/utils/pool/KeyedResourcePool.java
    - refactor to clean up checkin method
    - TODOs for further code cleanup
    - cleaned up all methods for tracking stats, added stats tracking of length of synchronous queue
    - various aspects of (commented out) socket checkout queuing policies
    
    src/java/voldemort/utils/pool/QueuedKeyedResourcePool.java
    - fixes to async socket checkout
    - various aspects of (commented out) socket checkout queuing policies
    - TODOs for further code cleanup
    - cleaned up stats tracking for async queue length
    
    *Test.java
    - minor tweaks/cleanup
    jayjwylie committed Oct 4, 2012
  10. src/java/voldemort/store/routed/action/PerformParallelPutRequests.java

    - clean up handling of ObsoleteVersionException to do what the
      client said should be done. This will stop
      ObsoleteVersionExceptions from being escalated to
      InsufficientOperationalNodes exceptions.
    
    test/integration/voldemort/performance/benchmark/Benchmark.java
    - Add connections/node to configuration of benchmark tool
    - Set client timeouts to recommended values
    
    src/java/voldemort/store/socket/clientrequest/ClientRequestExecutor.java
    - refactored a synchronized method to isolate the bare minimum
      steps that need to be synchronized. This allows local.complete
      to be called outside of 'synchronized' which avoids deadlocking
      nio selector threads.
    jayjwylie committed Oct 3, 2012
  11. Addressed most of the feedback from the code review.

    - Renamed many variables, methods & classes
    - Addressed most of the TODOs in my changes based on the feedback
    jayjwylie committed Sep 13, 2012
  12. Fix the problems with the RebalanceTest when using the revised

    KeyedResourcePool. The test setup worked before the changes
    because the KeyedResourcePool did not aquire resources until
    full. With the new behavior, there is some mismatch between pool
    sizes and numbers of threads. Increasing the number of admin
    threads fixes the issue. There may be a better way of fixing this
    problem.
    jayjwylie committed Sep 11, 2012
  13. Addressed most feedback from reviews by refactoring:

    - Made ResourceRequest a first class entity rather than a nested interface
    - Refactored TimeoutConfig to tease apart an OpTimeMap which may be more generally useful.
    - Renamed slow storage configs in Voldemort config with to make their testing nature more clear.
    - Dropped OperationDelays object from SlowStorage in favor of OpTimeMap
    jayjwylie committed Sep 7, 2012
  14. Wrapped tests that hang because of my changes with timeouts. This

    is necessary for me to debug these tests and easily run all the
    other tests. (By 'hang', I mean get into a state where clients
    run forever complaining that there are not enough servers.)
    Wrapping with timeouts is done both at the Junit
    level (i.e., "@Test(timeout = ...")  and at the ant level (i.e.,
    for an entire unit test).
    
    Switched the tests I was touching to use Junit4 idiom rather than
    Junit3 idiom. I.e., removed the 'extends TestCase' from the class
    declaration.
    
    Upped the memory allowed for all junit tets in ant from 1024m to
    2048m since I ran into ant out of memory errors.
    jayjwylie committed Sep 7, 2012
  15. Change SlowStorageEngineTest to avoid JUnit assertions in worker

    threads. Also removed JUnit4 style annotations (@Test) since this
    is actually a Junit3 style test (because it extends other Junit3
    style tests that extend TestCase).
    jayjwylie committed Sep 5, 2012
  16. Made ant target 'junit-test' produce a report like all the other

    junit ant targets. The report is useful because for a test class
    with many sub-tests, the report clearly explains which sub-tests
    failed. Unlike other ant junit targets, this target also produce
    a plain text report. The plain text report is useful for running
    a single test in a loop until it fails.
    
    Clarified the help message for the 'junit-test' ant target.
    jayjwylie committed Sep 5, 2012
  17. Added unit test for QueuedKeyedResourcePool. Minor clean up of

    QueuedKeyedResourcePool and of KeyedResourcePoolTest. Still one
    outstanding TODO in QueuedKeyedResourcePool wrt semantics of
    close(K key) method.
    jayjwylie committed Aug 30, 2012
  18. Clean up of KeyedResourcePool and significant hardening of the unit t…

    …est.
    
    src/java/voldemort/utils/pool/KeyedResourcePool.java
    - Documented the invariants (or lack thereof) guaranteed by this class.
    - Documented this classes expectations of its users.
    - Moved attemptGrow into the inner Pool class.
    - Got rid of the attemptCheckoutGrowCheckout method. ;)
    
    src/java/voldemort/utils/pool/QueuedKeyedResourcePool.java
    - tweaked to match revised attemptGrow interface
    
    test/unit/voldemort/utils/pool/KeyedResourcePoolTest.java
    - Added a bunch of 'negative' tests. I.e., they demonstrate
      non-desirable behavior of current KeyedResourcePool.
    - Added a contention test that has many threads checkout,
      possibly invalidate, and then checkin resources for some key.
    jayjwylie committed Aug 29, 2012
  19. Initial commit of asynchronous checkouts.

    src/java/log4j.properties
    - print out more concise timestamp (don't print the date)
    - print thread at end of message
    
    src/java/voldemort/utils/pool/KeyedResourcePool.java
    - reverted all protected data members back to private
    - refactored attemptGrow to do the size check to determine if it
      is worth trying to attempt to grow. This made the method more
      useful to subclasses and cleaned up the local member that
      called it.
    - added an internalClose method that returns whether or not the
      caller is "the one thread" responsible for closing everything
      down
    
    src/java/voldemort/utils/pool/QueuedKeyedResourcePool.java
    - first complete implementation of this class
    - Four TODOs left in the code for the sake of interim code review
    
    test/integration/voldemort/nonblocking/E2ENonblockingCheckoutTest.java
    - minor tweaks to the test
    jayjwylie committed Aug 27, 2012
  20. Additional refactoring and code prep before making connection checkou…

    …t async.
    
    src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorPool.java
    
    Substantial refactoring and preparation of
    ClientRequestExecutorPool to actually do an asynchronous
    checkout. I believe I have essentially preserved prior
    behavior. I.e., checkout of destination still blocks. But, the
    pattern now looks a lot more like what is needed to do the
    checkout asyncrhnously and then callback with the checkedout
    resource. The exact exception strings likely changed during this
    refactoring (but control flow should be the same).
    
    src/java/voldemort/utils/pool/QueuedKeyedResourcePool.java
    
    Skeleton of new class QueuedKeyedResourcePool that extends
    KeyedResourcePool.
    jayjwylie committed Aug 24, 2012
  21. Code clean up and refactoring in preparation of adding truly nonblock…

    …ing requests.
    
    src/java/voldemort/store/socket/SocketStore.java
    - Moved logic from request async into ClientRequestExecutorPool::submitAsync
    - Also moved private class NonblockingStoreCallbackClientRequest into ClientRequestExecutorPool
    
    src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorPool.java
    - added private class AsyncRequestContext to start parceling up request context
    - added submitAsync and submitAsyncRequest methods. These currently do exactly what was done by SocketStore:requestAsync
    - I.e., I do not believe I have changed the behavior of any of this code. I just moved it to where I need to change it.
    
    src/java/voldemort/utils/pool/KeyedResourcePool.java
    - made some private methods protected with the expectation of
      extending this class with a subclass that can queue up async
      requests for resources
    - changed many variable names to be more consistent in naming/usage
    - refactored attemptCheckout to be simpler; added attemptCheckoutGrowCheckout to parcel up old behavior of attemptCheckout
    - Do not believe I changed any behavior with these code changes.
    - added getResourcePoolForExistingKey method
      - Throws IllegalArgumentException whenever a non existing pool is requested
      - changed all bare resourcePoolMap.get(key) accesses in other methods to use this method
      - May have changed behavior in that NPEs due to pool==null will now be IllegalArgumentExceptions in some methods
    - Documented that some getFooCount methods are approximate in face of concurrency via comments in the code
    jayjwylie committed Aug 24, 2012
  22. he current nonblocking API can actually block if a server slows

    down. The server slowdown can result in a socket checkout
    blocking because all sockets are exhausted and there are a queue
    of threads doing a blockingGet to get the socket. Adding an
    integration test, E2ENonblockingCheckoutTest, that demonstrates
    this undesirable behavior. Once a fix is in place, this
    integration test may become a unit test for the correct behavior.
    
    test/integration/voldemort/nonblocking/E2ENonblockingCheckoutTest.java
    - Demonstrates the blocking behavior of the current socket
      checkout code. The integration test sets up a local voldemort
      instance with three servers, one of which is slow. Puts are
      done by two threads to a specific key that uses a fast node as
      the master, then the slow node, followed by the other fast node
      for the parallel puts. The second thread to attempt parallel
      operations blocks upon the first returning the checkout
      socket. This is observed in the console output.
    
    test/common/voldemort/ServerTestUtils.java
    - Added another startVoldemortServer method that takes a VoldemortConfig
    
    src/java/voldemort/store/socket/SocketStore.java
    - minor fix to logger.debug print out so .toString is not called on null object
    jayjwylie committed Aug 23, 2012
  23. Adding a SlowStorageEngine to permit end-to-end testing with slow

    servers in a cluster.
    
    src/java/voldemort/store/memory/SlowStorageConfiguration.java
    src/java/voldemort/store/memory/SlowStorageEngine.java
    - The SlowStorageEngine/Configuration is inspired by the
      SlowStore used in other unit tests. The SlowStorageEngine
      produces delays on a per-operation-type basis. Delays can
      either be concurrent or queued: concurrent delays can overlap
      in time, queued delays occur in serial.
    
    src/java/voldemort/server/VoldemortConfig.java
    - Added config options for SlowStorageEngine
    
    test/unit/voldemort/store/memory/SlowStorageEngineTest.java
    - Unit test to confirm queued/concurrent delay behavior of a single SlowStorageEngine
    jayjwylie committed Aug 22, 2012
  24. Minor cleanup in preparation of making socket checkouts along non-blo…

    …cking code paths asynchronous.
    
    src/java/voldemort/store/nonblockingstore/ThreadPoolBasedNonblockingStoreImpl.java
    - fix missing ns to ms conversion
    
    src/java/voldemort/store/socket/clientrequest/ClientRequestExecutor.java
    src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorFactory.java
    - remove unnecessary selectionKey from checkTimeout interface
    
    src/java/voldemort/utils/pool/KeyedResourcePool.java
    - removed extraneous maxCreateAttempts
    - attemptGrow returns true if pool grew
    - refactored checkoutOrCreateResource
      - renamed to attemptCheckout
      - attemptCheckout is non blocking
      - attemptCheckout does not check timeouts
    - refactored checkout
      - removed unreachable code (only one attempt could ever be made so attempts and maxCreateAttempts are extraneous.
      - made it clearer that exceptions control flow through try block
    * even though code looks a lot different, I believe exact
      functionality is preserved except for one thing: the
      possibility of a single additional non-blocking get on the
      pool (resources) when attemptGrow returns true.
    jayjwylie committed Aug 22, 2012
Commits on Oct 4, 2012