Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Jan 12, 2011

  1. Matt Ingenthron

    Catch RuntimeException instead.

    Timeouts from the get() without a time value specified will return
    simply a RuntimeException, while those from calling the get() with
    a time value can receive a TimeoutException.
    
    This also removes some debugging traces that were left in
    unfortunate places which could also cause test failures.
    
    Change-Id: Ie64aa5bedcbe36b4717c17750a63a08a7de1f12e
    Reviewed-on: http://review.membase.org/4248
    Tested-by: Matt Ingenthron <matt@northscale.com>
    Reviewed-by: Michael Wiederhold <mike@membase.com>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed

Jan 11, 2011

  1. Dustin Sallings

    Avoid potential NPE as reported by eclipse.

    Change-Id: I2bedcea366bca83597cc9863da9c63a9966eeee9
    Reviewed-on: http://review.membase.org/4232
    Tested-by: Matt Ingenthron <matt@northscale.com>
    Reviewed-by: Matt Ingenthron <matt@northscale.com>
    authored Matt Ingenthron committed
  2. Dustin Sallings

    Some import cleanups.

    Change-Id: I54bdc264566684208e5273ce51d56f38d14be852
    Reviewed-on: http://review.membase.org/4231
    Tested-by: Matt Ingenthron <matt@northscale.com>
    Reviewed-by: Matt Ingenthron <matt@northscale.com>
    authored Matt Ingenthron committed

Jan 10, 2011

  1. Matt Ingenthron

    Fixes to testSyncGetTimeouts.

    Previous to enforcing the timeouts at an operation level, this test
    would pass.  In fact, Dustin said the test had never failed before.
    
    However, it turns out that the really short default timeouts would
    be too short and not waiting a bit after encountering a timeout would
    still see timeouts.
    
    Change-Id: If1fbe77aa02f7cacabca91915927bf7b5e086284
    Reviewed-on: http://review.membase.org/4211
    Reviewed-by: Michael Wiederhold <mike@membase.com>
    Tested-by: Dustin Sallings <dustin@spy.net>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed
  2. Matt Ingenthron

    Add a TIMEDOUT state to ops and make callbacks correct.

    There would be some situations where the latch would not expire
    due to the callback not having been called.  Callbacks were
    typically called on state transition for the operation, so I
    thought it appropriate to add a TIMEDOUT state.
    
    Change-Id: Ia02b5bf6a91cf987dae3fc9faf02a41751653773
    Reviewed-on: http://review.membase.org/4212
    Reviewed-by: Michael Wiederhold <mike@membase.com>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed
  3. Matt Ingenthron

    Recognize operation may be null at times. e.g.: flush

    Both the timeout changes from myself and some of the continuious timeout
    changes from Boris assumed there would always be an operation.  In
    some cases, like flush, that is not necessarily the case.
    
    Looking at the existing code, there were lots of guards against null
    access already, so I just continued that tradition.
    
    The tradition may need to be broken though in the future.
    
    Change-Id: Ic1344ef2df2ab0ba4c03b4e401a4f98436a39772
    Reviewed-on: http://review.membase.org/4206
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Reviewed-by: Michael Wiederhold <mike@membase.com>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed
  4. Matt Ingenthron

    Fix for stats sizes test.

    Change-Id: I4d9a13f55ec0c15ebb07c924584aa33492a57a12
    Reviewed-on: http://review.membase.org/4209
    Reviewed-by: Michael Wiederhold <mike@membase.com>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed
  5. Matt Ingenthron

    Test fixes after adding new timeout logic.

    Several tests were expecting things to happen within 1ms,
    which is too short.  The new timeout functionality made these
    tests fail, where before they'd pass.
    
    Change-Id: I81473b25cfd4aa73c8c4473c1f337338162a0222
    Reviewed-on: http://review.membase.org/4210
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Reviewed-by: Michael Wiederhold <mike@membase.com>
    Tested-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed
  6. Matt Ingenthron

    Test for timeout from operation epoch.

    Change-Id: I81530461187509026cc18e995b3ceddcc3c76afb
    Reviewed-on: http://review.membase.org/4208
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Reviewed-by: Michael Wiederhold <mike@membase.com>
    Tested-by: Dustin Sallings <dustin@spy.net>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed
  7. Matt Ingenthron

    Increased default timeout to 2500ms.

    The increase of the timeout to this seemingly high value is due to
    a few findings.
    
    First, by default, garbage collection times may easily go over 1sec.
    Testing with simple toy tests shows this quite clearly, even on
    systems with lots of CPUs and a decent amount of memory.  Of course,
    much of this can be controlled with GC tuning on the JVM.  With the
    hotspot JVM, look to this whitepaper:
    http://java.sun.com/j2se/reference/whitepapers/memorymanagement_whitepaper.pdf
    
    Testing showed the following to be particularly useful:
    -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=850
    
    There is a CPU time tradeoff for this.
    
    Even with these, testing showed some 1 second timeouts when GCs near a
    half a second.  To use this software though, we shouldn't expect people
    to have to tune the GC, so raising the default seems like the
    right thing to do.
    
    Second, many systems use spymemcached on virtualized or cloud environments.
    The processes running there do not have any guarantee of execution
    time.  It'd be really unlikely for a thread to be starved for more than
    a second, but it is possible and shouldn't make things stop.  Raising this
    a bit will help.
    
    Third, and perhaps most importantly, most people run applications on
    networks that do not offer any guarantee around response time.  If
    the network is oversubscribed or even minor blips occur on the network
    can cause TCP retransmissions.  While many TCP implementations ignore
    it, RFC 2988 specifies rounding up to 1sec when calculating
    TCP retransmit timeouts.  Blips will occur, and rather than force
    this seemingly synchronous get to timeout, it may be better to
    just wait a bit longer by default.
    
    Change-Id: Ie53ca774458466d9a2e6f70e65ea6663699a9f6f
    Reviewed-on: http://review.membase.org/4207
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Reviewed-by: Michael Wiederhold <mike@membase.com>
    Tested-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed
  8. Matt Ingenthron

    Do not write timedout operations to the MemcachedNode.

    This commit and related ones add support to an operation to
    have new methods and a state of TIMEDOUT.  The intent is to
    keep track of when an operation is created and if it either
    times out due to a latch timeout expiration or it is found
    to be already too old when thinking about sending the op
    to the network, just consider it timed out then and there.
    
    Note, object creation time is actually possibly quite a bit
    after when the request is made, depending on how that request
    is made.  Any number of things could have happened in
    between with GC, JIT, scheduling, etc.
    
    Also note that in order to avoid needing to rely on the latch
    which is in a different thread, this allows us to keep track
    of the creation time of the operation and check for whether or
    not it has timed out via the isTimedOut() method on the
    operation.
    
    Doing this correctly and with as little API change as possible
    required getting the default operation timeout down to the
    MemcachedNode level.  That information was not previously known
    to the Operation or the node, but simply relied on a latch.
    
    Change-Id: I60228433bfa121ed031dd81fc05a9d65cae5bf20
    Reviewed-on: http://review.membase.org/4204
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    ingenthr authored committed

Jan 09, 2011

  1. Blair Zajac

    Tiny performance improvement.

    It's OK to have the method return an interface, but use the concrete
    class name in the method so it doesn't need to invoke the methods
    through the interface.
    
    Change-Id: Ibd3638e574f9bd0c0928af5bada53de72a59e9f1
    Reviewed-on: http://review.membase.org/3641
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Reviewed-by: Matt Ingenthron <matt@northscale.com>
    Tested-by: Matt Ingenthron <matt@northscale.com>
    blair authored Matt Ingenthron committed

Jan 07, 2011

  1. Blair Zajac

    Minor performance improvement for bulk gets.

    If the size of an ArrayList is known before construction, then pass
    the size to the constructor.  This will either save a tiny bit of
    memory or save reallocation's, depending upon the number of elements
    that will be inserted into the ArrayList.
    
    Change-Id: If1db3a8578e2d8603e0c6dbbe781ed7258908eee
    Reviewed-on: http://review.membase.org/3640
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Reviewed-by: Matt Ingenthron <matt@northscale.com>
    Tested-by: Matt Ingenthron <matt@northscale.com>
    blair authored Matt Ingenthron committed

Jul 18, 2010

  1. Boris Partensky

    support timeout based disconnects for bulk ops

    Refactored some repeating logic into helper methods,
    track continuous timeouts from bulk operations,
    log at warning.
    
    Change-Id: Ic0e3d14c8d7ff7001c3440683fa4274b119e4d31
    Reviewed-on: http://review.northscale.com/999
    Reviewed-by: Matt Ingenthron <matt@northscale.com>
    Tested-by: Matt Ingenthron <matt@northscale.com>
    magictoken authored Matt Ingenthron committed

Jun 30, 2010

  1. Blair Zajac

    Allow per-key transcoders to be used with asyncGetBulk().

    This change allows the transcoder to save state for each key.  An
    example is shown in the unit test that encodes into the byte array
    sent to the memcached server the key along with the value.  Upon
    decoding a value from memcached, the actual and expected keys are
    compared.
    
    Change-Id: Ie4697bc3f9923e7c2ba981ca334b0df9d1ab7315
    Reviewed-on: http://review.northscale.com/936
    Tested-by: Dustin Sallings <dustin@spy.net>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    blair authored committed
  2. Blair Zajac

    Add an iterator that returns a single element forever.

    This iterator will be used to add a version of asyncGetBulk() that
    allows per-key transcoders to be used.  To efficiently allow the
    normal use of multiple keys with a single transcoder, instead of
    creating an array or list of identical transcoders, other
    asyncGetBulk() methods will create a SingleElementInfiniteIterator
    instead.
    
    Change-Id: Ica58e45f3e0e49a72c7a7a8743bf9180ea9cb7ed
    Reviewed-on: http://review.northscale.com/935
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    blair authored committed

Jun 26, 2010

  1. Support daemon mode for TranscodeService threads.

    Change-Id: I28b58a9e1832abedfd8e4177bb38e5cdac158bcb
    Reviewed-on: http://review.northscale.com/920
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    ddlatham authored committed
  2. Dustin Sallings

    Some minor fixes to make eclipse happy with the code again.

    - Removed @Override annotations where there are no overrides.
    - Renamed a couple of variables that were shadowing fields.
    
    Change-Id: I7e0f74da72214cbe4c72cd693ee11461138f172b
    Reviewed-on: http://review.northscale.com/919
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    authored

Jun 17, 2010

  1. Boris Partensky

    plug potential file descriptor leak

    There is a problem in open/connect sequence that may produce
    file descriptor leaks in some abnormal conditions:
    First, SocketChannel.open() is called,
    then ch.connect(a.getSocketAddress())
    is called and it may throw an IOException under certain conditions.
    Then we catch exception and re-queue the node.
    This will produce a leak since the channel was not closed.
    
    The problem surfaced because of some faulty changes that I made
    and which I fixed. But it's nevertheless a problem.
    
    Sun  bug reference. http://bugs.sun.com/view_bug.do?bug_id=6548464
    They fixed it in a helper method that opens/connects in a single call.
    But in our case the client needs to take care of it.
    
    One more small unrelated change is to
    catch an unchecked exception and log an error:
    it is a serious case because the node
    will be essentially lost and never re-queued.
    
    Change-Id: I54930eb03f5c07fc6966f8d4d5db42548c63f6bd
    Reviewed-on: http://review.northscale.com:8080/630
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    magictoken authored committed

Jun 16, 2010

  1. Boris Partensky

    return partial data from timed out getBulk

    Augmented Future<T> object returned from asynGetBulk
    with an option to return partial data in case timeout is reached.
    
    Change-Id: I3bac849cee69fd6b57b20139832193b97975f6f6
    Reviewed-on: http://review.northscale.com:8080/563
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    magictoken authored committed

Jun 15, 2010

  1. Blair Zajac

    Fix compilation with JDK 1.5.

    Don't use methods and enum's that only exist in JDK 1.6.
    
    The changes successfully compile and the unit tests pass with JDK 1.6,
    but with 1.5 I consistently get a unit test failure here:
    
        [junit] Testcase: testSimpleLoading took 0.265 sec
        [junit] 	      Caused an ERROR
        [junit] java.lang.RuntimeException: blah
        [junit] java.util.concurrent.ExecutionException: java.lang.RuntimeException: blah
        [junit] 					     at net.spy.memcached.internal.ImmediateFuture.<init>(ImmediateFuture.java:25)
        [junit] 					     at net.spy.memcached.util.CacheLoaderTest.testSimpleLoading(CacheLoaderTest.java:48)
        [junit] 					     at org.jmock.core.VerifyingTestCase.runBare(VerifyingTestCase.java:39)
        [junit] Caused by: java.lang.RuntimeException: blah
        [junit]
        [junit] TEST net.spy.memcached.util.CacheLoaderTest FAILED
        [junit] Tests FAILED
    The following tests failed:
    net.spy.memcached.util.CacheLoaderTest
    
    Change-Id: I34921695bec8bea5f4b8b0bace13951a41b3230a
    Reviewed-on: http://review.northscale.com:8080/561
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    blair authored committed

Jun 12, 2010

  1. Blair Zajac

    Delete a duplicate unit test.

    Change-Id: Ibdbbf79455c2e497ae7e121bd88a4f260e24fa54
    Reviewed-on: http://review.northscale.com:8080/416
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    blair authored committed

Jun 08, 2010

  1. Blair Zajac

    Use a faster method to get a MD5 MessageDigest instance.

    Instead of getting a fresh instance with MessageDigest.getInstance(),
    clone an existing MessageDigest that has not been updated with any
    bytes.
    
    Change-Id: If72e112e93014631767ed68d758728f372e9a7d8
    Reviewed-on: http://review.northscale.com:8080/292
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    blair authored committed

Jun 03, 2010

  1. Blair Zajac

    No need to call MessageDigest#reset() on a brand new MessageDigest.

    Change-Id: I214f4fe0edb07c821f139b61526a2547d9772324
    Reviewed-on: http://review.northscale.com:8080/290
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    blair authored committed

Jun 02, 2010

  1. Blair Zajac

    Use a private static final byte array for "\r\n" instead of always

    converting the string into a byte array.
    
    Change-Id: I68fc5e9dea99a25e708808f14c8d0a58bd314336
    Reviewed-on: http://review.northscale.com:8080/269
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Reviewed-by: Eric Lambert <eric.d.lambert@gmail.com>
    Tested-by: Eric Lambert <eric.d.lambert@gmail.com>
    blair authored committed

May 27, 2010

  1. Move continuous timeout counter to individual connections.

    Change-Id: I0d275a824017865714af23abbb0eb61418d5d116
    Reviewed-on: http://review.northscale.com:8080/25
    Reviewed-by: Eric Lambert <eric.d.lambert@gmail.com>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: SeongHwa Ahn <neoconnected@gmail.com>
    Andrey Kartashov authored committed

Apr 21, 2010

  1. Matt Ingenthron

    Changed ports in tests for non-listening to something higher.

    Change-Id: I49a3f3238e80c0e9c445df6ac2f32f634300811f
    Reviewed-on: https://review.northscale.com:8443/1298
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Reviewed-by: Eric Lambert <eric.d.lambert@gmail.com>
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Matt Ingenthron <matt@northscale.com>
    ingenthr authored committed

Apr 12, 2010

  1. Matt Ingenthron

    Actually use the args to SASLConnectReconnect; shutdown nicely.

    Change-Id: I7517a355707ed9440974d5b5b947fb13a8ba7142
    ingenthr authored
  2. Matt Ingenthron

    Log operation failures as potential auth failures.

    Change-Id: I8e3f5d7cafda8b6007c96799cb8c09b7c780abc1
    ingenthr authored
  3. Matt Ingenthron

    Handle auth failures more gracefully; maximum failures.

    This change will ensure that autentication failures do not
    hang the thread adding an operation.  The default behavior
    is to try to authenticate forever and whine to the log if
    authentication continues to fail.
    
    Optionally, a developer may define
    net.spy.memcached.auth.AuthThreshold with an integer value
    to set a maximum number of authentication attempts before
    shutting down.
    
    Authentication threads are now spawned and managed by an
    AuthTreadMonitor class, ensuring that for any given
    connection there can only be one outstanding AuthThread.
    
    Change-Id: I0af4ab41d27239e7b555dca7d993e6228b7df5b8
    ingenthr authored
  4. Matt Ingenthron

    Minor fixes to SASL reconnect test.

    Change-Id: Ib7a560c209dd9ac3d8ce0ea36d429228bd92e647
    ingenthr authored

Apr 07, 2010

  1. Greg Kim

    Implementing read-only methods in MemcachedNodeROImpl - issue86

    Change-Id: I3437c4a69d9673d3fc7a2d0611012389b28e3d2f
    Reviewed-on: https://review.northscale.com:8443/1287
    Reviewed-by: Dustin Sallings <dustin@spy.net>
    Tested-by: Dustin Sallings <dustin@spy.net>
    gkim authored committed

Mar 31, 2010

  1. Matt Ingenthron

    Enhanced ConnectionFactoryBuilder test for auth.

    ingenthr authored
  2. Matt Ingenthron

    Manual test to ensure correct connection handling with SASL.

    ingenthr authored
  3. Matt Ingenthron

    Changed AuthTest description to match reality.

    ingenthr authored
Something went wrong with that request. Please try again.