Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Comparing changes

Choose two branches to see what's changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
base fork: voldemort/voldemort
base: release-1.1.1
...
head fork: voldemort/voldemort
compare: release-1.1.9
Checking mergeability… Don't worry, you can still create the pull request.
Commits on Oct 30, 2012
@jayjwylie jayjwylie Bumped curr.release to 1.1.1 3c99c01
Commits on Oct 31, 2012
@jayjwylie jayjwylie Revert return type of Versioned.getVersion() to be Version rather tha…
…n VectorClock
ba8c6fb
@jayjwylie jayjwylie Prepared release 1.1.2 84eda3a
Commits on Nov 21, 2012
@abh1nay abh1nay Fixed mapper issue 60a987c
Commits on Nov 26, 2012
@abh1nay abh1nay fixed avro mapper d56a614
Commits on Nov 27, 2012
@abh1nay abh1nay Changed release number 349f852
Commits on Nov 29, 2012
@abh1nay abh1nay Implemented a jna mlock to map and ping index files for RO stores in
memory
4db8cc3
@abh1nay abh1nay Code cleanup 32fb76a
@abh1nay abh1nay Hardcoding indexmlock to true 763ff89
@abh1nay abh1nay debug msgs# Explicit paths specified without -i nor -o; assuming --on…
…ly paths...
220c5f1
@abh1nay abh1nay Fixed bug with typesetting of the native args by wrapping it into a
native wrapper
2f5e776
@abh1nay abh1nay Fixed constructor to actually take the mlock parameter
made it true by default
7c49c64
@abh1nay abh1nay Adding mlock for RO stores 99dc97f
@abh1nay abh1nay Added release notes 374d02a
Commits on Dec 06, 2012
@abh1nay abh1nay Add support for kerberized grids in the job by supporting protocols daa49bf
Chinmay Soman Added configurable Kerberos support to HdfsFetcher and upgraded hadoo…
…p jars to 1.0.2
c745f80
Chinmay Soman Fixed the main method params in HdfsFetcher 7693d6b
Chinmay Soman Doing authentication in a synchronized block for the Hdfs fetcher, se…
…tting correct permission for the hadoop files
e6b24c6
Chinmay Soman Fixed a jmx unregister bug in hdfs fetcher 0f2f85e
Chinmay Soman Fix in FsPermission constructor to maintain hadoop jar backwards comp…
…atibility
c8d7a6c
Chinmay Soman Fixing a bug where we dont need to have hadoop conf in the classpath 2965762
Chinmay Soman Correcting usage: proxyUser to kerberosUser 577378e
Chinmay Soman Standardizing the Kerberos login phase : explicitly specify the Hadoo…
…p config and keytab path. Also assumes that extra kerberos related config parameters are passed to the Java process
44c4667
Chinmay Soman Finalized changes to HdfsFetcher to make it work with a Kerberized Ha…
…doop cluster over webhdfs
22b99c3
Chinmay Soman Adding a hack to bypass Kerberos authentication for hftp protocol. TO…
…DO: remove this bypass once the libhadoop on Solaris is well tested
e7b93ba
Commits on Dec 07, 2012
Chinmay Soman Adding KDC info and other JVM arguments to the voldemort scripts 7d93e6d
Chinmay Soman Removing the additional JVM args from the Voldemort scripts ef5420f
Chinmay Soman Creating constants for the default kerberos principal and keytab path fbe6718
Commits on Jan 02, 2013
@vinothchandar vinothchandar Stats to understand NIO layer performance + BDB exception counts et al dd29d0e
@vinothchandar vinothchandar NIO + BDB stats - Code review comments 5aa3716
Commits on Jan 04, 2013
Chinmay Soman Updated release_notes and release version to 1.1.7 including Kerberos…
… related changes
db493a1
Commits on Jan 10, 2013
@jayjwylie jayjwylie Fix tests for testStartVoldemortCluster to not consume so much memory
test/common/voldemort/ServerTestUtils.java
- remove stress test from normal testing
- test the method startVoldemortCluster once and confirm it returns a non-null Cluster object

test/common/voldemort/ServerTestUtils.java
- clarified a comment
4ae1336
Commits on Jan 11, 2013
@vinothchandar vinothchandar Monitoring for streaming operations 903e749
@vinothchandar vinothchandar Improve batch modifications on BDB-JE ec3b37e
Chinmay Soman Added the ability to delete old checksum files in the Build and Push …
…reducer. Also updated the hadoop core library to version 1.0.4-p2-rc2 containing the fix to the User ID command usage.
4f5ce5b
Chinmay Soman Updated the hadoop-core jar to 1.0.4-p2 c611a8c
Chinmay Soman Fixing .classpath which had the wrong hadoop-core version. Also chang…
…ed the Mlock related info messages to debug.
a18d91b
Commits on Jan 12, 2013
@vinothchandar vinothchandar Minor debug fixes f97699b
Commits on Jan 14, 2013
Chinmay Soman Bug fixes in HdfsFetcher revealed by HdfsFetcherTest (NPE and hiding …
…the VoldemortSerializationException)
3b6c282
@vinothchandar vinothchandar Fixing intermittent test failures 2d12189
Commits on Jan 15, 2013
@vinothchandar vinothchandar Release notes & version update for 1.1.8 adc3eb9
@jayjwylie jayjwylie Change async checkout behavior to be more like sync checkout: only c…
…reate resources (connections) when they are needed.

    Now, connections should only be created on demand. The initial code created connections until the max limit of the pool was

    Some minor tweaks to test that confirm the desired behavior at a unit test level.
2b30b33
@jayjwylie jayjwylie Changed reset and destroyRequestQueue to avoid any heavy weight synch…
…ronization methods. This should fix deadlock issue.
2768eef
@jayjwylie jayjwylie Remove an unnecessary synchronization primitive from keyedresourcepool. 1ac1b32
@jayjwylie jayjwylie Fixed Histogram: halved memory footprint, test boundary conditions, d…
…ropped unnecessary binary search.

These changes preserve/correct behavior of the current Histogram.

src/java/voldemort/store/stats/Histogram.java
- Halved memory footprint by dropping unnecessary "bounds" array
- Dropped unnecessary binary search, making insert O(1) rather than O(log(nBuckets))
- Improved documentation
- Made interface consistent for type of values inserted/got from histogram (i.e., all are 'long')

test/unit/voldemort/store/stats/HistogramTest.java
- Added tests for boundary conditions: -ive values are dropped on
  insert, "too large" values are bucketed in the final bucket on
  insert.

Minor fixes to RequestCounter calls to histogram to conform to 'long' interfaces.
90aabc3
@jayjwylie jayjwylie Added INFO level messages to better understand statistics tracking (S…
…toreStats, SocketClientStats, Histogram) behavior. Expect to remove most of these messages after debugging.

src/java/voldemort/store/stats/Histogram.java
- one more check to harden the imnplementation of insert

src/java/voldemort/store/stats/RequestCounter.java
- durationMs from int to long
- print out timing of histogram reset (q95, q99, reset)
b4994bb
@jayjwylie jayjwylie Added INFO level messages to better understand connection creation. E…
…xpet to remove most of this after debugging.

src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorFactory.java
- print out time to establish connection (if it takes longer than 1 ms)

src/java/voldemort/utils/pool/KeyedResourcePool.java
- print out info about object creation (connection establishment). In
  particular, how many outstanding creations (connection
  establishments) are in flight and how many idle resources are in the
  pool after the newly created resource is checked in.
e6ec7a0
@jayjwylie jayjwylie Added INFO level messages to better understand resetting stats in Cli…
…entSocketStats. Expect to remove this after debugging.
adf1c36
@jayjwylie jayjwylie Reverting minor-minor version change in build.properties. 515b139
@jayjwylie jayjwylie Minor tweak to reduce INFO logging for KeyedResourcePool::attemptGrow 7b8d30c
@jayjwylie jayjwylie Added INFO level messages to better understand the performance of per…
…-request instrumentation. Expect to remove most of this after debugging.
e29b222
@jayjwylie jayjwylie Clean up INFO debugging messages to be DEBUG. Minor other clean up too. 03214c9
@jayjwylie jayjwylie Change monitoring interval in ClientSocketStats to have larger window. 4c69fb6
@jayjwylie jayjwylie Minor code clean up based on review feedback
Protected all logger.(debug|info) statements I added with an is(Debug|Info)Enabled() check.

Made AsyncRecoveryFailureDetector less verbose. When it polls a server to see if it is available, it now prints out a clean INFO level message. It had been printing out a WARN level message *with* a stacktrace that made this expected behavior look much scarier than it really is.
e365419
@jayjwylie jayjwylie Less verbose logging in the face of expected exceptions and error log…
…ging in the face of a slop not being written.

src/java/voldemort/cluster/failuredetector/AbstractFailureDetector.java
- less verbose logging when a node is unavailable

src/java/voldemort/store/routed/action/AbstractAction.java
- less verbose logging in the face of expected "exceptional" responses.

src/java/voldemort/store/slop/HintedHandoff.java
- ensure that an error message is logged if a slop is not written (we can grep for "Slop write of key.*was not written" in logs)
- added TODO because ObsoleteVersionExceptiosn are neither treated as failures or successes in the callback to sendHintParallel.

test/integration/voldemort/nonblocking/E2ENonblockingCheckoutTest.java
- minor fix

test/unit/voldemort/server/socket/ClientRequestExecutorPoolTest.java
- from junit3 to junit4
3aaabcc
@jayjwylie jayjwylie Reduce granularity of failure detector locking and do not destroy all…
… enqueued requests upon setUnavailable.

src/java/voldemort/cluster/failuredetector/ThresholdFailureDetector.java
- Reduce amount of work done within synchronized section to reduce lock granularity and so ensure "side effects" of node being marked (un)available are not w/in sync section.

src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorPool.java
- Added TODO/comment to decide whether we want to actively destroy all connections upon node being marked unavailable
- Switched behavior to lazily destroying connections.
551b96d
@jayjwylie jayjwylie Refactor all PerformParallel*Request classes.
- got rid of anonymous call back classes
- factored out waitForResponses logic and processResponses logic for most of these classes. GetAll stands out as being fairly different from the others.
- did not refactor to the point of sharing common code across classes, just refactored within each class.
- added many TODOs to the code for further refactoring.
3162ba2
@jayjwylie jayjwylie Interim checkin with an ugly example of how threads could be created …
…to handle callback work. The ugly code is commented out.
ddfbb17
@jayjwylie jayjwylie Added stress test and cleaned up client request executor pool reset.
reset(dest)
- changed behavior to match original KeyedResourcePool implementation. The original QueuedKeyedResourcePool.reset() was an unnecessary/bad behavior change that canceled enqueued requests. The original behavior was to destroy idle resources whenever pool is reset.

test/integration/voldemort/socketpool/E2EClientRequestExecutorPoolAndFailureDetectorTest.java
- stress test that has put and get threads contend for slow servers in such a manner as to trigger failure detection to mark nodes unavailable. This excercises connection tear down, reset(), and build up again. This also exercises the code paths in which callbacks do heavyweight work.
526deff
@jayjwylie jayjwylie Clean up of TODOs and cruft code that had been introduced. efe9f21
@jayjwylie jayjwylie Fix for possible race condition for resource creation in KeyedResourc… 5686a52
@jayjwylie jayjwylie Minor tweak to single stress test parameters. 4a3b985
@jayjwylie jayjwylie Minor changes to end-to-end stress test. 1c15b5a
@jayjwylie jayjwylie Substantial refactoring of (Queued)KeyedResourcePool tests.
- factored out a common base class in which all of the nested helper classes are defined
- separated out the various types of tests into files:
  - simple/basic tests
  - contention tests that spawn threads to generate contention
  - specific race condition test
- The specific race condition test for KeyedResourcePool shows that google issue 276 is resolved: http://code.google.com/p/project-voldemort/issues/detail?id=276
1f3a466
@jayjwylie jayjwylie Minor changes to tests
- renamed base keyedresourcepool test to avoid pattern that ant/junit uses to try and run tests.
- tweaked stress tests parameters once more to make it easier to run locally.
076229f
@jayjwylie jayjwylie Updated copyright notices in all files I touched. 79277d1
@jayjwylie jayjwylie Moved stress test for connection checkout/checkin and failure detecto…
…r to long unit test. Cleaned up comments in ClientRequestExecutorPool to make intended semantics clearer.
eecaba0
@jayjwylie jayjwylie Reverted refactoring of PerformParallel* classes committed in f37b25e…
…cdd14ac27480a8ca353cae919464df3af
b120f45
@jayjwylie jayjwylie Fix to copyright notices that I accidentally changed by one year... a2c30c0
@jayjwylie jayjwylie Update release to 1.1.9 and updated release notes. 10353a5
@jayjwylie jayjwylie Prepare to release version release-1.1.9 292d278