Permalink
Switch branches/tags
vector-clock-fixes v1.4.0 release-1.10.25-cutoff release-1.10.24-cutoff release-1.10.23-cutoff release-1.10.22-cutoff release-1.10.21-cutoff release-1.10.20-cutoff release-1.10.19.1-cutoff release-1.10.19-cutoff release-1.10.18-cutoff release-1.10.17-cutoff release-1.10.16-cutoff release-1.10.15-cutoff release-1.10.14-cutoff release-1.10.13-cutoff release-1.10.12-cutoff release-1.10.11-cutoff release-1.10.10-cutoff release-1.10.9-cutoff release-1.10.8-cutoff release-1.10.7-cutoff release-1.10.6-cutoff release-1.10.5-cutoff release-1.10.4-cutoff release-1.10.3-cutoff release-1.10.2-cutoff release-1.10.1-cutoff release-1.10.0-cutoff release-1.9.22-cutoff release-1.9.21-cutoff release-1.9.20-cutoff release-1.9.19-cutoff release-1.9.18-cutoff release-1.9.17-cutoff release-1.9.16-cutoff release-1.9.15-cutoff release-1.9.14-cutoff release-1.9.13-cutoff release-1.9.12-cutoff release-1.9.11-cutoff release-1.9.10-cutoff release-1.9.9-cutoff release-1.9.8-cutoff release-1.9.7-cutoff release-1.9.6-cutoff release-1.9.5-cutoff release-1.9.4-cutoff release-1.9.3-cutoff release-1.9.2-cutoff release-1.9.1-cutoff release-1.9.0-cutoff release-1.8.16-cutoff release-1.8.15-cutoff release-1.8.14-cutoff release-1.8.13-cutoff release-1.8.12-cutoff release-1.8.11-cutoff release-1.8.10-cutoff release-1.8.9-cutoff release-1.8.8-cutoff release-1.8.5-cutoff release-1.8.4-cutoff release-1.8.3-cutoff release-1.8.1-cutoff release-1.7.3-cutoff release-1.7.2-cutoff release-1.7.1-cutoff release-1.7.0-cutoff release-1.6.8-cutoff release-1.6.6-cutoff release-1.6.4 release-1.6.4-cutoff release-1.6.3-cutoff release-1.6.2-cutoff release-1.6.1-cutoff release-1.6.0-cutoff release-1.5.9-cutoff release-1.5.8-cutoff release-1.5.7-cutoff release-1.5.4-cutoff release-1.5.2-cutoff release-1.5.1-cutoff release-1.3.0-cutoff before-replicatype-was-removed before-donorbased-was-removed
Nothing to show
Commits on Jul 27, 2017
  1. Releasing Voldemort 1.10.25

    cqgao committed with gaojieliu Jul 27, 2017
Commits on Jul 21, 2017
  1. Release Voldemort 1.0.24

    cqgao committed Jul 21, 2017
  2. Added optional CDN feature to the BnP pipeline.

    Currently the Voldemort Build and Push (BnP) plugin tells Voldemort
    cluster to fetch twice from the source HDFS cluster.
    This optional CDN feature will instead copy files to dedicated CDN
    clusters, and therefore reduce bandwidth requirement for the source.
    
    The following are new attributes that are related to this release:
    1. push.cdn.enabled
        The global switch.
        Example: true
        Default: false
    
    2. push.cdn.cluster
        A list of "destination|cdn" pairs separated by comma, where "destination"
        is a Voldemort cluster, and "cdn" is the corresponding HDFS cluster used
        as CDN. if "cdn" is "null", V-cluster will fetch directly from the source
        HDFS cluster instead, in which case the behavior is identical to the
        previous versions of Voldemort.
        Example: tcp://v-cluster1:6666|hdfs://cdn1:9000,tcp://v-cluster2:6666|webhdfs://cdn2:50070,tcp://v-cluster3:6666|null
        Default: null
    
    3. push.cdn.prefix
        A directory on the CDN cluster as the root for all distcp copied files.
        Example: /jobs/VoldemortBnP
        Default: null
    
    4. push.cdn.readByGroup
        Set true if CDN files are read by a different user in the same group.
        Example: true
        Default: true
    
    5. push.cdn.readByOther
        Set true if CDN files are read by a different user in a different group.
        Example: true
        Default: true
    
    6. push.cdn.writtenByGroup
        Set true if CDN files are written by a different user in the same group.
        Example: true
        Default: true
    
    7. push.cdn.writtenByOther
        Set true if CDN files are written by a different user in a different group.
        Example: true
        Default: true
    
    8. push.cdn.storeWhitelist
        A comma-separated list of Voldemort store names to which the CDN feature will
        apply. (for testing purposes)
        Default: null (means apply to every store)
    
    This release is compatible with existing job configurations. The default
    behavior is identical to the previous version.
    cqgao committed with gaojieliu Jul 21, 2017
Commits on Jun 9, 2017
  1. Merge pull request #476 from FelixGV/fix_client_shell_list_printing_l…

    …ogic
    
    Fixed the client shell's list parsing logic
    singhsiddharth committed on GitHub Jun 9, 2017
Commits on Jan 13, 2017
  1. Fix the connection leaking issue. (#469)

    gaojieliu committed on GitHub Jan 13, 2017
Commits on Jan 7, 2017
Commits on Jan 6, 2017
  1. Added build.replica.factor check when decide whether Voldemort should…

    … fail the
    
    data push when data fetches fail in some nodes in HA mode.
    gaojieliu committed Jan 6, 2017
Commits on Nov 29, 2016
Commits on Nov 28, 2016
  1. Adhere target/source java compatibility for contrib projects

    We are using Java 1.7 on our hadoop cluster, but locally I use JDK 1.8
    for building, so contrib got build with 1.8 by default because its
    source and target version is not specified explicitly. Consequentially
    the resulting jar didn't work in our cluster environment. This
    reconfiguration fixes this.
    
    But since the contrib projects don't build anymore with 1.6 source
    compatibility I bumped up the configured javac.version to 1.7, even
    though two other options are available:
    
    1. Fix contrib to make it 1.6 compatible again
    2. Specify different versions for the main and contrib projects
    
    But since Java 1.6 is EOL since quite a while I suggest that no
    Voldemort servers with a Java 1.6 runtime should be running anymore and
    it should be save to upgrade and keep the configuration simple.
    bitti committed with FelixGV Oct 13, 2016
  2. Fix detection for fetcher protocol warning

    Fetcher protocol warning was shown if the recommended protocol is
    specified explicitly, which is unnecessary confusing
    bitti committed with FelixGV Oct 19, 2016
Commits on Nov 24, 2016
  1. Make nodes option working again for generate_cluster_xml.py

    Option -n or --nodes is broken since d2452a9 when the
    input file check was added.
    bitti committed Nov 24, 2016
Commits on Nov 10, 2016
  1. Releasing Voldemort 1.10.23

    FelixGV committed Nov 10, 2016
  2. BnP now retries fetches when cluster.xml is stale.

    Previously, there was a race condition where a BnP job would initialize
    its AdminClients at the beginning of the job, and then hang on to that
    Cluster state throughout the job. If a maintenance is going on while
    the job is running, then it's possible that by the time the job gets to
    the "Push" phase, the Cluster representation may be stale. In those
    cases, it is possible that a BnP job may attempt to push to a node
    which has been swapped out of the cluster. This may cause BnP HA to
    trigger even though the cluster is actually healthy at that time.
    
    In order to fix this, two changes are made in this commit:
    1. In the VoldemortSwapJob, the AdminClient is constructed from scratch
       rather than being created based on the previous Cluster state. This
       should minimize the window during which it is possible to change
       the cluster.xml and make BnP hit the wrong node, but it does not
       completely eliminate the race condition.
    2. In the AdminStoreSwapper, the invokeFetch() code will check if an
       exception is caused by a stale cluster state. If it is, it will
       get a fresh AdminClient and retry the operation. This should totally
       prevent the race condition.
    
    The BnP job will do a limited amount of fetch retries (10 attempts with
    30 seconds of wait time between each) and only when hitting soft errors
    (i.e.: connection failure, etc.).
    FelixGV committed Nov 8, 2016
Commits on Nov 9, 2016
  1. Tweaked the AdminClient's currentVersion so that it is not stale.

    Previously, there could be a case where an AdminClient is created from
    a stale Cluster instance, which would lead to isClusterModified() not
    returning the correct result. This was because the AdminClient would
    always set its currentVersion to the current time, no matter how long
    ago the passed in Cluster instance was originally generated. In cases
    where the cluster.xml configuration is altered after the Cluster
    instance is constructed, but before the AdminClient is constructed,
    then there is a potentially very long window during which the wrong
    currentVersion would be set.
    FelixGV committed Nov 8, 2016
Commits on Nov 8, 2016
  1. BnP now kills an async job that it is waiting on if that job times out.

    Previously, BnP would just leave the aync job running if it timed out,
    which is wasteful, and could cause a subsequent job retry to fail if
    there are two fetch jobs running concurrently for the same store.
    FelixGV committed Nov 8, 2016
Commits on Nov 7, 2016
  1. Changed the DeleteAllFailedFetchStrategy so that it affects all nodes.

    Previously, the DeleteAllFailedFetchStrategy would only attempt to
    delete data from nodes which succeeded in their fetch. In some failure
    modes, this is appropriate, but in other cases, it isn't. In any case,
    there is no harm in trying to delete data on all nodes, even those
    that failed their fetch. This commit makes it so.
    FelixGV committed Nov 7, 2016
Commits on Nov 4, 2016
  1. Made admin connection/socket timeout configurable in BnP.

    Also changed the default socket timeout to 180 seconds.
    
    This fixes the following problem: when a node is unreachable and
    completely shut down, requests to it will time out, which takes
    60 seconds. When BnP notices this, it will reach one of the live
    nodes in the cluster and ask it to deal with the failure. The
    live node will try to talk to the dead node, which will also
    take 60 seconds to time out. By the time the live node decides
    that the dead node is unreachable, and responds to the BnP job,
    the BnP job will have already timed out. Then, the BnP job will
    think that the HandleFailedFetchRequest could not complete
    successfully (even though it did in fact complete successfully)
    and BnP HA will be aborted.
    
    The solution is that the BnP job's socket timeout must be greater
    than the server's default connection timeout.
    
    This was not an issue before when we had insanely long time outs,
    but those time outs have been reduced considerably in commit
    34debd3. This is likely when we
    regressed on the handling of this failure mode.
    FelixGV committed Nov 4, 2016
  2. Fix lots of typos and spelling mistakes

    This shouldn't entail any functional changes (besides some corrected log
    or assertion error messages)
    bitti committed with FelixGV Oct 20, 2016
Commits on Oct 5, 2016
  1. Added some extra logging when OOM occurs in BnP.

    The AvroStoreBuilderMapper can OOM when manipulating certain bad Avro records.
    
    This change does not actually prevent the OOM, but merely prints some useful
    info before dying.
    FelixGV committed Oct 4, 2016
Commits on Sep 26, 2016
  1. Replaced the following instances with http://www.project-voldemort.com

    …since urls don't resolve from some places
    
    $ grep -r 'http://project-voldemort' .
    ./clients/python/setup.py:      url='http://project-voldemort.com',
    ./NOTES:For the most up-to-date information see http://project-voldemort.com
    ./contrib/collections/src/java/voldemort/collections/VStack.java: *        voldemort JSON formats: http://project-voldemort.com/design.php
    mattwisein committed Sep 26, 2016
Commits on Sep 20, 2016
  1. Releasing Voldemort 1.10.22

    mattwisein committed Sep 20, 2016
Commits on Sep 13, 2016
  1. The BnP job should be resilient to colo failures, but this regressed.

    This commit adds a safe guard to bring back resilience to full colo
    failures.
    
    Now, if a colo is unreachable, the BnP job will still push to the
    other (healthy) colos, but it will fail the job afterwards with a
    message saying which colo failed.
    FelixGV committed Sep 12, 2016
Commits on Sep 6, 2016
  1. Python client has an issue with inconsistent indentation (#446)

    The indentation in the code is mostly spaces while the offending
    line is tab indented. Hence, importing and initializing the client
    fails with an Indentation error.
    esawtooth committed with mattwisein Sep 6, 2016
Commits on Aug 30, 2016
  1. Provide chunk size suggestion for BnP jobs with chunk overflow except…

    …ions and fix num chunks algorith to round up
    mattwisein committed Aug 29, 2016
Commits on Aug 29, 2016
  1. Introduced new boolean "readonly.omit.port" server configuration.

    When set to true, the port will be removed from the fetch URI. In
    this case, the already-existing "readonly.modify.port" setting is
    ignored.
    
    When set to false (which is the default), then the port will be
    left as part of the fetch URI (according to the already-existing
    "readonly.modify.port" setting).
    FelixGV committed Aug 25, 2016
Commits on Aug 17, 2016
  1. vadmin.sh stream support for system stores

    The commands
    bin/vadmin.sh stream fetch-entries
    bin/vadmin.sh stream fetch-keys
    
    does not work on System stores like voldsys$_client_registry
    
    There is a client side check for valid stores, which only is
    validating the user stores. Added a check to include the system
    stores as well.
    arunthirupathi committed Aug 17, 2016
  2. Data Cleanup job Does not run on system stores

    1) Client registry System store is a in-memory store and supposed to be cleaned up after 7 days.
    Last change to the DataCleanupJob made the system stores fail with the missing store exception.
    
    Clients re-use the same client id, so unless lots of clients become
    dead and removed, this will not cause a leak on the server resources. The effect is negligible.
    
    Now the DataCleanupJob checks for both system stores and normal stores for a store definition.
    
    2) If the store retention days is modified to zero, then the store will
    delete all the records. But if the store is started with 0 retention days
    it means the data retention is not enabled. Fixed the discrepancy.
    arunthirupathi committed with arunthirupathi Aug 17, 2016
Commits on Aug 11, 2016
  1. Revert "Provide chunk size suggestion for BnP jobs with chunk overflo…

    …w exceptions and fix num chunks algorith to round up"
    
    This reverts commit fdd2ca9.
    squarY committed Aug 11, 2016
Commits on Aug 10, 2016
  1. Releasing Voldemort 1.10.21

    Fix release notes.
    squarY committed Aug 10, 2016
  2. Merge pull request #436 from squarY/timeoutfix

    Fix: Extend the timeout of admin request
    squarY committed on GitHub Aug 10, 2016
  3. Fix: extend admin request time out from 1min to 5min.

    Add more logs when handing failed fetch request.
    
    Fix issues based on RB.
    squarY committed Aug 10, 2016
Commits on Aug 5, 2016
  1. Provide chunk size suggestion for BnP jobs with chunk overflow except…

    …ions and fix num chunks algorith to round up
    FelixGV committed with mattwisein Jun 10, 2016
Commits on Jul 28, 2016