branch-3.4 -- bugfix -- ZOOKEEPER-2890#358
Closed
xoiss wants to merge 444 commits into
Closed
Conversation
…hanian via phunt) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1535279 13f79535-47bb-0310-9956-ffa450edef68
…fpj via thawan) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1535497 13f79535-47bb-0310-9956-ffa450edef68
…ction fails (Benjamin Jaton via phunt) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1540960 13f79535-47bb-0310-9956-ffa450edef68
connection string is literal IP address. (George Cao via camille) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1541359 13f79535-47bb-0310-9956-ffa450edef68
…(thawan, fpj via thawan) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1541814 13f79535-47bb-0310-9956-ffa450edef68
(Niraj Tolia via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542356 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542489 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542710 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1544858 13f79535-47bb-0310-9956-ffa450edef68
…h (michim via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1545883 13f79535-47bb-0310-9956-ffa450edef68
…lanco via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1546231 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1547703 13f79535-47bb-0310-9956-ffa450edef68
the transaction log files on shutdown (Rakesh R via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1548826 13f79535-47bb-0310-9956-ffa450edef68
…ul Gutierrez Segales via michim) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1549963 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1550006 13f79535-47bb-0310-9956-ffa450edef68
…in the watch data structures (Germán Blanco and Michael Morello via camille) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1550220 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1550902 13f79535-47bb-0310-9956-ffa450edef68
… Blanco via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1550923 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1551112 13f79535-47bb-0310-9956-ffa450edef68
…ewton via phunt) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1551118 13f79535-47bb-0310-9956-ffa450edef68
… of 0 (Eric Lindvall via michim) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1551367 13f79535-47bb-0310-9956-ffa450edef68
…-transaction api. (Rakesh R via marshallm, phunt) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1551625 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1551930 13f79535-47bb-0310-9956-ffa450edef68
(michim via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1551985 13f79535-47bb-0310-9956-ffa450edef68
(michim, Jeffrey Zhong via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1551987 13f79535-47bb-0310-9956-ffa450edef68
cases (Germán via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1552447 13f79535-47bb-0310-9956-ffa450edef68
…avan via phunt) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1552470 13f79535-47bb-0310-9956-ffa450edef68
socket on 4 letter word requests (Rakesh R, Germán Blanco via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1553672 13f79535-47bb-0310-9956-ffa450edef68
…etter word requests. (Rakesh R, Germán Blanco via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1553694 13f79535-47bb-0310-9956-ffa450edef68
…(Rakesh R via fpj) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1554979 13f79535-47bb-0310-9956-ffa450edef68
…eTest.testConnectionEvents When a client connection occurs, the `ClientCnxn`'s `state` is changed and then a watcher event is queued. This test is unreliable because it expects the watcher to fire immediately after the state changes. This test adds a `CountDownLatch` to the watcher, to better track when it has been fired. This flaky test can be reliably reproduced by adding a `Thread.sleep` right before `states.add(event.getState());`. Author: Abraham Fine <afine@apache.org> Reviewers: Michael Han <hanm@apache.org> Closes apache#270 from afine/ZOOKEEPER-2798
As referenced in https://issues.apache.org/jira/browse/ZOOKEEPER-1748 and apache#83, add the option to use keepalived on quorum connections. These connections are often idle and long-lived, thus tend to be silently dropped by intermediate networking infrastructure (AWS Security Groups' state tables, for example). This PR adds the option to use the system's keepalive functionality when creating the socket for quorum connections. It does not change existing behavior. Author: Ben Sherman <bsherman@axon.com> Author: Ben <bensherman@gmail.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#274 from bensherman/ZOOKEEPER-1748
…rror Same as PR apache#254 but this is for branch-3.4 Author: Mohammad Arshad <arshad@apache.org> Reviewers: Michael Han <hanm@apache.org> Closes apache#278 from arshadmohammad/ZOOKEEPER-2775-branch-3.4
…NonExistingOpCode
On branch 3.4 we attempt to check that an invalid opcode in a request causes the server to disconnect the client with:
```
try {
zk.exists("/m1", false);
fail("The connection should have been closed");
} catch (KeeperException.ConnectionLossException expected) {
}
```
This can run into a race with the reconnection logic in `ClientCnxn.java`https://github.com/apache/zookeeper/blob/branch-3.4/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1052 We should use a watcher instead to track disconnects.
Author: Abraham Fine <afine@apache.org>
Reviewers: Michael Han <hanm@apache.org>
Closes apache#260 from afine/ZOOKEEPER-2786
…nationalization Warnings …stematic manner Author: gosubpl <github@gosub.pl> Reviewers: Michael Han <hanm@apache.org>, Rakesh Radhakrishnan <rakeshr@apache.org> Closes apache#285 from gosubpl/ZOOKEEPER-2730
The configuration option should not have zookeeper as its prefix.
…NonExistingOpCode See apache#286 for description. Author: Abraham Fine <afine@apache.org> Reviewers: Michael Han <hanm@apache.org> Closes apache#287 from afine/ZOOKEEPER-2786_3.4_fix
Author: b00902108 <brahmareddy.battula@huawei.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#291 from brahmareddybattula/ZOOKEEPER-2818 (cherry picked from commit 7db83eb) Signed-off-by: Michael Han <hanm@apache.org>
…ument. With this fix branch-3.4 pre-commit should generate a green build when all unit tests pass. Author: Michael Han <hanm@apache.org> Closes apache#305 from hanm/ZOOKEEPER-2811
…rite to channel to prevent resource leak. This bug happily hides in code base for 7 years until it's revealed from a failed unit tests. This commit is cherry picked from ZOOKEEPER-2737 / 5c356f5. Author: Michael Han <hanm@apache.org> Reviewers: Brian Nixon <nixon@fb.com>, Abraham Fine <afine@apache.org>, Rakesh Radhakrishnan <rakeshr@apache.org>, Patrick Hunt <phunt@apache.org> Closes apache#207 from hanm/ZOOKEEPER-2737
See https://issues.apache.org/jira/browse/ZOOKEEPER-2856 for details. When upstream like HBase call ZooKeeperSaslClient with security enabled, we sometimes get error in HBase logs like: `SASL authentication failed using login context 'Client'.` This error occures when getting SaslException in ZooKeeperSaslClient#respondToServer : `catch (SaslException e) {` `LOG.error("SASL authentication failed using login context '" +` ` this.getLoginContext() + "'.");` `saslState = SaslState.FAILED;` `gotLastPacket = true;` ` }` This error makes user confused without explicit exception message. So I think we can add exception message to the log. The patch uses parameterized logging to add the exception message to the log. Author: panyuxuan <panyuxuan@cmss.chinamobile.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#318 from pyx1990/ZOOKEEPER-2856 (cherry picked from commit 41b30a7) Signed-off-by: Michael Han <hanm@apache.org>
…ugh Java annotation. This patch uses Apache Yetus audience annotations to label our publicly available interfaces and then generate our javadoc based on the annotations. The javadoc generated by this patch should be identical to our javadoc before with a few extra classes (that I think should have been included before anyway). HostProvider Record StaticHostProvider Transaction The "gotcha" with this patch is the way that java classes generated by jute are handled. There are four of these classes that need to be publicly documented: ACL, Id, Stat, StatPersisted (in addition to their superclass Record). I thought it would be safest to have the jute compiler always label these as "Public" and then we can filter out the ones we don't want in the javadoc ant task (by excluding the org.apache.zookeeper.server package and then pulling in the tools classes separately). See apache#316 Author: Abraham Fine <abefine@cloudera.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#317 from afine/ZOOKEEPER-2829_3.4
…be correct (Erik Anderson via michim) git-svn-id: https://svn.apache.org/repos/asf/zookeeper/trunk@1448007 13f79535-47bb-0310-9956-ffa450edef68
This patch primarily adds a cross-platform CMake build system. This is in addition to the Autotools system on Linux, until it can be deprecated, and this replaces the existing committed Visual Studio Solutions for Windows. As this commit deprecates (and breaks) the previously existing Visual Studio solutions and projects, they've been removed. Building on Windows now requires CMake, but this enables the support of any code-supported Visual Studio version. Support for Visual Studio 2017 was explicitly added by this patch, and support for future versions, barring software bugs, should be available automatically through CMake. The notable features lacking in comparison to Autotools are Solaris support, shared library builds, whitelisted symbol exports, libtool integration, and doxygen document generation. Almost everything else from Autotools has been ported, including header/function/library checks, and all targets (zookeeper, hashtable, cli, load_gen, and tests). The primary work involved for CMake (other than the writing of `CMakeLists.txt`) was transitioning the hand-written `winconfig.h` to an auto-generated `config.h` file, and guarding code with `#ifdef HAVE_FEATURE`. The `cmake_config.h.in` template was modeled after the Autotools config file so that the feature guards share the same names. This patch also refactors the Windows port of the C client. Notably, it moves as much porting code as possible out of the publicly included `winconfig.h` header and into the relevant source files, or the private `winport.h` header. While `load_gen.c` looks at first glance as if it were ported to Windows, it never actually was, so the erroneous `#include "win32port.h"` was removed, and the target is not built on Windows. The `include/winstdint.h` header was removed as it has been replaced by `<stdint.h>`. This might break very old versions of Visual Studio; but in those cases, previous versions of the client are available. A bug for upstream libraries including the ZooKeeper headers was fixed by removing `#undef AF_INET6` entirely, and removing `#include <windows.h>`, as it "pulls in the world" and should not be included publicly. A guard was placed around `#define snprintf` in order to enable compiling with modern versions of MSVC, including Visual Studio 2015. A problem with `DLL_EXPORT` and `USE_STATIC_LIB` being redefined was fixed. There are existent warnings which this patch did not attempt to fix. Building tests on Windows is not supported, but was not previously supported. The tests need to be ported. Future work should resolve the `ACL` / `ZKACL` conflict, and potentially remove `include/winconfig.h` altogether. Author: Andrew Schwartzmeyer <andrew@schwartzmeyer.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#313 from andschwa/cmake-backport-3.4
This is a backport of ZOOKEEPER-1576 to the 3.4-line. When running Zookeeper as an ensemble in a dynamic environment such as Kubernetes, the DNS entry of a Zookeeper pod is apparently instantly purged as one of the nodes goes down. This leads to an UnknownHostException when interacting with the cluster, even though a healthy majority of nodes is still working. This behavior is also observed in a firewall situation as described in ZOOOKEEPER-1576. This fix catches and logs the UnkownHostException and continues trying the next node. Thanks to Vishal Khandelwal for providing the patch. Author: Thomas Schüttel <thomas.schuettel@daimler.com> Reviewers: Edward Ribeiro <edward.ribeiro@gmail.com>, Michael Han <hanm@apache.org> Closes apache#320 from tschuettel/ZOOKEEPER-2614
…le reading the proposal packet. This commit is a port of the commit ca22b3d from branch-3.5 to branch-3.4. Changes include a few interfaces that required by the test case. The test case itself is also updated so it works with 3.4 code base. arshadmohammad rakeshadr Could you please review this to close the loop of ZOOKEEPER-2355? Author: Michael Han <hanm@apache.org> Reviewers: Mohammad Arshad <arshad@apache.org> Closes apache#304 from hanm/ZOOKEEPER-2355
…s of sessions expired same time The issue is raised while tens thousands of clients try to reconnect ZooKeeper service. Actually, we came across the issue during maintaining our HBase cluster, which used a 5-server ZooKeeper cluster. The HBase cluster was composed of many many regionservers (in thousand order of magnitude), and connected by tens thousands of clients to do massive reads/writes. Because the r/w throughput is very high, ZooKeeper zxid increased quickly as well. Basically, each two or three weeks, Zookeeper would make leader relection triggered by the zxid roll over. The leader relection will cause the clients(HBase regionservers and HBase clients) disconnected and reconnected with Zookeeper servers in the mean time, and try to renew the sessions. In current implementation of session renew, NIOServerCnxnFactory will clone all the connections at first in order to avoid race condition in multi-threads and go iterate the cloned connection set one by one to find the related session to renew. It's very time consuming. In our case (described above), it caused many region servers can't successfully renew session before session timeout, and eventually the HBase cluster lose these region servers and affect the HBase stability. The change is to make refactoring to the close session logic and introduce a ConcurrentHashMap to store session id and connection map relation, which is a thread-safe data structure and eliminate the necessary to clone the connection set at first. Author: Sun Qi <sunqi@qiyi.com> Reviewers: Edward Ribeiro <edward.ribeiro@gmail.com>, Michael Han <hanm@apache.org> Closes apache#312 from CheneySun/branch-3.4
…signed. This is a port of the same patch committed to master and branch-3.5, after resolving merge conflicts.
For example: `python check_compatibility.py branch-3.5..branch-3.4 ` Author: Abraham Fine <afine@apache.org> Reviewers: Michael Han <hanm@apache.org> Closes apache#329 from afine/ZOOKEEPER-2864 (cherry picked from commit 82be7f6) Signed-off-by: Michael Han <hanm@apache.org>
Author: Fangmin Lyu <allenlyu@fb.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#331 from lvfangmin/ZOOKEEPER-2870 (cherry picked from commit 0c5b320) Signed-off-by: Michael Han <hanm@apache.org>
The libraries `libm` and `librt` (providing `math.h` and `clock_gettime` respectively) cannot be linked on OS X. Instead, this functionality is "free" with `libSystem`. Note that the `hashtable` library has the dependency on `libm`, not the `zookeeper` library. This was an error carried over from the Autotools build. Also finishes a TODO for `pthread` by importing it using the `FindThreads` CMake package. Author: Andrew Schwartzmeyer <andrew@schwartzmeyer.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#319 from andschwa/macos (cherry picked from commit 5bcffe9) Signed-off-by: Michael Han <hanm@apache.org>
When building in Debug configuration, this logic ensures that `/MTd` is used instead of just `/MT`, which on Windows means to link to the multi-threaded (debug) version of the standard library. When the user does not add `/MT` as a compile option manually, CMake would otherwise link to the correct one. Because we are overriding it for threaded compilations, we also must ensure that Debug configurations are specially handled. Furthermore, this must be done using a generator expression over configuration time logic because the Visual Studio CMake generators are "multi-configuration generators", that is, the configuration is chosen at build time instead of compile time. The generator expression handles this scenario, but checking `CMAKE_BUILD_TYPE` would not. Author: Andrew Schwartzmeyer <andrew@schwartzmeyer.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#335 from andschwa/ZOOKEEPER-2874 (cherry picked from commit ab182d4) Signed-off-by: Michael Han <hanm@apache.org>
Author: yaniv.kunda <yaniv@kundas.net> Reviewers: Abe Fine <afine@apache.org>, Michael Han <hanm@apache.org> Closes apache#323 from ykunda/master (cherry picked from commit 09742f2) Signed-off-by: Michael Han <hanm@apache.org>
- Change format of README.txt to markdown. Author: Manoj Mallela <manojmallela@users.noreply.github.com> Reviewers: Michael Han <hanm@apache.org> Closes apache#341 from manojmallela/master (cherry picked from commit 86577c9) Signed-off-by: Michael Han <hanm@apache.org>
Contributor
Author
|
Sorry, I wanna propose it not for the master branch -- closed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2890