Skip to content

xx#494

Closed
brettKK wants to merge 368 commits intomasterfrom
branch-3.5
Closed

xx#494
brettKK wants to merge 368 commits intomasterfrom
branch-3.5

Conversation

@brettKK
Copy link

@brettKK brettKK commented Mar 26, 2018

No description provided.

rakeshadr and others added 30 commits September 17, 2015 07:10
logs are at different locations (Arshad Mohammad via rgs)

git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.5@1713297 13f79535-47bb-0310-9956-ffa450edef68
reading trace mask argument (Chris Nauroth via rgs)

git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.5@1713301 13f79535-47bb-0310-9956-ffa450edef68
spaces in the data (Ryan Lamore via rgs)

git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.5@1713304 13f79535-47bb-0310-9956-ffa450edef68
…om Info

to Warn level (Lin Yiqun via rgs)

git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.5@1713313 13f79535-47bb-0310-9956-ffa450edef68
…ation

and on website (Shawn Heisey via rgs)

git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.5@1713321 13f79535-47bb-0310-9956-ffa450edef68
(Marshall McMullen via rgs)

git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.5@1718208 13f79535-47bb-0310-9956-ffa450edef68
cdonati and others added 27 commits November 21, 2017 09:47
When a sequence counter exceeds 2147483647, the next value is -2147483648.

Author: Chris Donati <chris.donati@appscale.com>

Reviewers: phunt@apache.org

Closes #421 from cdonati/ZOOKEEPER-2944

Change-Id: If9ef0705e533ea9bc91d6da0af1984d5c93d89e5
(cherry picked from commit cbe524c)
Signed-off-by: Patrick Hunt <phunt@apache.org>
…kwardElectionRoundTest.testBackwardElectionRound

Change-Id: Ia92eb2b2dfbd9b7c464abdc7ae018f426e591538
(cherry picked from commit 851075b)
Author: Abraham Fine <afine@apache.org>

Reviewers: phunt@apache.org

Closes #422 from afine/ZOOKEEPER-2948

Change-Id: I8d5d88d6b5863346dbb963f979ad9e6c9b33025c
(cherry picked from commit 81fe526)
Signed-off-by: Patrick Hunt <phunt@apache.org>
Missed the quorum.cfg file which also needs to be updated for the new location.

Change-Id: Ifa65812c2b9c3e11b3f2137aa4d471964c5b4ad6
(cherry picked from commit 748585d)
Same as #397 just targeting branch-3.5

Author: Abraham Fine <afine@apache.org>

Reviewers: phunt@apache.org

Closes #426 from afine/ZOOKEEPER-2915_3.5

Change-Id: I5801b259b458b3d6495c252da0f0d9d5f0db985e
This patch is the trunk version of #409

Author: Andor Molnár <andor@cloudera.com>

Reviewers: phunt@apache.org

Closes #420 from anmolnar/ZOOKEEPER-2924-trunk

Change-Id: I2260247e271eb86a6d37fe297e9fb5a7be3babc0
(cherry picked from commit d857e04)
Signed-off-by: Patrick Hunt <phunt@apache.org>
Author: Radu Brumariu <radu@groupon.com>

Reviewers: phunt@apache.org

Closes #410 from fr0stbyte/ZOOKEEPER-2338

Change-Id: I7b060a2dc473b34aecc0cfac71d39720369bd635
 fixes ZOOKEEPER-2951

Author: Jorg Heymans <jorg.heymans@gmail.com>

Reviewers: phunt@apache.org

Closes #431 from jorgheymans/master

Change-Id: Id47651ebdbb29574b47863640e928b6a99af6920
(cherry picked from commit dcfbe45)
Signed-off-by: Patrick Hunt <phunt@apache.org>
…zookeeper.py

Add keys for the zxid and its component pieces: epoch and transaction counter. These are not reported by the 'mntr' command so they must be obtained from 'stat'. The counter is useful for tracking transaction rates, and epoch is useful for tracking leader churn.

 zk_zxid         - the 64bit zxid from ZK
 zk_zxid_counter - the lower 32 bits, AKA the counter
 zk_zxid_epoch   - the upper 32 bits, AKA the epoch

Author: Alex Bame <alex@724support.com>

Reviewers: phunt@apache.org

Closes #425 from alexbb/add_zxid_stats and squashes the following commits:

8dfdc9f [Alex Bame] leave zk_zxid in hex
874628d [Alex Bame] add keys for the Zxid from the stat command:  zk_zxid         - the 64bit zxid from ZK  zk_zxid_counter - the lower 32 bits, AKA the counter  zk_zxid_epoch   - the upper 32 bits, AKA the epoch

Change-Id: I5b43b8e27deab23669c7b44f777ddb02f50db4aa
(cherry picked from commit 9e30b9b)
Signed-off-by: Patrick Hunt <phunt@apache.org>
https://issues.apache.org/jira/browse/ZOOKEEPER-2952

Author: Andor Molnar <andor@cloudera.com>

Reviewers: phunt@apache.org

Closes #429 from anmolnar/ZOOKEEPER-2952 and squashes the following commits:

b488bf4 [Andor Molnar] ZOOKEEPER-2952. Updated slf4j license
949ada4 [Andor Molnar] ZOOKEEPER-2952. Upgrade third party libs: netty, slf4j

Change-Id: I7cce9cdbefab6c560a582c4dae7b4e00c73b1529
(cherry picked from commit aa8b606)
Signed-off-by: Patrick Hunt <phunt@apache.org>
Author: Abraham Fine <afine@apache.org>

Reviewers: phunt@apache.org

Closes #433 from afine/ZOOKEEPER-2953_3.5 and squashes the following commits:

a06f84f [Abraham Fine] use shared Servers and numServers
6930955 [Abraham Fine] improve assertion logging
5e4da2d [Abraham Fine] fix whitespace
36abecd [Abraham Fine] ZOOKEEPER-2953: Flaky Test: testNoLogBeforeLeaderEstablishment

Change-Id: I9b45324b65d3199dfbba4a936f5b58fb8f4a2d3d
…t to server

'addr' variable is used to identify which server to connect to.
I've made this available for error handling code in order to let it fallback to this address if the remote socket hasn't been initialised yet. This will give us better error messages if the client is unable to connect to server for some reason.

Author: Andor Molnar <andor@cloudera.com>

Reviewers: phunt@apache.org

Closes #430 from anmolnar/ZOOKEEPER-2893 and squashes the following commits:

aa73554 [Andor Molnar] ZOOKEEPER-2893. Use log4j message templates
47a8cf4 [Andor Molnar] ZOOKEEPER-2893. Make serverAddress local variable of run(). Separate SocketExceptions from generic ex handler and log at info level.
6ea4cb2 [Andor Molnar] ZOOKEEPER-2893. Renamed addr to serverAddress, use serverAddress in log message, it's always populated with the correct remote endpoint
fbe4ccd [Andor Molnar] ZOOKEEPER-2893. Make 'addr' variable available for error handling code to give a chance to fallback if the socket hasn't been initialized yet

Change-Id: I22becf9c1f923a28c82f263b604239fde9bc0ce4
(cherry picked from commit e129e7a)
Signed-off-by: Patrick Hunt <phunt@apache.org>
…data

This bug is caused by an issue in our file padding logic. We calculate an incorrect position to add padding when appending to the transaction log, this often corrupts a transaction. When the log is read the CRC check will correctly fail.

Author: Abraham Fine <afine@apache.org>

Reviewers: phunt@apache.org

Closes #436 from afine/ZOOKEEPER-2249 and squashes the following commits:

f3bc06c [Abraham Fine] Improve testing and respond to phunt's comment
44ad057 [Abraham Fine] ZOOKEEPER-2249: CRC check failed when preAllocSize smaller than node data

Change-Id: Id8ba9ad730760cb78672127b8c0e02db60b4e87d
(cherry picked from commit 4d629d2)
Signed-off-by: Patrick Hunt <phunt@apache.org>
This test relies on hooking into our logging system and creates a new appender using a PatternLayout object shared with the CONSOLE appender. PatternLayout has some synchronization issues (https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html) so we should create a new instance of it.

Author: Abraham Fine <afine@apache.org>

Reviewers: phunt@apache.org

Closes #437 from afine/ZOOKEEPER-2961_master

Change-Id: I1c81d37b94b12e0721a671deb2fc983773f88fd9
(cherry picked from commit 77d6b76)
Signed-off-by: Patrick Hunt <phunt@apache.org>
See [https://issues.apache.org/jira/browse/ZOOKEEPER-2964](url) for details.

The bug affects versions newer than 3.5. According to Andor Molnar‘s [review](https://issues.apache.org/jira/browse/ZOOKEEPER-2964?focusedCommentId=16330018&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16330018) this patch can be applied to master and branch-3.5 branches.

Thanks all for reviewing this issue.

Author: Qihong Xu <qihongxu@bu.edu>

Reviewers: phunt@apache.org

Closes #449 from qihongxu/ZOOKEEPER-2964

Change-Id: Ie7de251afd5cca64f4735d75d9c95a886ff75328
(cherry picked from commit 78708a8)
Signed-off-by: Patrick Hunt <phunt@apache.org>
- more details in [JIRA:ZOOKEEPER-1580](https://issues.apache.org/jira/browse/ZOOKEEPER-1580)
- I forget fetching the upstream codes, make a mistake in the origin [PR-446] (#446) which includes some review history,so I close it and open a new one

Author: maoling <maoling199210191@sina.com>

Reviewers: Patrick Hunt <phunt@apache.org>, Andor Molnár <andor@cloudera.com>, Abraham Fine <afine@apache.org>

Closes #452 from maoling/ZOOKEEPER-1580 and squashes the following commits:

29a5aba [maoling] remove the setter and return back to setting the running field directly
28de1e8 [maoling] ZOOKEEPER-1580:QuorumPeer.setRunning is not used

(cherry picked from commit d1b07d5)
Signed-off-by: Abraham Fine <afine@apache.org>
If the server has more than one host name, and serve each host name with different certificates. then the ssl client must provide the server name in the ssl Hello packet, to tell the server which certificate to use.
This is especially important when the client connect to a load balancer with different backend services.

https://en.wikipedia.org/wiki/Server_Name_Indication

Author: f00231050 <shaobao.feng@huawei.com>

Reviewers: Andor Molnár <andor@cloudera.com>, Abraham Fine <afine@apache.org>

Closes #423 from abel-von/ZOOKEEPER-2949

(cherry picked from commit 6655421)
Signed-off-by: Abraham Fine <afine@apache.org>
…g#commit` to solve the confuse that reason is too large log or too busy disk I/O

`FileChannel#size` info should be added to `FileTxnLog#commit` to solve the confuse that reason is too large log or too busy disk I/O

------
Example from jenkins [log](https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/830/console):
```java
2017-06-28 10:25:16,645 [myid:] - WARN  [SyncThread:0:FileTxnLog341] - fsync-ing the write ahead log (1945616 bytes) in SyncThread:0 took 2240ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
```

Author: asdf2014 <benedictjin2016@gmail.com>
Author: asdf2014 <1571805553@qq.com>

Reviewers: Patrick Hunt <phunt@apache.org>, Andor Molnár <andor@cloudera.com>, Abraham Fine <afine@apache.org>

Closes #296 from asdf2014/ZOOKEEPER-2824 and squashes the following commits:

7cf4b68 [asdf2014] Improve latency log description
dd2268f [asdf2014] ZOOKEEPER-2824: `FileChannel#size` info should be added to `FileTxnLog#commit` to solve the confuse that reason is too large log or too busy disk I/O

(cherry picked from commit 069c3e4)

Signed-off-by: Abraham Fine <afine@apache.org>

Signed-off-by: Abraham Fine <afine@apache.org>
https://issues.apache.org/jira/browse/ZOOKEEPER-2939
- Refactor proposal serialization logic to a common place (SerializeUtils),
- Add JMX metric to monitor jute.maxbuffer setting
- Add JMX metric to monitor min/max/last size of generated proposals
- Add proposal stats info to 'stat' 4lw command
- Add proposal stats resetting to 'srst' 4lw command
- Unit tests

Author: Andor Molnár <andor@cloudera.com>
Author: Andor Molnar <andor@cloudera.com>

Reviewers: Patrick Hunt <phunt@apache.org>, Abraham Fine <afine@apache.org>

Closes #415 from anmolnar/ZOOKEEPER-2933 and squashes the following commits:

cf6ea5e [Andor Molnar] ZOOKEEPER-2933. Added proposal size statistics to 'mntr' command
cb28df6 [Andor Molnár] ZOOKEEPER-2939. Fixed findbugs issue: newline in format string
8e6b65f [Andor Molnár] ZOOKEEPER-2939: Added last/min/max proposal size JMX beans

(cherry picked from commit aefb13f)
Signed-off-by: Abraham Fine <afine@apache.org>
afine This PR is a fix for branch-3.5 builds. Please take this as high priority.

Author: Andor Molnar <andor@cloudera.com>

Reviewers: Mark Fenes <mfenes@cloudera.com>, Abraham Fine <afine@apache.org>

Closes #463 from anmolnar/ZOOKEEPER-2939_3.5
…rs at startup

ZOOKEEPER-2967: Add check to validate dataDir and dataLogDir parameters at startup

This PR adds a check to protect ZK against configuring dataDir and dataLogDir opposingly.

When FileTxnSnapLog is created, it checks if transaction log directory contains snapshot files or vice versa, snapshot directory contains transaction log files. If so, the check throws LogdirContentCheckException or SnapdirContentCheckException, respectively, which translates to DatadirException at ZK startup in QuorumPeerMain and ZooKeeperServerMain.

If the two directories are the same, then no check is done.

For testing, I've added 4 new unit tests which cover the following cases:

transaction log and snapshot directories are different and they are used correctly (no Exception)
transaction log and snapshot directories are the same (in this case no check is done)
transaction log and snapshot directories are different and transaction log directory contains snapshot files (LogdirContentCheckException -> ZK quits)
transaction log and snapshot directories are different and snapshot directory contains transaction log files (SnapdirContentCheckException -> ZK quits)

Author: Mark Fenes <mfenes@cloudera.com>

Reviewers: Andor Molnár <andor@cloudera.com>, Abraham Fine <afine@apache.org>

Closes #458 from mfenes/ZOOKEEPER-2967_3.5
At line 518, 'existWatches.remove(clientPath)' is null because watches for clientPath is already removed.

Author: 07c00h <07c00h@gmail.com>

Reviewers: Abraham Fine <afine@apache.org>

Closes #461 from achimbab/master

(cherry picked from commit f0b67b6)
Signed-off-by: Abraham Fine <afine@apache.org>
This is the version of #453 for the 3.5 branch

Author: Robert Evans <evans@yahoo-inc.com>

Reviewers: Abraham Fine <afine@apache.org>, Mark Fenes <mfenes@cloudera.com>, Andor Molnár <andor@cloudera.com>, Kishor Patil <kpatil@yahoo-inc.com>

Closes #454 from revans2/ZOOKEEPER-2845-3.5
Author: asutosh936 <asutosh.pandya@hotmail.com>

Reviewers: Andor Molnár <andor@cloudera.com>, Abraham Fine <afine@apache.org>

Closes #473 from asutosh936/master and squashes the following commits:

6a17613 [asutosh936] ZOOKEEPER-2936 - Deleted duplicate code
a02ef3d [asutosh936] ZOOKEEPER-2936 - Removed duplicate code

(cherry picked from commit ecb1162)
Signed-off-by: Abraham Fine <afine@apache.org>
…ction: http->https

ZOOKEEPER-2992 is the Jira issue.

Author: Shawn Heisey <elyograg@apache.org>

Reviewers: Edward Ribeiro <edward.ribeiro@gmail.com>, Andor Molnár <andor@cloudera.com>, Michael Han <hanm@apache.org>

Closes #479 from elyograg/master
Missing `String.format`.

Author: sel-fish <fqlgy@hotmail.com>

Reviewers: Edward Ribeiro <edward.ribeiro@gmail.com>, Andor Molnár <andor@cloudera.com>, Michael Han <hanm@apache.org>, maoling <maoling199210191@sina.com>

Closes #492 from sel-fish/ZOOKEEPER-3001

(cherry picked from commit 8cfca3a)
Signed-off-by: Michael Han <hanm@apache.org>
By removing this code, CMake will use its own defaults for the CRT
flags (e.g., `/MDd` for debug configurations). With it removed, the
user can override this behavior by setting the `CMAKE_CXX_FLAGS`
manually when configuring ZooKeeper.

Author: Andrew Schwartzmeyer <andrew@schwartzmeyer.com>

Reviewers: Michael Han <hanm@apache.org>

Closes #485 from andschwa/ZOOKEEPER-2997

(cherry picked from commit 13c9f89)
Signed-off-by: Michael Han <hanm@apache.org>
@brettKK brettKK changed the title Branch 3.5 ZOOKEEPER-3007:Potential NPE in ReferenceCountedACLCache#deserialize Mar 26, 2018
@brettKK brettKK changed the title ZOOKEEPER-3007:Potential NPE in ReferenceCountedACLCache#deserialize xx Mar 26, 2018
@brettKK brettKK closed this Mar 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.