Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZOOKEEPER-4492: Merge readOnly field into ConnectRequest and Response #38

Closed
wants to merge 500 commits into from

Conversation

anurag-harness
Copy link
Owner

According to this comment in ZOOKEEPER-102 I introduce a Protocol abstraction and going to moving all wire protocol concept into cnxn and this scope, so that client and server's business logics handle only deserialized/real record.

cc eolivelli maoling Randgalt

This supersedes apache#1832.

Author: tison wander4096@gmail.com

Reviewers: Enrico Olivelli eolivelli@apache.org, Mate Szalay-Beko symat@apache.org

Closes apache#1837 from tisonkun/protocol

anmolnar and others added 30 commits August 14, 2020 16:44
Recently I noticed some weird null pointer exception issues during compile in out new builds. I usually resolve this by a full "git clean" locally, so I added it to the Jenkinsfiles too.

Author: Andor Molnar <andor@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>

Closes apache#1431 from anmolnar/ZOOKEEPER-3896
- Upgrade Netty to its latest version 4.1.50.Final which includes both security fixes and AArch64 performance improvements
Refer release notes for detail: https://netty.io/news/2020/05/13/4-1-50-Final.html
- Upgraded surefire to 3.0.0-M5 due to compatibility

Author: odidev <odidev@puresoftware.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1428 from odidev/netty_upgrade
Currently, the documentation for this property just mentions TBD (check https://zookeeper.apache.org/doc/r3.5.7/zookeeperAdmin.html). I believe this was ignored until versions 3.5.7 (https://issues.apache.org/jira/browse/ZOOKEEPER-3674) and a fix was made. It will be good to add some documentation around it.

Author: Sankalp Bhatia <sankalpbhatia92@gmail.com>
Author: Sankalp <sankal@amazon.com>

Reviewers: Christopher Tubbs <ctubbsii@apache.org>, Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1405 from sankalpbhatia/master
…ooKeeper/

Author: Enrico Olivelli <eolivelli@apache.org>

Reviewers: Christopher Tubbs <ctubbsii@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1440 from eolivelli/fix/ZOOKEEPER-3801-2
…n the binary tarball in 3.6.2 rc0

- Fix license file names
- Remove legacy pom.template file (for Ant based build ?)

Author: Enrico Olivelli <eolivelli@apache.org>

Reviewers: andor@apache.org

Closes apache#1449 from eolivelli/fix/ZOOKEEPER-3924-netty-lic
After upgrading to Zookeeper 3.6, Nagios tests for zk_avg_latency started failing due to floating point metrics being compared as strings.

Author: Petter A. Urkedal <paurkedal@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1452 from paurkedal/check_zookeeper-float
Author: Jie Huang <jiehuang@fb.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Michael Han <hanm@apache.org>

Closes apache#1380 from jhuan31/ZOOKEEPER-3863
LiYvbo eolivelli

Author: tison <wander4096@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>

Closes apache#1459 from TisonKun/patch-2
It looks like the precommit job is performing an additional "git clone" that overwrites the PR

Author: Enrico Olivelli <eolivelli@apache.org>

Reviewers: andor@apache.prg

Closes apache#1460 from eolivelli/fix/pr-job-fix
… for read only observers to work).

Problem

ZOOKEEPER-3863 requires local session to be enabled for read only server to work. When merge ZOOKEEPER-3863 (in c47ef90), we only updated Java tests, and missed C tests. As a result, Jenkins build is broken.

Solution

Enable local session for C client tests.

Notes
This was not caught by Jenkins when we merge ZOOKEEPER-3863 because previous Jenkins was testing master branch (as discussed in apache#1460).

Author: Michael Han <lhan@twitter.com>

Reviewers: Enrico Olivelli <eolivelli@gmail.com>

Closes apache#1462 from hanm/fixme
commons-cli 1.2 was released in 2009, time to upgrade it to the newest version (1.4).
To avoid deprecation messages we have to upgrade the Java code too.

Author: Tamas Penzes <tamaas@cloudera.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1463 from tamaashu/zookeeper-3941
…tiation

This patch prevents SASL-enabled C clients from sending request packets for as long as the SASL negociation is not complete.

"Ideally," applications should wait for `ZOO_CONNECTED_STATE` or `ZOO_READONLY_STATE` events before issuing requests, but this is not what happens in the wild.

Without this patch, it was easy to cause a desynchronization by creating a handle and blasting requests right away.  (The added test simply avoids doing a `ctx.waitForConnected()`.)

Author: Damien Diederen <dd@crosstwine.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1457 from ztzg/ZOOKEEPER-3937-c-client-out-of-order-packets
Within ```zoo_sasl_client_destroy```, it carefully free all members of the sasl client, while here it doesn't free the struct itself before set it to NULL. This error is detected by leak sanitizer.

Author: Xiaotian <523950475@qq.com>

Reviewers: Damien Diederen <dd@crosstwine.com>, Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1466 from xiaotian-qi/ZOOKEEPER-3944
Author: dengliming <liming.d.pro@gmail.com>

Reviewers: maoling, hanm

Closes apache#1394 from dengliming/ZOOKEEPER-3841
describe InfluxDB monitoring solution

Author: Ivan Kudibal <8028110+ivankudibal@users.noreply.github.com>

Reviewers: maoling, hanm

Closes apache#1453 from ivankudibal/patch-1
Author: Byungwoo <bulee@whatap.io>

Reviewers: maoling, hanm

Closes apache#1430 from comelf/master
Author: Jie Huang <jiehuang@fb.com>

Reviewers: Michael Han <hanm@apache.org>

Closes apache#1378 from jhuan31/ZOOKEEPER-3858
…or#processRequest

As per [ZOOKEEPER-3798](https://issues.apache.org/jira/browse/ZOOKEEPER-3798), I removed the unwanted commented code.

Author: Nishanth Entoor <entoor.nishanth@gmail.com>

Reviewers: hanm, maoling, TisonKun, ravowlga123

Closes apache#1335 from nishanth-entoor/ZOOKEEPER-3798
We should test Curator's `TestingCluster` too

Author: Jordan Zimmerman <jordan@jordanzimmerman.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Micheal Han <hanm@apache.org>

Closes apache#1375 from Randgalt/ZOOKEEPER-3831-add-compatibility-test-module-pt2
…void ping being blocked by long socket closing time

Author: Jie Huang <jiehuang@fb.com>

Reviewers: ztzg, hanm, eolivelli, symat

Closes apache#1301 from jhuan31/ZOOKEEPER-3774
We use a really old version of hamcrest-library where we could use a newer one (version 2.2 instead of 1.3).
Time to update it.

Author: Tamas Penzes <tamaas@cloudera.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org>

Closes apache#1479 from tamaashu/ZOOKEEPER-3953
The BCFKS key store format is widely used in the industry, as it provides an open source alternative if someone has to use FIPS compliant key stores due to some regulatory constraints.

Currently in the ZooKeeper java client, only PEM, JKS and PEM12 is supported. I extend the list of supported key store formats with BCFKS.

I also tested this patch on a real FIPS compliant cluster, having the appropriate java security configs, security providers and also running a RedHat-based Linux distro (Centos 7.8) with FIPS mode enabled.

I tested both the client and the quorum SSL too. If someone wants to test this patch, and the keystore/truststore file names are not ending with ".bckfs", then (beside the usual SSL configs) make sure to also set the following parameters in the zoo.cfg:
```
ssl.keyStore.type=bcfks
ssl.trustStore.type=bcfks
ssl.quorum.keyStore.type=bcfks
ssl.quorum.trustStore.type=bcfks
```

and also provide the following parameters for the command line java client:
```
 -Dzookeeper.ssl.keyStore.type=bcfks -Dzookeeper.ssl.trustStore.type=bcfks
```

This patch doesn't contain any modification for the c-client (that can be handled with a separate Jira, but I don't plan to work on that part right now).

Author: Mate Szalay-Beko <symat@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org>, Andor Molnar <andor@apache.org>

Closes apache#1480 from symat/zookeeper-3950
Removed commons-lang from main project, replaced functionality with standard Java code.

Author: Tamas Penzes <tamaas@cloudera.com>

Reviewers: Christopher Tubbs <ctubbsii@apache.org>, Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>, tison <wander4096@gmail.com>

Closes apache#1478 from tamaashu/ZOOKEEPER-3952
Running the dependency check and collecting dependency-check-* files as artifacts.
Build that is validating the patch:
https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/ZOOKEEPER-3957/

Author: Andor Molnar <andor@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>

Closes apache#1483 from anmolnar/ZOOKEEPER-3957
JSON-Simple was only used in SnapshotFormatter in one place and also in Contrib's LogGraph component.
It's development has discontinued since 2012, so it's time to remove it.
I have replaced it with Jackson2, which we use in ZooKeeper anyway.
Added some unit tests to LogGraph as it didn't have any.

Author: Tamas Penzes <tamaas@cloudera.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org>

Closes apache#1484 from tamaashu/ZOOKEEPER-3956
…ters

The Programmer's guide doesn't contain the parameters needed to be changed in order to specify keystore and truststore file formats for the Java Clients.
see: https://zookeeper.apache.org/doc/r3.6.2/zookeeperProgrammers.html#sc_java_client_configuration

Author: Mate Szalay-Beko <symat@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org>

Closes apache#1488 from symat/zookeeper-3960
Create an .asf.yaml file to specify certain github repository settings,
such as description, topic, URL, etc. and to configure notification
settings and JIRA integration.

This file can also be later used to add automation for building the
project's website, in whole or in part.

See https://cwiki.apache.org/confluence/display/INFRA/git+-+.asf.yaml+features
for more information on available features.

Author: Christopher Tubbs <ctubbsii@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1489 from ctubbsii/ZOOKEEPER-3962-add-asf-yaml
The in this PR avoids a confusing and scary compilation issue [encountered by Michael Hudson-Doyle](https://issues.apache.org/jira/browse/ZOOKEEPER-3954) when building the C client with GCC 10 and "aggressive" optimization settings:

> `free_auth_completions` is being inlined into `free_completions`, and this lets gcc see that members of `a_list` are being accessed without initialization

This is (fortunately!) a red herring: what GCC doesn't see is that, in practice, `zoo_lock_auth` always returns zero, and that `a_list` is always initialized in the conditional block which follows it.

That issue is easily "fixed" by removing the `if`.  The rest of the client code doesn't check `zoo_lock_auth`'s return value--and we have bigger issues if unconditional locks start failing anyway.

See also apache#1481 and apache#1486.

Author: Damien Diederen <dd@crosstwine.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1487 from ztzg/ZOOKEEPER-3954-gcc10-compilation-fixes-master
The `sasl_client` field doesn't exist when configured without SASL.  Use an accessor outside of the blocks already guarded by `HAVE_CYRUS_SASL_H`.

(Reported by [Parag](https://issues.apache.org/jira/browse/ZOOKEEPER-3951).)

Author: Damien Diederen <dd@crosstwine.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1477 from ztzg/ZOOKEEPER-3951-no-sasl-compilation
The digest.enabled is true by default, not false (as the admin guide suggested before)

The digest feature was added to detect the data inconsistency inside ZooKeeper when
loading database from disk, catching up and following leader.

This fix is needed on master and branch-3.6. (the feature was not available in 3.5)

Author: Mate Szalay-Beko <symat@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>

Closes apache#1496 from symat/ZOOKEEPER-3808
anurag-harness and others added 24 commits January 13, 2023 22:49
…y match to jmx session (#15)

zooinspector now show "Ephemeral Owner" in 10 base number while JMX connections show in hex format. this makes it a little hard to match them.

It will be much better if hex format is shown too, I will create a pr add hex format just beside the 10 based one.

Author: 67 <67@gd67.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mohammad Arshad <arshad@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1813 from iamgd67/ZOOKEEPER-4464

Co-authored-by: 67 <67@gd67.com>
…le internal value (#16)

When packets are added to ClientCnxn's outgoing packet queue we ensure there's no conflict with an ongoing flush of that queue because of connection loss.

Synchronization used to be on the state field's value. This value is both not stable (its value changes over time), possibly causing improper synchronization, and global, which can cause contention in applications that run several ZooKeeper clients.

We now synchronize on outgoingQueue which is both local to a ClientCnxn's instance and stable.

Author: Sylvain Wallez <sylvain@bluxte.net>

Reviewers: maoling <maoling@apache.org>, Mohammad Arshad <arshad@apache.org>

Closes apache#1257 from swallez/ZOOKEEPER-3652 and squashes the following commits:

82e2cad [Sylvain Wallez] Instruct SpotBugs that we know what we're doing when synchronizing on outgoingQueue
b0bc03d [Sylvain Wallez] ZOOKEEPER-3652: Synchronize ClientCnxn outgoing queue flush on a stable internal value

Co-authored-by: Sylvain Wallez <sylvain@bluxte.net>
…#17)

CVE-2020-36518 vulnerability affects jackson-databind in Zookeeper (see GHSA-57j2-w4cx-62h2).

Upgrading to jackson-databind version 2.13.2.1 should address this issue.

Author: Edwin Hobor <edwinhobor@gmail.com>

Reviewers: maoling <maoling@apache.org>,Enrico Olivelli <eolivelli@apache.org>, Mohammad Arshad <arshad@apache.org>

Closes apache#1842 from edwin092/edwin/ZOOKEEPER-4505

Co-authored-by: Edwin Hobor <edwinhobor@gmail.com>
The newly added script runs maven with the clover profile which generates the test coverage data and then uploads it to SonarQube if the necessary parameters were given.

The required parameters for publishing to SonarQube are:
- Host URL
- Login Credentials
- Project Key

Example commands:
- Run clover code analysis (without publishing to SonarQube)
`sh tools/sonar/code-coverage.sh`

- Run clover code analysis and publish the results to SonarQube
`sh tools/sonar/code-coverage.sh -l ProjectCredentials -u https://exampleserver.com  -k Project_Key`

Author: Dora Horvath <dora.horvath@cloudera.com>

Reviewers: Norbert Kalmar <nkalmar@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1835 from horvathdora/ZOOKEEPER-4490

Co-authored-by: Dora Horvat <horvath.dora06@gmail.com>
Changes:
* Format `OpCode.addWatch` as "addWatch" in `Request.op2String`.

Author: Kezhu Wang <kezhuw@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>

Closes apache#1819 from kezhuw/ZOOKEEPER-4467-op_code_addWatch_string and squashes the following commits:

97f891d [Kezhu Wang] fixup! ZOOKEEPER-4467: Format OpCode.addWatch in Request.op2String
ebe6faa [Kezhu Wang] ZOOKEEPER-4467: Format OpCode.addWatch in Request.op2String

Co-authored-by: Kezhu Wang <kezhuw@gmail.com>
…nctionality (#20)

Make ZKUtil#deleteRecursive API fully compatible with older versions

Author: Mohammad Arshad <arshad@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1843 from arshadmohammad/ZOOKEEPER-4504-DeleteRecursive and squashes the following commits:

851bb1e [Mohammad Arshad] Added javadoc for ZKUtil#deleteRecursive(zk, pathRoot, batchSize) API
e7b3311 [Mohammad Arshad] Added test case to verify ZKUtil.deleteRecursive() in sync and async mode
008b2bd [Mohammad Arshad] ZOOKEEPER-4504: ZKUtil#deleteRecursive causing deadlock in HDFS HA functionality

Co-authored-by: Mohammad Arshad <arshad@apache.org>
Moved channel object null check to sendPkt method to cover all calling scenarios

Author: Mohammad Arshad <arshad@apache.org>

Reviewers: Mate Szalay-Beko <symat@apache.org>

Closes apache#1854 from arshadmohammad/ZOOKEEPER-4514-npe

Co-authored-by: Mohammad Arshad <arshad@apache.org>
1. For connection closing state scenario, changed the log level to debug
2. When JVM exiting with code 0, then logging info instead of error

Author: Mohammad Arshad <arshad@apache.org>

Reviewers: tison <wander4096@gmail.com>, Enrico Olivelli <eolivelli@apache.org>

Closes apache#1856 from arshadmohammad/ZOOKEEPER-4515-cli and squashes the following commits:

e7e248b [Mohammad Arshad] Logging error only when exit code is non zero
31e124a [Mohammad Arshad] ZOOKEEPER-4515: ZK Cli quit command always logs error 1. For connection closing state scenario, changed the log level to debug 2. When JVM exiting with code 0, then logging info instead of error

Co-authored-by: Mohammad Arshad <arshad@apache.org>
…sEvent (#23)

Author: Mohammad Arshad <arshad@apache.org>

Reviewers: Mate Szalay-Beko <symat@apache.org>, Enrico Olivelli <eolivelli@apache.org>

Closes apache#1855 from arshadmohammad/ZOOKEEPER-1875-npe and squashes the following commits:

4a7d471 [Mohammad Arshad] Corrected impacted test cases
12f44d6 [Mohammad Arshad] ZOOKEEPER-1875: NullPointerException in ClientCnxn$EventThread.processEvent

Co-authored-by: Mohammad Arshad <arshad@apache.org>
…ng ssl support (#24)

Currently, zktreeutil tool **doesn't compile/work**. This PR brings back `zktreeutil` to life and adds SSL support to connect to TLS enabled ZooKeeper server.

**Following are the issues fixed with this PR.**

1) [ZOOKEEPER-2108](https://issues.apache.org/jira/browse/ZOOKEEPER-2108)  **(Compilation error in ZkAdaptor.cc with GCC 4.7 or later)**
    The compilation error in `ZkAdaptor.cc` is fixed by including the header file. Also, updated the `ZOOKEEPER_PATH` in `configure.ac`

2) [ZOOKEEPER-3908](https://issues.apache.org/jira/browse/ZOOKEEPER-3908) **(zktreeutil multiple issues)**
   Fixed the issues reported in this ticket.

3) [ZOOKEEPER-4491](https://issues.apache.org/jira/browse/ZOOKEEPER-4491) **(Adding SSL support to Zktreeutil)**

    -    Adds the SSL support to `zktreeutil` to talk to TLS enabled ZooKeeper server, If SSL params are passed, invoke `zookeeper_init_ssl` method to connect the server.

    -    Used the same OpenSSL library auto-detect code from the c-client `configure.ac`  (apache#1159)

Attached the testing log.

[zktreeutil_testing_connection.txt](https://github.com/apache/zookeeper/files/8585349/zktreeutil_testing_connection.txt)

Author: Manu Mathew <manu.mathew@netapp.com>
Author: mathewmanu <manmathew@cs.stonybrook.edu>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1870 from mathew-manu/ZOOKEEPER-4491

Co-authored-by: Manu Mathew <manu.mathew@netapp.com>
…VE-2020-9493, CVE-2022-23307 (#25)

Upgraded dependency-check-maven plugin from 6.5.3 to 7.1.0

Author: Mohammad Arshad <arshad@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>, ZhangJian He <shoothzj@apache.org>

Closes apache#1872 from arshadmohammad/ZOOKEEPER-4510-upgrade

Co-authored-by: Mohammad Arshad <arshad@apache.org>
upgrading netty in master as per review in : apache#1867

Author: Ananya Singh <purpul90@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Shoothzj <shoothzj@gmail.com>, Mohammad Arshad <arshad@apache.org>, Brahma Reddy Battula <brahma@apache.org>

Closes apache#1869 from AnanyaSingh2121/ZK-4529

Co-authored-by: Ananya Singh <purpul90@gmail.com>
Removed tcnative depdency as it is required after netty upgrade to 4.1.76
Removed tcnative CVE supressions

Author: Brahma Reddy Battula <brahma@apache.org>

Reviewers: Mohammad Arshad <arshad@apache.org>

Closes apache#1873 from brahmareddybattula/master

Co-authored-by: Brahma Reddy Battula <brahma@apache.org>
ZooKeer currently has support for reloading the Quorum Truststore & Keystore automatically when the certificate files change in the filesystem without server restart (apache#680)

However, Reloading of key and trust store for **ClientX509Util** is not present; i.e., the server presented certs to the clients will not get reloaded automatically if the certificates in the filesystem change, short-lived certs requires the process restart.

Changes:

-  A new config property "zookeeper.client.certReload" is added, if it's true - ClientX509Util is reloaded automatically.

-  ZK uses an _X509AuthenticationProvider_ which is backed by an X509TrustManager and an X509KeyManager to perform _remote host certificate authentication_. We need to update the X509AuthenticationProvider's TrustStore as part of the X509Util file-watcher.
- Junit test case to verify the cert reload.

Author: Manu Mathew <manu.mathew@netapp.com>
Author: mathewmanu <manmathew@cs.stonybrook.edu>
Author: Manu Mathew <101424654+mathew-manu@users.noreply.github.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1839 from mathew-manu/ZOOKEEPER-3806

Co-authored-by: Manu Mathew <manu.mathew@netapp.com>
This is backwards compatible, since the new method on the `Testable` interface defaults to a no-op.

Author: Houston Putman <houston@apache.org>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>

Closes apache#1863 from HoustonPutman/zk-testable-socket-close and squashes the following commits:

1f9d814 [Houston Putman] Fix style
475c9f5 [Houston Putman] ZOOKEEPER-4519: Add closeSocket method to Testable interface

Co-authored-by: Houston Putman <houston@apache.org>
Zookeeper server can get stuck when it has just one client and the only way it recovers is due to a socket timeout or another client commit request.
Sync thread reads commitIsWaiting outside of a sync block and acts on this information in a sync block later. The actual status of commitIsWaiting can change between the time where commitIsWaiting is read and acted upon because commit thread updates it outside a sync block.
Fix here is to ensure that we read and process commitIsWaiting inside a sync block.

https://issues.apache.org/jira/browse/ZOOKEEPER-4537

Author: jithin23 <jithin.girish@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1877 from jithin23/ZOOKEEPER-4537

Co-authored-by: jithin23 <jithin.girish@gmail.com>
…ling (#31)

This test failed following assertions in ci:
1. `RequestThrottlerTest.testRequestThrottler:206 expected: <5> but was: <4>`

   This is caused by no happens-before relationship between
   `connectionLossCount` and `disconnected.await`. Places
   `disconnected.countDown()` after `connectionLossCount++` to solve
   this.

2. `RequestThrottlerTest.testLargeRequestThrottling:297 expected: <2> but was: <0>`

   Large request throttling is handled in io thread, while
   `prep_processor_request_queued` metric is updated in processor
   thread. Places metric assertion after `finished.await` to solve this.

Additionally, I find one more potential flaky case. After connection
closed due to throttling third request, reconnecting could fail this
test in slow sending environment. It is easy to reproduce by adding
`Thread.sleep(i * 100)` in sending loop.

Author: Kezhu Wang <kezhuw@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1821 from kezhuw/ZOOKEEPER-4327-flaky-RequestThrottlerTest.testLargeRequestThrottling and squashes the following commits:

e21c2f8 [Kezhu Wang] ZOOKEEPER-4327: Fix flaky RequestThrottlerTest.testDropStaleRequests
3df34b8 [Kezhu Wang] ZOOKEEPER-4327: Fix flaky RequestThrottlerTest.testLargeRequestThrottling

Co-authored-by: Kezhu Wang <kezhuw@gmail.com>
… connection (#32)

Motivation:
See more here https://issues.apache.org/jira/browse/ZOOKEEPER-4551

Modifications:
Do not log the stacktrace, just write the message at INFO level

Author: Enrico Olivelli <eolivelli@apache.org>

Reviewers: Mate Szalay-Beko <symat@apache.org>

Closes apache#1889 from eolivelli/fix/remove-spam-stacktrace

Co-authored-by: Enrico Olivelli <eolivelli@apache.org>
…st method (#33)

ZOOKEEPER-4518 : remove useless log in the PrepRequestProcessor#pRequest method

Author: vinayvinayu <vinayvinayu@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>

Closes apache#1862 from Vinayvinayu/ZOOKEEPER-4518

Co-authored-by: vinayvinayu <vinayvinayu@gmail.com>
We run two ZooKeeperServerEmbedded in one JVM and find that ProviderRegistry was initialized repeatedly.

Author: lanicc <lanettiesso@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, tison <wander4096@gmail.com>, maoling <maoling@apache.org>

Closes apache#1888 from lanicc/ZOOKEEPER-4549

Co-authored-by: lanicc <lanettiesso@gmail.com>
…d" (#35)

This reverts commit 3fd25d4.

Co-authored-by: maoling <maoling@apache.org>
There are several problems in this test:
* It uses `ParameterizedTest` which run tests in single jvm. But
  `ZooKeeperServer.enableEagerACLCheck` is `static` and loaded from env
  variable.
* It uses `assertNotSame` which assert on object reference equiality.
* It asserts on `zkLeader.getLastLoggedZxid()` while client connect to
  `connectedServer`. There is no happen-before between
  `zkLeader.getLastLoggedZxid()` and successful response from other
  server. The commit and response are routed to different servers and
  performed asynchronous in each server.

Author: Kezhu Wang <kezhuw@gmail.com>

Reviewers: maoling <maoling199210191@sina.com>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1851 from kezhuw/ZOOKEEPER-4308-EagerACLFilterTest

Co-authored-by: Kezhu Wang <kezhuw@gmail.com>
…napLogMetrics (#37)

This test writes txns to trigger snapshot and expects some txns remain
in txn log. But snapshot taking is asynchronous, thus all txns could be
written to snapshot. So in restarting, it is possible that no txns to
load after snapshot restored. This will fail assertion.

This commit solves this by disable automic snapshot taking by
`SyncRequestProcessor.setSnapCount(Integer.MAX_VALUE)`.

Author: Kezhu Wang <kezhuw@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1852 from kezhuw/ZOOKEEPER-4511-FileTxnSnapLogMetricsTest-testFileTxnSnapLogMetrics

Co-authored-by: Kezhu Wang <kezhuw@gmail.com>
According to [this comment in ZOOKEEPER-102](https://issues.apache.org/jira/browse/ZOOKEEPER-102?focusedCommentId=16977000&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16977000) I introduce a `Protocol` abstraction and going to moving all wire protocol concept into `cnxn` and this scope, so that client and server's business logics handle only deserialized/real record.

cc eolivelli maoling Randgalt

This supersedes apache#1832.

Author: tison <wander4096@gmail.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>

Closes apache#1837 from tisonkun/protocol
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet