Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FLAKY_TEST] EtcdLedgerManagerTest > testRegisterLedgerMetadataListener hangs out forever #3044

Closed
nicoloboschi opened this issue Feb 11, 2022 · 1 comment
Closed

[FLAKY_TEST] EtcdLedgerManagerTest > testRegisterLedgerMetadataListener hangs out forever #3044

nicoloboschi opened this issue Feb 11, 2022 · 1 comment
Labels
flaky-tests type/bug

Comments

@nicoloboschi
Copy link
Contributor 

2022-02-11T21:36:44.5636373Z org.apache.bookkeeper.metadata.etcd.EtcdLedgerManagerTest > testRegisterLedgerMetadataListener STANDARD_OUT
2022-02-11T21:36:44.5637220Z     2022-02-11T21:36:44,459+0000 INFO  [Test worker] o.a.b.m.e.t.EtcdTestBase@82 - Successfully build etcd client to endpoint http://localhost:49163 {}
2022-02-11T21:36:44.5638407Z     2022-02-11T21:36:44,468+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManager@103 - Create ledger metadata under key ZotzDnfa/ledgers/00000000-0000-0000-0000-017eeab78bb4 {}
2022-02-11T21:36:44.5639231Z     2022-02-11T21:36:44,473+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManager@220 - read ledger metadata under key ZotzDnfa/ledgers/00000000-0000-0000-0000-017eeab78bb4 {}
2022-02-11T21:36:44.5640795Z     2022-02-11T21:36:44,485+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManagerTest@264 - Create ledger metadata : LedgerMetadata{formatVersion=3, ensembleSize=3, writeQuorumSize=3, ackQuorumSize=2, state=OPEN, digestType=CRC32C, *** ensembles={0=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183]}, customMetadata={}} {}
2022-02-11T21:36:44.5641995Z     2022-02-11T21:36:44,486+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManagerTest@273 - Registered first listener for ledger 1644615404468 {}
2022-02-11T21:36:44.5643285Z     2022-02-11T21:36:44,490+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManagerTest@270 - [listener1] Received ledger 1644615404468 metadata : Versioned(value=LedgerMetadata{formatVersion=3, ensembleSize=3, writeQuorumSize=3, ackQuorumSize=2, state=OPEN, digestType=CRC32C, *** ensembles={0=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183]}, customMetadata={}}, version=205) {}
2022-02-11T21:36:44.5644495Z     2022-02-11T21:36:44,508+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManagerTest@289 - Registered second listener for ledger 1644615404468 {}
2022-02-11T21:36:44.5645787Z     2022-02-11T21:36:44,508+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManagerTest@286 - [listener2] Received ledger 1644615404468 metadata : Versioned(value=LedgerMetadata{formatVersion=3, ensembleSize=3, writeQuorumSize=3, ackQuorumSize=2, state=OPEN, digestType=CRC32C, *** ensembles={0=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183]}, customMetadata={}}, version=205) {}
2022-02-11T21:36:44.5646798Z     2022-02-11T21:36:44,512+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManager@220 - read ledger metadata under key ZotzDnfa/ledgers/00000000-0000-0000-0000-017eeab78bb4 {}
2022-02-11T21:36:44.5648278Z     2022-02-11T21:36:44,521+0000 INFO  [etcd-watcher-scheduler-OrderedScheduler-1-0] o.a.b.m.e.EtcdLedgerManagerTest@286 - [listener2] Received ledger 1644615404468 metadata : Versioned(value=LedgerMetadata{formatVersion=3, ensembleSize=3, writeQuorumSize=3, ackQuorumSize=2, state=OPEN, digestType=CRC32C, *** ensembles={0=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183], 10=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183]}, customMetadata={}}, version=206) {}
2022-02-11T21:36:44.5650036Z     2022-02-11T21:36:44,523+0000 INFO  [etcd-watcher-scheduler-OrderedScheduler-1-0] o.a.b.m.e.EtcdLedgerManagerTest@270 - [listener1] Received ledger 1644615404468 metadata : Versioned(value=LedgerMetadata{formatVersion=3, ensembleSize=3, writeQuorumSize=3, ackQuorumSize=2, state=OPEN, digestType=CRC32C, *** ensembles={0=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183], 10=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183]}, customMetadata={}}, version=206) {}
2022-02-11T21:36:44.5651142Z     2022-02-11T21:36:44,526+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManager@220 - read ledger metadata under key ZotzDnfa/ledgers/00000000-0000-0000-0000-017eeab78bb4 {}
2022-02-11T21:36:44.5652578Z     2022-02-11T21:36:44,537+0000 INFO  [etcd-watcher-scheduler-OrderedScheduler-1-0] o.a.b.m.e.EtcdLedgerManagerTest@270 - [listener1] Received ledger 1644615404468 metadata : Versioned(value=LedgerMetadata{formatVersion=3, ensembleSize=3, writeQuorumSize=3, ackQuorumSize=2, state=OPEN, digestType=CRC32C, *** ensembles={0=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183], 20=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183]}, customMetadata={}}, version=207) {}
2022-02-11T21:36:44.5653891Z     2022-02-11T21:36:44,540+0000 INFO  [etcd-watcher-scheduler-OrderedScheduler-0-0] o.a.b.m.e.EtcdLedgerManager@372 - Closed ledger metadata watcher on ledger 1644615404468 since there are no listeners any more. {}
2022-02-11T21:36:44.6636472Z     2022-02-11T21:36:44,641+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManager@220 - read ledger metadata under key ZotzDnfa/ledgers/00000000-0000-0000-0000-017eeab78bb4 {}
2022-02-11T21:36:44.6637543Z     2022-02-11T21:36:44,643+0000 INFO  [Test worker] o.a.b.m.e.EtcdLedgerManagerTest@336 - Registered first listener for ledger 1644615404468 again {}
2022-02-11T21:36:44.6639117Z     2022-02-11T21:36:44,648+0000 INFO  [pool-76-thread-2] o.a.b.m.e.EtcdLedgerManagerTest@270 - [listener1] Received ledger 1644615404468 metadata : Versioned(value=LedgerMetadata{formatVersion=3, ensembleSize=3, writeQuorumSize=3, ackQuorumSize=2, state=OPEN, digestType=CRC32C, *** ensembles={0=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183], 30=[127.0.0.1:3181, 127.0.0.1:3182, 127.0.0.1:3183]}, customMetadata={}}, version=208) {}
2022-02-11T22:30:48.7339364Z ##[error]The operation was canceled.

https://pipelines.actions.githubusercontent.com/EMeCU0s3nV8mEzpqEGqbbDjtOUKCLJV4IPXg8mh4qtgdpzCGW6/_apis/pipelines/1/runs/22296/signedlogcontent/2?urlExpires=2022-02-11T22%3A39%3A36.7885983Z&urlSigningMethod=HMACV1&urlSignature=3qSfC9etL1o8Vw3X4I9uy5oLDi3VbGALXrpnBO%2FysiI%3D
This was referenced Feb 15, 2022
Closed
Merged
zymap pushed a commit that referenced this issue Feb 22, 2022
@dlg99
ISSUE 3044: ETCD tests hang. Added global timeout, fork tests jvm, fi…
1cf282c 
…xed noop slf4j to see log in case of hang (#3051)

Descriptions of the changes in this PR:

### Motivation

ETCD test flake / hang occasionally causing CI job timeout.

### Changes

Added global timeout - kill test early if hanged
fork tests jvm - I think it helped locally (no repro) but possibly just reduced frequency of hangs 
fixed noop slf4j warning, also to see log in case of hang 

Master Issue: #3044
@shoothzj
Copy link
Member

shoothzj commented May 3, 2024

there is no testRegisterLedgerMetadataListener method now. Close this issue.

@shoothzj shoothzj closed this as completed May 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flaky-tests type/bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@shoothzj @nicoloboschi