Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dead letters logging in sandbox when shutting down #1886

Closed
gerolf-da opened this issue Jun 26, 2019 · 0 comments · Fixed by #2471
Closed

Dead letters logging in sandbox when shutting down #1886

gerolf-da opened this issue Jun 26, 2019 · 0 comments · Fixed by #2471
Labels
component/ledger Sandbox and Ledger API

Comments

@gerolf-da
Copy link
Contributor

gerolf-da commented Jun 26, 2019 •
edited
Loading

Sandbox logs the following message about dead letters when shutting down.
We should find out which part (rs-grpc-bridge in the log is probably a good hint) isn't being shutdown in time and make sure that it happens.

[INFO] [06/26/2019 11:42:18.335] [scala-execution-context-global-1661] [akka://sandbox/user/api-server-damlonx-rs-grpc-bridge-f93307ec-0109-4eef-9b4a-389477ec6c4d-0] Message [com.digitalasset.grpc.adapter.server.rs.ServerSubscriber$$Lambda$24530/2145535365] without sender to Actor[akka://sandbox/user/api-server-damlonx-rs-grpc-bridge-f93307ec-0109-4eef-9b4a-389477ec6c4d-0#-1289723685] was not delivered. [1] dead letters encountered. If this is not an expected behavior, then [Actor[akka://sandbox/user/api-server-damlonx-rs-grpc-bridge-f93307ec-0109-4eef-9b4a-389477ec6c4d-0#-1289723685]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
@gerolf-da gerolf-da added the component/ledger Sandbox and Ledger API label Jun 26, 2019
bitonic added a commit that referenced this issue Jul 4, 2019
@bitonic
add scenario tests for #1886 (WIP)
d54a5ae
gerolf-da added a commit that referenced this issue Aug 9, 2019
@gerolf-da
Fix shutdown order to avoid dead letter logging when stopping sandbox
7f18e9f 
It looks like that closing the LedgerApiServer before closing the
AkkaExecutionSequencerPool (AESP) leads to a non-deterministic bunch of dead
letter warnings when stopping sandbox.

Closing the AESP first avoids these warnings.

Fixes #1886
@gerolf-da gerolf-da mentioned this issue Aug 9, 2019
Merged
6 tasks
@mergify mergify bot closed this as completed in #2471 Aug 9, 2019
mergify bot pushed a commit that referenced this issue Aug 9, 2019
@gerolf-da @mergify
Fix shutdown order to avoid dead letter logging when stopping sandbox (…
353a19c 
…#2471)

It looks like that closing the LedgerApiServer before closing the
AkkaExecutionSequencerPool (AESP) leads to a non-deterministic bunch of dead
letter warnings when stopping sandbox.

Closing the AESP first avoids these warnings.

Fixes #1886
@gerolf-da gerolf-da mentioned this issue Aug 9, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component/ledger Sandbox and Ledger API
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix shutdown order to avoid dead letter logging when stopping sandbox digital-asset/daml
1 participant
@gerolf-da