-
Notifications
You must be signed in to change notification settings - Fork 132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
jormungandr 0.8.2 stops syncing and node stats inconsistent #1374
Comments
I saw a similar problem where my block height stopped going up and got greater than 200 behind. I only saw it once, and a restart of the node temporarily cleared the problem. |
Dec 16 06:43:16.124 WARN blockchain is not moving up, the last block was 3603 seconds ago, task: stuck_notifier It happened again. |
Same problem on https://pooltool.io apparently |
Seeing the same issue. Apparantly I'm receiving blocks in the wrong order and afterwards it stops syncing. I see a critical Task Panick. It looks like I'm getting block announcements but they are no longer being applied. Dec 16 14:32:01.694 WARN response canceled by the processing task, request: UploadBlocks, sub_task: server, task: network After this I'm not getting any blocks applied. Finally I also notice a lot of task responces being canceled. |
Now I am seeing this error: Dec 16 16:17:08.946 ERRO cannot compute the time passed since the last block: second time provided was later than self, task: stuck_notifier |
Had the same problem with stuck node, restarted it and I'm having lots of these problems: I'll try stop the node and delete the storage to see if it makes a difference |
Entering this for Mr. Bojangles: I got Dec 16 21:52:48.911 ERRO cannot compute the time passed since the last block: second time provided was later than self, task: stuck_notifier |
I have a similar problem. This morning when I called the REST API with the following command: ./jcli rest v0 node stats get -h "http://127.0.0.1:3100/api". I only got the information that the call run into a timeout. All the other calls seem to work. (Balance check, Logs ...) After restarting the node i had many WARN entries in my LOG:
Yesterday i was able to connect to the trusted peers. |
Deadalus ITN having same problem now (in addition to standalone jormungandr). Possibly relevant Deadalus logs: |
I just rebooted my server and restarted jormungandr. There seems to be a problem with the following trusted peers:
The other ones seem to be fine. So the good thing is that my node is synching again. lastBlockHeight and blockRecvCnt is going up again and also lastBlockTime seems fine again. I will keep monitoring the logs. |
I just received a CRIT in my logs.
|
My logs filled with these:
|
still having this issue with version 0.8.3 |
@aalkilani regarding @Mixxaz @Rugbertl regarding @lordwotton29 @rickymac68 regarding @Rugbertl regarding timeouts of |
Describe the bug
A clear and concise description of what the bug is.
jormungandr running. cardano-wallet (12/13 release) serve mode running connected to jormungandr. As far as I can tell, this likely happened the first time cardano-wallet connected.
Symptoms:
Node stats (jcli rest v0 node stats get) reports a recent, up to date, lastBlockTime however upon examining the lastBlockHeight, it appears the height is not moving forward and does not match what the lastBlockTime is saying it should be.
At first, did not notice an issue however the cardano-wallet sync percent kept decreasing so I figuring something isn't fetching correctly. Given the lastblockTime, I thought Jormungandr was fine. Finally noticed lastBlockHeight was not moving at all. No error messages on either side (wallet or jormungandr).
Additional thoughts:
Reason I think it was the initial (1st time) cardano-wallet connected to jormungandr that caused this is that I never say cardano-wallet report that it was in sync and ready.. it started showing in the 99% and then just kept dropping so looks like upon connection, something happened.
Resolution:
Stopped cardano-wallet
Stopped jurmangandr
Started jurmangandr
Started cardano-wallet
Mandatory Information
jcli --full-version
output;jcli 0.8.2 (HEAD-fe327f9, release, linux [x86_64]) - [rustc 1.39.0 (4560ea788 2019-11-04)]
jormungandr --full-version
output;jormungandr 0.8.2 (HEAD-fe327f9, release, linux [x86_64]) - [rustc 1.39.0 (4560ea788 2019-11-04)]
To Reproduce
See above
Expected behavior
See above
Additional context
See above
The text was updated successfully, but these errors were encountered: