-
Notifications
You must be signed in to change notification settings - Fork 206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bad address - cannot read snapshot for v10:zoe - KERNEL PANIC #3877
Comments
I'm working on getting the relevant contents of |
Aug 26 06:06:48 agoric ag-chain-cosmos[16025]: cannot read snapshot /root/.ag-chain-cosmos/data/ag-cosmos-chain-state/xs-snapshots/3c12d2556f426be51c154bd623dc8d4aeb5c56d9e80f254a632ff55f14bf7c26-load-qVodTc.xss: Bad address have same issue. |
same here Aug 26 08:13:58 agoric-validator ag-chain-cosmos[1097]: cannot read snapshot /root/.ag-chain-cosmos/data/ag-cosmos-chain-state/xs-snapshots/0a5139af6d5c231df506b10f6c18e856caade55a2cbe4566d6a8822c22b00381-load-JDg594.xss: Bad address |
I uploaded slog file and the folder of snapshots as requested https://www.dropbox.com/s/ycz0cbwe2f58yk0/humantraffic-agorictest17-chain.slog.gz?dl=0 |
I have the same issue after performing my latest restart task. Aug 26 07:38:02 agoric ag-chain-cosmos[272306]: cannot read snapshot /home/***/.ag-chain-cosmos/data/ag-cosmos-chain-state/xs-snapshots/0662072aa8268afa30d4805714dba5b02b52e542e15115b5bea678b4076f482d-load-g145I9.xss: Bad address Slog File: https://drive.google.com/file/d/1XPOp4l8HzE5hKJxzipLEd-6URxzDZVOd/view |
THE SAME: (
|
Same here, I have got it on my RPC node Aug 26 10:47:28 Agoric-RPCnode ag-chain-cosmos[1037588]: cannot read snapshot /root/.ag-chain-cosmos/data/ag-cosmos-chain-state/xs-snapshots/2b3a7bd6674097027e4094b27e52a01382161337c8ee7a968dda4b48feb4ef9f-load-8WdaNm.xss: Bad address https://www.dropbox.com/s/7uu886yow5e1ds5/nataagoricRPC-xs-snapshots.tar.gz?dl=0 |
I have same issue. |
Hello, after restart i have this error too. This is full logs https://docs.google.com/spreadsheets/d/1H2QPSKDmtv2b5wUn8EVz2CI4S8Lo2cDQq-XABAlzy8A/edit?usp=sharing
|
i have this error! |
I have an error Bad address on the advice of the admin from the Discord, I publish here the files from the folder xs-snapshots |
https://drive.google.com/file/d/1PA9iun7nPk11EuaMsRIiAHXfYEnHBhpa/view?usp=sharing |
Hello, I also had the same issue commented here.
You can find here my xs-snapshots files : https://www.dropbox.com/s/qyz5clq5osc2wa2/xs-snapshots.zip?dl=0 |
gor same error. node failed to restart full log - https://pastebin.com/7Y3J6tnN |
Hello, my xs-snapshots files are here: https://drive.google.com/file/d/1y76aeF4C_29wMkCmCkhqNkYgilA0YKPz/view?usp=sharing |
For each task where you are struggling due to an issue beyond your control (such as this one), go ahead and fill out the task in the knack portal before the deadline, and include the URL of this issue If you later accomplish the task, just submit again. |
Thank you, @humantraffic , @krisboit, @alipostaci2001 ; I managed to download your xs-snapshots directories. I probably don't need any more, but thanks, everybody! |
i restart my node and such an error appeared ``` |
Thanks, @mrixl ... but for others coming here, I don't think we need more logs that look pretty much the same. Feel free to just 👍 the issue or something, and as I say in https://github.com/Agoric/testnet-notes/issues/33#issuecomment-906471131 , cite the URL of this issue in knack portal submissions. |
@warner do want the whole ag-chain-cosmos state directory? |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Would a few of you please share your whole It would probably save us time in reproducing the problem. Sorry I didn't ask for it in the first place. @humantraffic , @krisboit, @alipostaci2001 @sshamanov @Syd-ai |
Hey! I have the same issue. https://www.dropbox.com/s/er4lmj3jlfc8fo7/log.txt?dl=0 UPD: |
+1, Looks like a popular issue 😛 😛 |
yeah, np. |
xs-snapshots from validator : https://drive.google.com/file/d/1nXYdzru5Eq_dzyi5FFOhL0TPtWU5yZ-Z/view?usp=sharing |
Here you go https://drive.google.com/file/d/1QoiLuAvlh9x5prb01KJ6Lk3ARNvJ7lRF/view?usp=sharing Happy investigation 🙏 |
|
Hello, exactly the same problem, my post Agoric/testnet-notes#38 |
Thanks. It looks like I have a couple full node state backups now.
p.s. I think object storage a better fit for .tar.gz files...
|
same problem happened to our node as well |
@dckc any idea what might went wrong? Should I share my xs-snapshot here as well? |
I can reproduce the symptoms by trying to load the snapshot into one of our tools:
I'm struggling to come up with a more detailed diagnosis. I have reached out to our collaborators at Moddable for help. p.s. @warner it does not look like a case of deleting a snapshot too early. The compressed snapshot is there in the contributed diagnostic materials and the uncompressed snapshot. It's a little interesting that we don't delete the uncompressed snapshot in this error case. I don't think that was by design, but it's somewhat fortunate in this case. |
sdfUsing the
earlier episode: replay tool crashed: Cannot read property 'unmetered' of undefinedOuch... now what? hm.
version info
How the log file was extracted
jupyter@slog45nb:~/33-panic$ ls ~/agoric-sdk/packages/SwingSet/bin/ |
* fix a major memory leak: 64 bytes per Map `delete()`, 32 per Set `delete()` * should: closes #3839 * unfortunately Map/Set deletion is now O(N) not O(1) * possibly fix #3877 "cannot read (corrupted?) snapshot" Note that this breaks snapshot compatibility, and probably metering compatibility. closes #3889
We upgrade the XS submodule to the latest version: Moddable-OpenSource/moddable@10cc52e This fixes a major memory leak: 64 bytes per Map `delete()`, 32 per Set `delete()`. We believe this should: closes #3839 Unfortunately Map/Set deletion is now O(N) not O(1). This version of XS also fixes a bug that might be the cause of #3877 "cannot read (corrupted?) snapshot", but we're not sure. Note that this breaks snapshot compatibility (snapshots created before this version cannot be loaded by code after this version). It might also break metering compatibility, but the chances seem low enough that we decided to leave the metering version alone. closes #3889
We upgrade the XS submodule to the latest version: Moddable-OpenSource/moddable@10cc52e This fixes a major memory leak: 64 bytes per Map `delete()`, 32 per Set `delete()`. We believe this should: closes #3839 Unfortunately Map/Set deletion is now O(N) not O(1). This version of XS also fixes a bug that might be the cause of #3877 "cannot read (corrupted?) snapshot", but we're not sure. Note that this breaks snapshot compatibility (snapshots created before this version cannot be loaded by code after this version). It might also break metering compatibility, but the chances seem low enough that we decided to leave the metering version alone. closes #3889
reported in https://discord.com/channels/585576150827532298/819073555446759444/880169455412457602
and https://discord.com/channels/585576150827532298/819073555446759444/880176218673131570
The text was updated successfully, but these errors were encountered: