Save file corruption

[naxuroqa]

Yesterday I was experimenting with the latest version of toxcore from git/master and in about one out of ten times of saving the data file, it got corrupted and could not be loaded anymore. I'll try to get some more info about what exactly fails to be restored, for now I just wanted to find out if somebody else experienced this.

This never happened to me with 0.2.8, so I think that's a regression.

toxcore version: 9770880

Arch Linux x86_64 4.14.63-1-lts
libsodium 1.0.16

[zoff99]

@naxuroqa are you sure you are not allocating on the stack?
also it could be #1242

[naxuroqa]

I'm creating a new uint8 array of size tox_get_savedata_size(), and call tox_get_savedata() with this array (https://github.com/naxuroqa/Venom/blob/develop/src/vapi/toxcore.vapi#L1205-L1218), which is called in the tox session destructor, after stopping and joining on the session thread (https://github.com/naxuroqa/Venom/blob/develop/src/tox/ToxSession.vala#L235-L244).
This then gets saved to the save file location (https://github.com/naxuroqa/Venom/blob/develop/src/core/Profile.vala#L72-L83).

Creating an array with new in vala allocates it on the heap.

There were no runtime errors in this process, otherwise I would have seen them logged, and neither has this ever happened with 0.2.8, ever.

[zoff99]

when it is allocated on the heap it should be ok. most likely you need to wait for #1242 to be merged.
there could be still other bugs, because even with #1242 i get corrupted profile with utox all the time still

[zugz]

* Tuesday, 2018-12-04 at 01:06 -0800 - naxuroqa <notifications@github.com>:

Yesterday I was experimenting with the latest version of toxcore from git/master and in about one out of ten times of saving the data file, it got corrupted and could not be loaded anymore.

That's worrying. More precise information would be helpful. One question which would might narrow it down: were you using groups at all when you ran into save corruption?

[naxuroqa]

@zugz I was, that's why I tried it out :)

Is there anything I can do to provide you with more info? I can try to get a savefile to corrupt and send it to you if that helps.

[zugz]

* Tuesday, 2018-12-04 at 09:38 -0800 - naxuroqa <notifications@github.com>:

I can try to get a savefile to corrupt and send it to you if that helps.

That would definitely help!

[zoff99]

does analyzing make sense? we should merge #1242 before that. i don't understand what the holdup is there?

[zugz]

Actually, I think I see what must be going on. If I'm right, it should be fixed in #1267. My theory is that the problem was that your public key somehow ended up appearing twice in the list of peers and frozen peers, with different peer numbers. That shouldn't be possible in #1267, but in current master it could have happened if you accepted two different invites to the same group, for example. The saving code implicitly assumed that this didn't happen, and wrote something unloadable if it did. I've just pushed a commit to #1267 which removes that assumption. So: could you try #1267 and report back, preferably with a corrupted save file, if you run into a problem again? Thanks for the report, in any case!

[zugz]

* Tuesday, 2018-12-04 at 18:54 +0000 - Zoff <notifications@github.com>:

does analyzing make sense? we should merge #1242 before that.

It looks like this one was my fault, so probably nothing to do with #1242.

[zoff99]

@zugz :-) ok mate
still we need to merge #1242
it's a critical bug with crashes

[sudden6]

This should be fixed with 0.2.9, right? can anyone confirm?

[naxuroqa]

@sudden6 I'll see if 0.2.9 fixed it, will close if it doesn't occur anymore.

[anthonybilinski]

Fixed for me, tested pre and post fix: #1267 (comment)
There's also a recovery option there if your save file was corrupted.

[naxuroqa]

oh, nice. So it was only partially corrupted?

You are right, I can't reproduce the corruption anymore.
Fixed for me as well.