save file can not be loaded

[jimishol]

On today's compilation, save button works but, load button remains inactive. Continue button finds errors on autosaved file (_exit file i believe). May be the bug is related with the disappearance of edit and delete custom logs despite that their disappearance occurred several weeks before today.
Now i play with the precompiled version 20210213
Through lua interpreter i manage to get my money, ship and equipment back but NOT my crew that was trained enough to have his statistics well above 10% than i can find now. (It is not clear to me if his training was because he got orders to drive the ship or he was just selftrained on my ship).
So, before getting to next precompiled version, pls fix at least saves if not the custom logs too.
I don't paste save file because i believe this bug happens to anyone.

[Web-eWorks]

Save file incompatibility between compiled version and development version is to be expected with a savegame bump. Additionally, it's possible that there is an error in the save file, or that an incompatibility between two development versions was created. We intentionally make no guarantees that save files are compatible between two development versions, and only strive to maintain a "best effort" compatibility with the previous stable build.

Without a save file and more detailed information about which build the save was created with and the build that cannot load it, we cannot even attempt to diagnose and resolve the issue you are experiencing.

[impaktor]

Just to repeat what I mentioned in IRC, that I think current master aught to be compatible with pioneer from November 2020, and onwards.

[impaktor]

I think I managed to load a new save sometimes perhaps something that can't be saved wasn't created in that station/world.

Here's a back trace after running gdb with catch throw. (I'm on Arch, gcc --version says 10.2.0, but pacman -Qi says 11.1.0)

Looks like gcc first is complaining about missing eh_throw.cc, then down into the json contrib library?

I've done a /bootstrap & make, from current master, e9fb865 (june 7)

Thread 1 "pioneer" hit Catchpoint 1 (exception thrown), 0x00007ffff6debf52 in __cxxabiv1::__cxa_throw (obj=0x555565a4b2b0, 
    tinfo=0x555555ad1778 <typeinfo for nlohmann::detail::parse_error>, dest=0x5555557b4b30 <nlohmann::detail::parse_error::~parse_error()>)
    at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:78
78	/build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc: No such file or directory.
(gdb) bt
#0  0x00007ffff6debf52 in __cxxabiv1::__cxa_throw (obj=0x555565a4b2b0, tinfo=0x555555ad1778 <typeinfo for nlohmann::detail::parse_error>, 
    dest=0x5555557b4b30 <nlohmann::detail::parse_error::~parse_error()>) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:78
#1  0x00005555555c52ec in nlohmann::detail::json_sax_dom_callback_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> >::parse_error (ex=..., this=<optimized out>)
    at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:4289
#2  0x00005555555c52f4 in nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> >::parse_error (this=<optimized out>, ex=...)
    at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:4047
#3  0x00005555557b6657 in nlohmann::detail::binary_reader<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>, nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >::parse_cbor_internal (
    this=0x7fffffffcd80, get_char=<optimized out>) at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:6430
#4  0x00005555557b643c in nlohmann::detail::binary_reader<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>, nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >::get_cbor_object (
    this=<optimized out>, len=3) at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:7060
#5  0x00005555557b643c in nlohmann::detail::binary_reader<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>, nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >::get_cbor_object (
    this=<optimized out>, len=30) at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:7060
#6  0x00005555557b643c in nlohmann::detail::binary_reader<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>, nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >::get_cbor_object (
    this=<optimized out>, len=5) at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:7060
#7  0x00005555557b6e46 in nlohmann::detail::binary_reader<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>, nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >::get_cbor_array (
    this=0x7fffffffcd80, len=323) at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:7018
#8  0x00005555557b643c in nlohmann::detail::binary_reader<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>, nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >::get_cbor_object (
    this=<optimized out>, len=3) at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:7060
#9  0x00005555557b643c in nlohmann::detail::binary_reader<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>, nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >::get_cbor_object (
    this=<optimized out>, len=16) at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:7060
#10 0x00005555557b6f80 in nlohmann::detail::binary_reader<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>, nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> > >::sax_parse (strict=true, 
    sax_=0x7fffffffcd50, format=nlohmann::detail::input_format_t::cbor, this=0x7fffffffcd80) at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:6049
#11 nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>::from_cbor (i=..., strict=strict@entry=true, allow_exceptions=allow_exceptions@entry=true)
    at /home/karlf/usr/src/pioneer/contrib/json/json.hpp:17804
#12 0x00005555557adbfb in JsonUtils::LoadJsonSaveFile (filename="savefiles/does_save_work", source=...) at /home/karlf/usr/src/pioneer/src/JsonUtils.cpp:98
#13 0x0000555555767192 in Game::LoadGameToJson (filename="does_save_work") at /home/karlf/usr/src/pioneer/src/Game.cpp:867
#14 0x000055555587c2e6 in l_game_savegame_stats (l=0x555555e18bd0) at /home/karlf/usr/src/pioneer/src/lua/LuaGame.cpp:104
#15 0x00005555558ca15e in luaD_precall (L=L@entry=0x555555e18bd0, func=<optimized out>, nresults=<optimized out>) at /home/karlf/usr/src/pioneer/contrib/lua/src/ldo.c:318
#16 0x00005555558ca4b2 in luaD_call (L=0x555555e18bd0, func=<optimized out>, nResults=<optimized out>, allowyield=<optimized out>)
    at /home/karlf/usr/src/pioneer/contrib/lua/src/ldo.c:394
#17 0x00005555558c9a9c in luaD_rawrunprotected (L=L@entry=0x555555e18bd0, f=f@entry=0x5555558c0b20 <f_call>, ud=ud@entry=0x7fffffffd540)
    at /home/karlf/usr/src/pioneer/contrib/lua/src/ldo.c:131
#18 0x00005555558ca71f in luaD_pcall (L=L@entry=0x555555e18bd0, func=func@entry=0x5555558c0b20 <f_call>, u=u@entry=0x7fffffffd540, old_top=944, ef=<optimized out>)
    at /home/karlf/usr/src/pioneer/contrib/lua/src/ldo.c:595
#19 0x00005555558c3337 in lua_pcallk (L=L@entry=0x555555e18bd0, nargs=<optimized out>, nresults=nresults@entry=-1, errfunc=errfunc@entry=0, ctx=ctx@entry=0, 
    k=k@entry=0x5555558c6870 <pcallcont>) at /home/karlf/usr/src/pioneer/contrib/lua/src/lapi.c:949
#20 0x00005555558c68d7 in luaB_pcall (L=0x555555e18bd0) at /home/karlf/usr/src/pioneer/contrib/lua/src/lbaselib.c:393
#21 0x00005555558ca15e in luaD_precall (L=L@entry=0x555555e18bd0, func=<optimized out>, func@entry=0x5555568bb220, nresults=nresults@entry=2)

[Web-eWorks]

Running the provided savefile against some CBOR validation tools, it looks like the issue is on the serialization side; the produced save file is unable to be read by the official CBOR tools.

does_save_work.zip

[akien-mga]

I seem to be having a similar issue with 20210203, which blocks the update I planned to provide to Mageia users (from 20200203). Any save file created with 20210203 seems to be corrupted.

For comparison, the file size between 2020 and 2021 saves changed significantly:

$ ls -l
total 456
-rw-r--r-- 1 akien akien 408177 Jun 23 18:05 Test2020
-rw-r--r-- 1 akien akien  56974 Jun 23 18:07 Test2021

(I did also try to remove ~/.pioneer before using 2021, the save files are still corrupted similarly.)

Test2021.zip

Steps to reproduce:

• Build and install 20210203 (here's my RPM spec file: http://svnweb.mageia.org/packages/cauldron/pioneerspacesim/current/SPECS/pioneerspacesim.spec?view=markup)
• rm -rf ~/.pioneer to be safe
• Run game, start Mars scenario, save file
• Check save or load menu, tooltip says the file is corrupted

[Web-eWorks]

For completeness sake, could you provide information about which compiler (and version) you're using to compile Pioneer? One user has reported they cannot reproduce the problem with a prebuilt 20210203 binary but can reproduce it with later development versions compiled locally, which makes me think this is caused by a compatibility issue between the json library and newer versions of GCC...

[akien-mga]

Sure, I'm using GCC 11.1:

$ gcc --version
gcc (Mageia 11.1.1-0.20210619.1.mga9) 11.1.1 20210619

[Web-eWorks]

@akien-mga / @jimishol I'd appreciate it if you can test the linked pull request; it appears the issue is caused by a GCC 11 optimization that makes our accidental use of UB break in the specific case of converting JSON data to CBOR output.

[jimishol]

I didn't know how to properly handle the linked PR, so, since it was only 4 lines to be added, i copied pasted them editing my local same files. After compilation, save load work fine, once again!!.
Congratulations and thank you very much.

[akien-mga]

I confirm the fix too with #5218 👍

[Web-eWorks]

Thank you for confirming! Glad to have it fixed! We'll likely issue a small intermediary release with this and a few other QOL fixes soon.