Serialization of types without default constructor

[krivenko]

Dear HPX developers,

Are there any plans to support serialization of std::vector<T> (and similar STL containers), where T has no default constructor? This feature exists in boost::serialization but not in hpx::serialization, as far as I can see.

Thanks in advance,
Igor

[AntonBikineev]

Hi Igor, Indeed it doesn’t exist, yet it should be trivial to implement. Will do this this week. Thanks!

[krivenko]

Thank you very much for adding this feature!
I've been trying to verify #2805, and there is something I don't quite understand.
What is the actual serialization protocol, or in other words, what should be done in serialize() and what in save_construct_data()/load_construct_data()?

To illustrate my problem I took tests/unit/serialization/serialization_list.cpp and added one more test function to it.

void test_non_default_constructible_single() {

    std::vector<char> buffer;
    hpx::serialization::output_archive oarchive(buffer);

    B single_b_1(1);
    single_b_1.set_b(2.0);
    oarchive << single_b_1;

    hpx::serialization::input_archive iarchive(buffer);

    B single_b_2(3);
    single_b_2.set_b(4.0);
    iarchive >> single_b_2;
    
    HPX_TEST_EQ(single_b_1.get_a(), single_b_2.get_a());
    HPX_TEST_EQ(single_b_1.get_b(), single_b_2.get_b());
}

Here is what I get:

hpx.git/tests/unit/serialization/serialization_list.cpp(262): test 'single_b_1.get_a() == single_b_2.get_a()' failed in function 'void test_non_default_constructible_single()': '1' != '3'
0 sanity checks and 1 test failed.

That check fails because load_construct_data() is never called. Therefore, deserialization of B::a never occurs.

[AntonBikineev]

Igor, In the provided patch save_construct_data and load_construct_data functions are only called when doing container serialization. There are a couple of reasons why I decided to stick to this design: 1) containers are responsible for allocating the storage of their value types and constructing them, 2) to be honest I think that these function-hooks already make serialization interface overcomplicated. Ideally, those functions should have been called in a different way so that they would have mentioned containers or something, but I decided to stay compatible with Boost.Serialization. Note that this implementation is compatible with that of Boost.Serialization — it only calls save_construct_data/load_construct_data for collections and pointers (this implementation is currently missing deserialization of pointers to non-default-constructible data). To answer your question, my reasoning about serialize and *_construct_data is that the former should be used to serialize the internal data of a class and the latter is used to serialize any external data that might be used for constructing an object of that class, so that containers would be able to deserialize them. Thanks.

…

On Aug 8, 2017, at 9:40 PM, Igor Krivenko ***@***.***> wrote: Thank you very much for adding this feature! I've been trying to verify #2805 <#2805>, and there is something I don't quite understand. What is the actual serialization protocol, or in other words, what should be done in serialize() and what in save_construct_data()/load_construct_data()? To illustrate my problem I took tests/unit/serialization/serialization_list.cpp and added one more test function to it. void test_non_default_constructible_single() { std::vector<char> buffer; hpx::serialization::output_archive oarchive(buffer); B single_b_1(1); single_b_1.set_b(2.0); oarchive << single_b_1; hpx::serialization::input_archive iarchive(buffer); B single_b_2(3); single_b_2.set_b(4.0); iarchive >> single_b_2; HPX_TEST_EQ(single_b_1.get_a(), single_b_2.get_a()); HPX_TEST_EQ(single_b_1.get_b(), single_b_2.get_b()); } Here is what I get: hpx.git/tests/unit/serialization/serialization_list.cpp(262): test 'single_b_1.get_a() == single_b_2.get_a()' failed in function 'void test_non_default_constructible_single()': '1' != '3' 0 sanity checks and 1 test failed. That check fails because load_construct_data() is never called. Therefore, deserialization of B::a never occurs. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2795 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFrQvmiD9dYQEyNNG5Rps1FAmzQW73nUks5sWKuigaJpZM4OqlGv>.

[krivenko]

Thanks a lot for the fix and the explanation!

Note that this implementation is compatible with that of Boost.Serialization — it only calls save_construct_data/load_construct_data for collections and pointers (this implementation is currently missing deserialization of pointers to non-default-constructible data).

Okay, I guess it indeed makes no sense to overcomplicate things and to go beyond what Boost does.

I'll then adjust my code to construct some dummy objects in save_construct_data/load_construct_data and to do the actual serialization in serialize(). This would still be much better than adding default constructors everywhere.

[hkaiser]

@krivenko I take it that you're fine with merging #2805 now. Is that assumption correct?

[krivenko]

@hkaiser Yes, go ahead!