Hpx serialization #1446

AntonBikineev · 2015-04-02T20:33:04Z

Some inconsistent stuff still exists. Mostly it's related to polymorphic serialization because current polymorphic serialization mechanism(intrusive and nonintrusive) has to be refined.

additional check in locality constructor. modified: hpx/runtime/parcelset/locality.hpp

…ypes

…ircular dependency

…from libhpx sources

Conflicts: docs/manual/build_system/cmake_variables.qbk examples/quickstart/non_atomic_rma.cpp hpx/components/vector/distribution_policy.hpp hpx/components/vector/vector_configuration.hpp hpx/lcos/broadcast.hpp hpx/lcos/fold.hpp hpx/lcos/reduce.hpp hpx/parallel/algorithms/detail/dispatch.hpp hpx/performance_counters/counters.hpp hpx/plugins/parcelport/mpi/locality.hpp hpx/plugins/parcelport/tcp/sender.hpp hpx/runtime/actions/action_support.hpp hpx/runtime/agas/request.hpp hpx/runtime/components/stubs/runtime_support.hpp hpx/runtime/naming/name.hpp hpx/runtime/parcelset/decode_parcels.hpp hpx/runtime/parcelset/encode_parcels.hpp hpx/runtime/parcelset/locality.hpp hpx/runtime/parcelset/policies/mpi/sender.hpp hpx/serialization/input_container.hpp hpx/serialization/output_container.hpp hpx/util/basic_binary_iprimitive.hpp hpx/util/basic_binary_oprimitive.hpp hpx/util/portable_binary_iarchive.hpp hpx/util/portable_binary_oarchive.hpp hpx/util/serialize_empty_type.hpp hpx/util/serialize_sequence.hpp hpx/util/tuple.hpp src/CMakeLists.txt src/components/dataflow/dataflow_component.cpp src/components/vector/vector_configuration.cpp src/performance_counters/counters.cpp src/runtime/agas/big_boot_barrier.cpp src/runtime/components/server/runtime_support_server.cpp src/runtime/parcelset/parcel.cpp src/util/portable_binary_iarchive.cpp src/util/portable_binary_oarchive.cpp

Making things work on MSVC

into hpx_serialization

K-ballo · 2015-04-05T21:59:56Z

CMakeLists.txt

@@ -810,7 +814,7 @@ if(HPX_COMPILER_WARNINGS)
      hpx_add_compile_flag_if_available(-Werror=uninitialized)
    endif()

-    #hpx_add_compile_flag_if_available(-Werror=missing-field-initializers)
+    hpx_add_compile_flag_if_available(-Wno-missing-field-initializers)


Why was this warning disabled? It seems it was considered an error before. In case this is actually needed, it seems it should be moved out of the warnings-as-errors section.

K-ballo · 2015-04-05T22:21:37Z

Is there a way to tell if an Archive type is an archive, and whether it's an input or output one? Or some other way to implement serialization support without ever including a single bit of the serialization library itself?

K-ballo · 2015-04-05T22:23:14Z

hpx/runtime/actions/manage_object_action.hpp

 #include <hpx/util/reinitializable_static.hpp>

+#define HPX_SINGLE_ARG(...) __VA_ARGS__
+#define HPX_COMMA ,


This doesn't seem to be the proper place for these macro definitions, nor they seem to be used anywhere either. Are they still needed?

Forgot to delete them

K-ballo · 2015-04-05T22:52:09Z

Nitpick: the placement of braces after namespace is inconsistent across this PR. The dominant style within existing code is:

namespace hpx { namespace whatever // newline
{ // opening brace

I don't think this is explicitly stated anywhere, so I'd appreciate if others would chime in.

K-ballo · 2015-04-05T22:58:47Z

hpx/runtime/serialization/intrusive_ptr.hpp

+    }
+}}
+
+#endif


This header should include whatever defines boost::intrusive_ptr, even while it's probably already included by serialization/detail/pointer.hpp

K-ballo · 2015-04-05T23:14:36Z

Some save/load interfaces use input/output_archive as argument, others use a template argument (which I'd rather see renamed to something other than just T). What's the guideline here, when should I pick one or the other?

K-ballo · 2015-04-05T23:35:55Z

hpx/runtime/serialization/detail/polymorphic_intrusive_factory.hpp

+#include <boost/preprocessor/stringize.hpp>
+#include <boost/noncopyable.hpp>
+#include <boost/unordered_map.hpp>
+#include <boost/atomic/atomic.hpp>


Should be just <boost/atomic.hpp> for backward compatibility. See it fail with older boost versions here http://hermione.cct.lsu.edu/builders/hpx_gcc_4_6_boost_1_49_debian_x86_64_debug/builds/266/steps/build_core/logs/stdio

AntonBikineev · 2015-04-06T17:32:18Z

@K-ballo I everywhere tried to use template <class Archive> instead of just template <class T>. I think, no guidelines on templated/nontemplated versions are there. It's just easier for user to have Boost.Serialization compatible interface. We can stick to one way inside HPX, though.

AntonBikineev · 2015-04-06T18:51:31Z

Is there a way to tell if an Archive type is an archive, and whether it's an input or output one? Or some other way to implement serialization support without ever including a single bit of the serialization library itself?

I don't see any, if I understand you correctly.

sithhell · 2015-04-06T20:43:08Z

hpx/runtime/serialization/array.hpp

+        array(const array& rhs):
+            m_t(rhs.m_t),
+            m_element_count(rhs.m_element_count)
+        {}


This is missing move construction and assignment.

hkaiser · 2015-04-06T23:10:07Z

Is there a way to tell if an Archive type is an archive, and whether it's an input or output one? Or some other way to implement serialization support without ever including a single bit of the serialization library itself?

I don't see any, if I understand you correctly.

Would it make sense to introduce special traits for this? @K-ballo: I'm not sure what you have in mind, but wouldn't traits handle your use case?

K-ballo · 2015-04-06T23:20:24Z

Would it make sense to introduce special traits for this? @K-ballo: I'm not sure what you have in mind, but wouldn't traits handle your use case?

Traits would still require including some library code in order to support serialization. With Boost.Serialization, it is sometimes possible to implement serialize SFINAEd on Archive::is_saving, and thus does not require including anything extra. The existing of the is_saving member is interpreted as Archive modeling the Archive concept, and the save/load split can be done based on its value. This was likely just by chance for Boost.Serialization, but I would really like to see some similar way of adding serialization support without actually introducing a hard dependency nor a single extra #include.

sithhell · 2015-04-14T06:19:54Z

Looks good to me! I think we should merge the changes as soon as possible now.

biddisco · 2015-04-14T06:59:07Z

Merging this will help with development of parcelport code which depends heavily on serialization.

Hpx serialization

sithhell and others added 30 commits October 25, 2014 12:53

First steps to replace boost.serialization

91168fe

Merge branch 'master' into hpx_serialization

f28aab6

Some ambiguous calls of map::erase has been prevented by adding

17df510

additional check in locality constructor. modified: hpx/runtime/parcelset/locality.hpp

Polymorphic serialization support has been added.

dad5043

Some useless comments in serialization tests have been removed

64d828f

Polymorphic nonintrusive serialization has been added for arbitrary t…

022f65e

…ypes

hpx::util::tuple has been changed to boost::fusion::vector to avoid c…

072713a

…ircular dependency

boost::fusion::vector has been changed to regular class

46e2159

typename is serialized together with hash

9b7cc23

boost unordered_map keys are strings

50d47f3

Polimorphic nonitrusive factory was slightly refined

616bebd

removed duplicate declarations

000a854

added support of map serialization

1b95e18

whole bunch of stuff was committed

3e12a5b

serialization functions has been moved to headers

12a6281

portable archives for boost.serialization have been deleted

d35a66a

array serialization has been added

40ddf24

Added support for boost and std array serialization.

928c970

base_object is set to be serialized as prvalue

66b3b41

Boost.Serialization has been dropped off from libhpx

2c9682d

The rest of dependencies on Boost.Serialization has been dropped off …

1712e29

…from libhpx sources

Serilization_fwd has been added

b12d9c9

Some cleaning up after merging

146e0a6

Making things work on MSVC

5982142

Thomas's suggestion about insertion hints has been applied

8adce5b

Merge pull request #1 from hkaiser/hpx_serialization

bf0caca

Making things work on MSVC

Merge branch 'hpx_serialization' of https://github.com/AntonBikineev/hpx

5e339f7

into hpx_serialization

Some artefacts have been deleted

3980921

Array optimization problem seems to be removed

7d81f96

K-ballo reviewed Apr 5, 2015
View reviewed changes

sithhell reviewed Apr 6, 2015
View reviewed changes

AntonBikineev self-assigned this Apr 8, 2015

AntonBikineev added 2 commits April 10, 2015 04:59

First comments from PR applied

bff71ba

deleted Boost.Foreach dependency; nullptr was changed to 0

6c25239

Merge branch 'master' into hpx_serialization

66c73e8

hkaiser added a commit that referenced this pull request Apr 14, 2015

Merge pull request #1446 from STEllAR-GROUP/hpx_serialization

66ec7ef

Hpx serialization

hkaiser merged commit 66ec7ef into master Apr 14, 2015

sithhell deleted the hpx_serialization branch April 16, 2015 07:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hpx serialization #1446

Hpx serialization #1446

AntonBikineev commented Apr 2, 2015

K-ballo Apr 5, 2015

K-ballo commented Apr 5, 2015

K-ballo Apr 5, 2015

AntonBikineev Apr 5, 2015

K-ballo commented Apr 5, 2015

K-ballo Apr 5, 2015

K-ballo commented Apr 5, 2015

K-ballo Apr 5, 2015

AntonBikineev commented Apr 6, 2015

AntonBikineev commented Apr 6, 2015

sithhell Apr 6, 2015

hkaiser commented Apr 6, 2015

K-ballo commented Apr 6, 2015

sithhell commented Apr 14, 2015

biddisco commented Apr 14, 2015

+                  }
+              }}
+              #endif

Hpx serialization #1446

Hpx serialization #1446

Conversation

AntonBikineev commented Apr 2, 2015

K-ballo Apr 5, 2015

Choose a reason for hiding this comment

K-ballo commented Apr 5, 2015

K-ballo Apr 5, 2015

Choose a reason for hiding this comment

AntonBikineev Apr 5, 2015

Choose a reason for hiding this comment

K-ballo commented Apr 5, 2015

K-ballo Apr 5, 2015

Choose a reason for hiding this comment

K-ballo commented Apr 5, 2015

K-ballo Apr 5, 2015

Choose a reason for hiding this comment

AntonBikineev commented Apr 6, 2015

AntonBikineev commented Apr 6, 2015

sithhell Apr 6, 2015

Choose a reason for hiding this comment

hkaiser commented Apr 6, 2015

K-ballo commented Apr 6, 2015

sithhell commented Apr 14, 2015

biddisco commented Apr 14, 2015