Implement 'migration' to and from disk

[hkaiser]

Implement an alternative backend for migration which stores the byte stream to disk and retrieves it back from disk.

[hkaiser]

I think we need to first solve the actual migration from somewhere's memory to somewhere else's disk. No AGAS has to get involved here. Just the actual operation of saving an object and of bringing it back. The next step would be to integrate this functionality into AGAS (you're right, AGAS needs to know whether a particular gid refers to an object which is alive).

So let's have a look at the first step. This involves two different things: a) storing an object, and b) bringing it back.

Currently, migrate looks like:

    template <typename Component>
    future<id_type> migrate(
        naming::id_type const& to_migrate,
        naming::id_type const& target_locality);

Here, target_locality is assumed to be a locality id, which will invoke the (plain) migrate_component-action on the locality where to_migrate currently lives (see: https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/runtime/components/server/migrate_component.hpp#L73). This action talks to the locality where the object should be migrated to, which causes the object to be 'pulled' over (see here: https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/runtime/components/server/migrate_component.hpp#L63). Since the target_locality is a locality id, the call ends up in the runtime_support component (see: https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/runtime/components/server/runtime_support.hpp#L153).

a) For the new 'migration' to disk, target_locality will not be a locality id, but will be the id of a special component which exposes a component-action with the same prototype as the migrate_component_to_here-action (for normal migration this is implemented in the runtime_support object). So you will need something like:

    class migrate_to_persistence 
      : public simple_component_base<migrate_to_persistence>
    {
    public:
        template <typename Component>
        naming::gid_type migrate_component_to_here(
            boost::shared_ptr<Component> const& p, naming::id_type);

        // ... macros, etc.
    };

Component actions which are templates are a bit tricky, but the template-accumulator example shows how to use them.

Having a component will make HPX distinguish between 'normal' migration and your new one. We could even use predefined ids for this component, i.e. derive it from fixed_component_base, but that's something for later.

All this will utilize the existing migrate infrastructure, it will just change the receiving end of the operation to save the object.

b) Bringing the object back is a bit more complicated. Here we can re-utilize the backend which already exists in the runtime_support object (receiving the data and recreating the object), but we need to provide a new implementation of the source of the byte stream. The current source of the byte stream is implemented by the hpx::detail::get_ptr_for_migration function here: https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/runtime/get_ptr.hpp#L72. The naming::address is what AGAS reports as the current location of the object.

What I'd suggest is to add the information that the object is currently on disk to the naming::address. Not sure how to do that at this point... We need to think about it. In the end, get_ptr_for_migration() will have to be changed to fetch the byte stream from disk, if needed.

This again would allow for keeping the existing migration infrastructure in place.

[EverYoung124]

Hi Hartmut,

I am thinking of using the tuple space to store the byte representation of components. The total scheme is as the follows:

1. De-serialize the component into a byte stream and store into tuple space as
   [ gid | bytesteam ]
2. Store the tuple space into disk frequently.
3. When trying to load the stored component, it will look up the tuple space.

I feel that this approach can help manage the total runtime environment and act as a storage locality.

[hkaiser]

Yah sure, that should work. At least it decouples the migration part from the underlying file system part...

[hkaiser]

This has been implemented by merging #1375: Fixing 1163

[sithhell]

While merging #1375 is the first into that direction, it hasn't implemented disk storage. reopening.

[hkaiser]

The first steps have been implemented, moving the rest to the next milestone...

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed. Please re-open if necessary.