Migrate components #1966

hkaiser · 2016-01-23T15:55:41Z

This fixes the remaining open issues in #559.

Transparent migration of arbitrary components is fully implemented now - \o/

…anage pin counts

sithhell · 2016-01-23T20:15:23Z

hpx/runtime/components/migrate_component.hpp

-        typedef server::trigger_migrate_component_action<Component> action_type;
-        return async<action_type>(naming::get_locality_from_id(to_migrate),
+        typedef server::perform_migrate_component_action<Component> action_type;
+        return hpx::detail::async_colocated<action_type>(to_migrate,


Why don't you use hpx::components::colocated here?

It's an internal invocation, so I directly call the implementation for the colocated async.

sithhell · 2016-01-23T21:21:05Z

The code currently tries to unpin an already migrated component right after the migration has been completed: https://gist.github.com/sithhell/44bf05eb58c1a9b83eaa

…comments.

…as located earlier

- added missing checks for was_migrated - routing of just received parcel is now done with normal priority (if local) - store stripped gids in AGAS was_migrated table - add more tests - flyby changes: -- add optional deadlock detection in spinlock (very crude!) -- minor move optimizations in parcel-port and AGAS -- renamed HPX_THREAD_MINIMAL_DEADLOCK_DETECTION to HPX_HAVE_THREAD_...

- flyby: write error message for failing tests

…nctional now - added more asserts to migration code - diagnostic printouts in migrate_component test

- adding serialization to distribution policies

hkaiser · 2016-01-31T16:53:27Z

All discussed issues have been addressed. This PR is ready to be merged.

- implemented segmented iterator for hpx::unordered_map - implemented serialization for std::unordered_map and corresponding test - refactored test for component storage - flyby: renamed partition_vector to partitioned_vector_partition this fixes #1163

- flyby: fixed un-registration of migrated simple components

hkaiser · 2016-02-03T13:04:47Z

@sithhell Which of the tests is causing this?

sithhell · 2016-02-03T14:53:09Z

@hkaiser migrate_component

hkaiser · 2016-02-03T15:19:05Z

@sithhell I understand that migrate_component fails. I meant what sub-test is failing? Is it blowing up always or is it a race?

sithhell · 2016-02-03T15:36:26Z

For completeness: The eariler post has been deleted because it used an earlier version. However, the problem is still there. Running two localities, one thread per locality.

On 02/03/2016 04:19 PM, Hartmut Kaiser wrote:

@sithhell https://github.com/sithhell I understand that
migrate_component fails. I meant what sub-test is failing? Is it blowing
up always or is it a race?

Seems to blow up always. Looks like I used a earlier version, however,
after updating, here is the full error:
https://gist.github.com/sithhell/c963f9a8957f6e1d2d16

A release build, without address sanitizer, segfaults as well and gives
this output:
https://gist.github.com/sithhell/70c29cbc3e9bf5482ca2

sithhell · 2016-02-03T15:43:55Z

There seems to be one leftover problem with caching. when disabling the cache, it works. However, once using more than one thread per locality, the test_migrate_busy_component2 test either hangs (in release):

[16:43:06]:heller@luna:/home/heller/build/hpx/release:0:$ mpirun -np 2 ./bin/migrate_component_test -Ihpx.stacks.small_size=0x20000 -t2 -Ihpx.agas.use_caching=0
test_migrate_component: ->{0000000200000000, 0000000000000000}
test_migrate_component: <-{0000000200000000, 0000000000000000}
test_migrate_busy_component: ->{0000000200000000, 0000000000000000}
test_migrate_busy_component: <-{0000000200000000, 0000000000000000}
test_migrate_component2: ->{0000000200000000, 0000000000000000}
....................................................................................................
test_migrate_component2: <-{0000000200000000, 0000000000000000}
....................................................................................................
test_migrate_busy_component2: ->{0000000200000000, 0000000000000000}
001.1100.001.100.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

or prints this output (debug + sanitizer):

mpirun -np 2 ./bin/migrate_component_test -Ihpx.stacks.small_size=0x20000 -t2 -Ihpx.agas.use_caching=0
test_migrate_component: ->{0000000200000000, 0000000000000000}
test_migrate_component: <-{0000000200000000, 0000000000000000}
test_migrate_busy_component: ->{0000000200000000, 0000000000000000}
test_migrate_busy_component: <-{0000000200000000, 0000000000000000}
test_migrate_component2: ->{0000000200000000, 0000000000000000}
....................................................................................................
test_migrate_component2: <-{0000000200000000, 0000000000000000}
......................................migrate_component_test: /home/heller/hpx/hpx/runtime/get_ptr.hpp:55: void hpx::detail::get_ptr_for_migration_deleter::operator()(Component *) [Component = test_server]: Assertion `was_migrated' failed.
[luna:mpi_rank_1][error_sighandler] Caught error: Aborted (signal 6)

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 22034 RUNNING AT luna
=   EXIT CODE: 6
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Aborted (signal 6)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions

hkaiser · 2016-02-03T17:27:32Z

Disabling caching just causes for more networking to happen as things will never be resolved by the AGAS caches. I think this has no bearing wrt the migration operation itself, just exposes the problem in a different way.

hkaiser · 2016-02-03T18:59:27Z

@sithhell https://github.com/sithhell I understand that migrate_component fails. I meant what sub-test is failing? Is it blowing up always or is it a race?

Seems to blow up always. Looks like I used a earlier version, however, after updating, here is the full error: https://gist.github.com/sithhell/c963f9a8957f6e1d2d16

Which of the localities is failing? the one where the object comes from or the one where it goes to?

sithhell · 2016-02-04T08:48:52Z

The latest commit makes it work for the one thread per locality case, the more than one thread per locality test is still failing with the same symptoms.

sithhell · 2016-02-08T07:20:43Z

\o/
The latest commits seem to fix the remaining issues with component migration!

The bad news ... the touched migration to storage tests fail now:

segfault in release mode
assertion failure in debug (https://gist.github.com/sithhell/e369b3879d9a17ba9a93)

hkaiser · 2016-02-08T13:49:01Z

The assertion you're seeing is the same as reported in #1944. It seems to be unrelated to the migration functionality. By the looks of it, the code which handles sending parcels to gids which are not AGAS cached misses to set the destination locality properly under certain circumstances.

- this removes the changes to migrate_[to|from]_storage

Migrate components

hkaiser added 5 commits January 18, 2016 18:39

Started to implement last missing pieces for component migration (see #…

48f97a4

…559)

Implemented migration of components which are busy

0e881f7

Fixing race in applier::schedule_action, using scoped pinned_ptr to m…

1d52d81

…anage pin counts

Simplify/extend test, fix lock being held during suspension

5717b94

Fix problems with examples

85e2482

hkaiser added category: AGAS type: enhancement category: components labels Jan 23, 2016

hkaiser added this to the 0.9.12 milestone Jan 23, 2016

hkaiser mentioned this pull request Jan 23, 2016

Add hpx::migrate facility #559

Closed

Comment fixes

4505899

hkaiser force-pushed the migrate_component branch 3 times, most recently from f402153 to 1762853 Compare January 23, 2016 18:48

Simplify implementation of was_object_migrated trait

c1886da

hkaiser force-pushed the migrate_component branch from 1762853 to c1886da Compare January 23, 2016 19:31

sithhell reviewed Jan 23, 2016
View reviewed changes

sithhell and others added 8 commits January 23, 2016 22:25

Extending migration unit test

255af58

Fixing a race between migration_support and AGAS, addressing various …

249a64a

…comments.

Remove id from AGAS table once migrated back to a locality where it w…

dd9c83e

…as located earlier

Ignore the AGAS lock while acquiring the component migration lock

502178c

Don't store migrating objects in AGAS cache

8ff8ec1

- flyby: write error message for failing tests

Fixed problem in AGAS migration table, migration seems to be fully fu…

6aad7d0

…nctional now - added more asserts to migration code - diagnostic printouts in migrate_component test

Adding migrate() function taking a distribution policy

5671622

- adding serialization to distribution policies

hkaiser force-pushed the migrate_component branch from 1e9f04d to 5671622 Compare January 31, 2016 02:42

hkaiser mentioned this pull request Jan 31, 2016

version 1.0? #1896

Closed

hkaiser added 4 commits January 31, 2016 19:01

Unregister all gids from component store at destruction

3a2ba58

- flyby: fixed un-registration of migrated simple components

Merge branch 'master' into migrate_component

df937d4

Silencing the inspect tool

da18bbd

Fixed a possible held lock while suspending

f2c928f

hkaiser mentioned this pull request Feb 3, 2016

Refactor action traits #1977

Merged

Making sure dont_store_in_cache for gids flag is always respected

1a29ae2

hkaiser added 3 commits February 4, 2016 13:35

Disambiuate calls to make_exceptional_future

031b935

Improve error handling

97627e3

Fixing migration of components.

6d46d68

hkaiser mentioned this pull request Feb 8, 2016

assertion 'p.destination_locality() == hpx::get_locality()' failed #1944

Closed

hkaiser added 3 commits February 8, 2016 10:45

Adding more diagnostics for issue reported in #1944

c7e53a7

Fixing more minor issues with caching, improving diagnostics

ecbdbae

Reducing this branch to touch up on base-migration only

87ad0ff

- this removes the changes to migrate_[to|from]_storage

hkaiser force-pushed the migrate_component branch from 1c1135a to 87ad0ff Compare February 8, 2016 21:06

hkaiser added a commit that referenced this pull request Feb 9, 2016

Merge pull request #1966 from STEllAR-GROUP/migrate_component

0a2c647

Migrate components

hkaiser merged commit 0a2c647 into master Feb 9, 2016

hkaiser deleted the migrate_component branch February 9, 2016 12:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate components #1966

Migrate components #1966

hkaiser commented Jan 23, 2016

sithhell Jan 23, 2016

hkaiser Jan 23, 2016

sithhell commented Jan 23, 2016

hkaiser commented Jan 31, 2016

hkaiser commented Feb 3, 2016

sithhell commented Feb 3, 2016

hkaiser commented Feb 3, 2016

sithhell commented Feb 3, 2016

sithhell commented Feb 3, 2016

hkaiser commented Feb 3, 2016

hkaiser commented Feb 3, 2016

sithhell commented Feb 4, 2016

sithhell commented Feb 8, 2016

hkaiser commented Feb 8, 2016

Migrate components #1966

Migrate components #1966

Conversation

hkaiser commented Jan 23, 2016

sithhell Jan 23, 2016

Choose a reason for hiding this comment

hkaiser Jan 23, 2016

Choose a reason for hiding this comment

sithhell commented Jan 23, 2016

hkaiser commented Jan 31, 2016

hkaiser commented Feb 3, 2016

sithhell commented Feb 3, 2016

hkaiser commented Feb 3, 2016

sithhell commented Feb 3, 2016

sithhell commented Feb 3, 2016

hkaiser commented Feb 3, 2016

hkaiser commented Feb 3, 2016

sithhell commented Feb 4, 2016

sithhell commented Feb 8, 2016

hkaiser commented Feb 8, 2016