Fixing action move tests

[hkaiser]

This fixes the action move test count problems introduced recently

[sithhell]

Future serialization is really getting out of hand :/ Those changes also mean a breaking change for user code, correct?
I would by far prefer to just wait in the future serialization code until it is ready instead of relying on those traits which are neither documented and also look very cumbersome to use.

[sithhell]

After going through the requested changes, the only difference between waiting directly in the future serialization code and the currently mechanism in place is the creation of a new thread. This is an important difference and not having such a separate 'waiting' thread might lead to deadlocks.

A proper solution could be to have various serialization passes with special archive types. The one checking for non-ready futures could have a shared state to which the non ready futures register themselves. Instead of a buffer, the result of this serialization would a future that marks the completion of all futures which are about to be serialized. This avoids the need for a special trait completely.

The same mechanism can be employed to calculate the exact size needed in a buffer to serialize a parcel at hand, again, without the need for a special trait.

[sithhell]

With all that said above, this doesn't affect this pull request as it fixes a problem we have right now. So I think it is good to be merged.

[hkaiser]

Future serialization is really getting out of hand :/

I agree.

Those changes also mean a breaking change for user code, correct?

No changes in user code are required.

I would by far prefer to just wait in the future serialization code until it is ready instead of relying on those traits which are neither documented and also look very cumbersome to use.

I'd prefer that as well, although I seem to remember that we agreed its not possible because of the limited amount of network connection we have available (are those still acquired before serialization?). If it is possible now, I'd be happy to remove all of that serialize_as_future trait.

[hkaiser]

After going through the requested changes, the only difference between waiting directly in the future serialization code and the currently mechanism in place is the creation of a new thread. This is an important difference and not having such a separate 'waiting' thread might lead to deadlocks.

I'm not sure to understand. Could you elaborate, please?

A proper solution could be to have various serialization passes with special archive types. The one checking for non-ready futures could have a shared state to which the non ready futures register themselves. Instead of a buffer, the result of this serialization would a future that marks the completion of all futures which are about to be serialized. This avoids the need for a special trait completely.

This would mean introducing a lot of overhead in most situations to be able to resolve single cases.

[sithhell]

On 04/22/2015 02:19 PM, Hartmut Kaiser wrote:

I would by far prefer to just wait in the future serialization code until it is ready instead of relying on those traits which are neither documented and also look very cumbersome to use.

I'd prefer that as well, although I seem to remember that we agreed its not possible because of the limited amount of network connection we have available (are those still acquired before serialization?). If it is possible now, I'd be happy to remove all of that serialize_as_future trait.

Should be possible now, but probably needs to be tested extensively
before merging that change.

[sithhell]

After going through the requested changes, the only difference between waiting directly in the future serialization code and the currently mechanism in place is the creation of a new thread. This is an important difference and not having such a separate 'waiting' thread might lead to deadlocks.

I'm not sure to understand. Could you elaborate, please?

possible deadlock in the parcelport due to a limited number of
connections, but that should be handled properly now.

A proper solution could be to have various serialization passes with special archive types. The one checking for non-ready futures could have a shared state to which the non ready futures register themselves. Instead of a buffer, the result of this serialization would a future that marks the completion of all futures which are about to be serialized. This avoids the need for a special trait completely.

This would mean introducing a lot of overhead in most situations to be able to resolve single cases.

We already have a overhead for that single case. But I certainly agree
that it is heavier than what we have right now. But it would make
everything correct, not just the cases where the trait is defined correctly.

[hkaiser]

We already have a overhead for that single case. But I certainly agree
that it is heavier than what we have right now. But it would make
everything correct, not just the cases where the trait is defined correctly.

This scheme would work for vanilla serialization function in user code only (i.e. only for things like ar & val1 & val2 & ...;). Anything beyond this would break or create un-necessary overhead. For things like zero-copy serialization this would still require some kind of mechanism to handle things correctly, though (such like an additional function besides serialize(). which would be used by the special archives).

[hkaiser]

possible deadlock in the parcelport due to a limited number of
connections, but that should be handled properly now.

Is the keyword 'should', or are we sure of that?

[sithhell]

possible deadlock in the parcelport due to a limited number of
connections, but that should be handled properly now.

Is the keyword 'should', or are we sure of that?

We didn't have a single deadlock in distributed for quite some time now due
to the serialization suspending the running thread (caused by credit
splitting), so I'm fairly certain that we got rid of that issue.

[hkaiser]

We didn't have a single deadlock in distributed for quite some time now due
to the serialization suspending the running thread (caused by credit
splitting), so I'm fairly certain that we got rid of that issue.

Does that mean we could get rid of the whole serialize_as_future business? Let's create a ticket for that.