Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add suspendables to throttler in consistent state #640

Conversation

matthias-wende-sociomantic
Copy link
Contributor

@matthias-wende-sociomantic matthias-wende-sociomantic commented Oct 8, 2018

This change ensures that the suspend state of an ISuspendable instance
when added to the throttler is consistent with the throttler state.

This patch somehow also works around the the following problem.

If a RequestSuspender instance is reused by a new Request it'll set the suspend_requested flag to false.

https://github.com/sociomantic-tsunami/dhtproto/blob/v14.x.x/src/dhtproto/client/legacy/internal/request/model/IBulkGetRequest.d#L133
https://github.com/sociomantic-tsunami/swarm/blob/v5.x.x/src/swarm/client/request/helper/RequestSuspender.d#L148

When adding this instance to the throttlers list of suspendables it will be already present and therefore neither appended to the list of suspendables nor suspended.

https://github.com/sociomantic-tsunami/ocean/blob/v4.x.x/src/ocean/io/model/ISuspendableThrottler.d#L77

Now if at the same time the application (i.e. the applications throttler) is in suspended state, the request will be started in a non suspended state and will not get suspended at least until the throttler resumes and suspends again.

https://github.com/sociomantic-tsunami/ocean/blob/v4.x.x/src/ocean/io/model/ISuspendableThrottler.d#L139

Note that that doesn't address the issue that a suspendable is kept in the list of suspendables when a request finishes (compere also the comment below). The possibility for clients to unregister suspendables from the Throttller when a request finishes will be addressed later.

@matthias-wende-sociomantic matthias-wende-sociomantic changed the title Suspend not suspended suspendables when added to throttler Suspend all suspendables when added to a suspended throttler Oct 8, 2018

if (this.suspended_ && !s.suspended())
{
s.suspend();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be suspended if it isn't added to the suspendables? Anyway if there is any reason to do so then the documentation of the method should be updated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's the whole point. Recycled instances are in the list and they should be suspended.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And good point. I updated the doc accordingly ;).

@codecov
Copy link

codecov bot commented Oct 8, 2018

Codecov Report

Merging #640 into v4.2.x will increase coverage by 0.09%.
The diff coverage is 90%.

@daniel-zullo
Copy link

If this is a bug fix then it would be good to have/add a unit test

@matthias-wende-sociomantic
Copy link
Contributor Author

matthias-wende-sociomantic commented Oct 8, 2018

If this is a bug fix then it would be good to have/add a unit test

Yeah, thanks for pointing that out. I've added one for this particular case.

@gavin-norman-sociomantic

Hmm wasn't the idea originally to add a method allowing the user to remove a suspendable from the throttler?

@matthias-wende-sociomantic
Copy link
Contributor Author

matthias-wende-sociomantic commented Oct 8, 2018

Yes, that was the old plan before this new one :).
I favour this solution for several reasons.

  • It fixes the bug as well.
  • It is much simpler (i.e. no changes in swarm and dhtproto).
  • It fixes the bug without changing clients.

At the same time the original idea introduced some new regression to the client and I have no intension to track this down if this patch works later.

@gavin-norman-sociomantic

You describe this situation:

  1. A suspendable is registered with the throttler.
  2. The suspendable ceases to be valid (e.g. the request it represented finished) and is recycled into a pool of suspendables. It is not unregistered from the throttler.
  3. The suspendable is acquired from the pool and reinitialised.
  4. The suspendable is registered with the throttler. As it is already registered, some special behaviour is required.

What I'm concerned about is this: what happens if, after step 2, the throttler tries to suspend? It would be calling a method of the recycled suspendable, which would do...? The fact that invalid suspendables can (must, currently) stay registered with the throttler seems like a bug to me, not something that we should patch up to make work.

What do you think?

@gavin-norman-sociomantic

Note that a recycled invalid suspendable is only one case. What if the invalid suspendable has been garbage collected, say?

@matthias-wende-sociomantic matthias-wende-sociomantic changed the title Suspend all suspendables when added to a suspended throttler Add suspendables to throttler in consistent state Oct 8, 2018
@matthias-wende-sociomantic
Copy link
Contributor Author

Yes that's a valid point a should be addressed as well. My description of this patch was somehow misleading. I've updated the PR and commit descriptions to what this patch is actually about.

suspendable_throttler.suspendAll();

// add a resumed ISuspendable to a suspended throttler
// note that the instance is "recycled"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one comment probably isn't needed any more.

Copy link

@gavin-norman-sociomantic gavin-norman-sociomantic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, the new explanation makes much more sense, outside of the context of the bug you were investigating.

suspendable.suspend();
suspendable_throttler.addSuspendable(suspendable);

test!("==")(suspendable.suspended(), false, "Unit test failed, " ~

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, I don't think you need to write "Unit test failed". It's clear from the context.

suspendable.resume();
suspendable_throttler.addSuspendable(suspendable);

test!("==")(suspendable.suspended(), true, "Unit test failed, " ~

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, I don't think you need to write "Unit test failed". It's clear from the context.

@gavin-norman-sociomantic

It seems like this should be a patch release.

This change ensures that the suspend state of an ISuspendable instance
when added to the throttler is consistent with the throttler state.
This test checks if an instance of ISuspendable that
is added to the throttler has the same suspend state as
the throttler.
@matthias-wende-sociomantic
Copy link
Contributor Author

Updated, base branch changed + rebased on top of new branch, accidentally deleted branch, recovered the branch and finally reopened the PR :).

@gavin-norman-sociomantic gavin-norman-sociomantic merged commit b10efc2 into sociomantic-tsunami:v4.2.x Oct 9, 2018
else if (!this.suspended_ && s.suspended())
{
s.resume();
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why make it simple if we could make it complicated?

        if (this.suspended_ != s.suspended())
        {
            if (this.suspended_)
                s.suspend();
            else
                s.resume();
        }

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR has been merged, but that's a good point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that's looks indeed better.

}
}

version (UnitTest){

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the format/style is wrong


public bool suspended ( )
{
return suspended_;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants