[FLINK-8089] Also check for other pending slot requests in SlotPool#offerSlot #5090
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tillrohrmann
force-pushed
the
fixSlotOffering
branch
from
b04dda4
to
32269bc
Compare
tillrohrmann
added a commit
to tillrohrmann/flink
that referenced
this pull request
Dec 1, 2017
Not only check for a slot request with the right allocation id but also check whether we can fulfill other pending slot requests with an unclaimed offered slot before adding it to the list of available slots. This closes apache#5090.
tillrohrmann
force-pushed
the
fixSlotOffering
branch
from
32269bc
to
b04dda4
Compare
tillrohrmann
added a commit
to tillrohrmann/flink
that referenced
this pull request
Dec 1, 2017
Not only check for a slot request with the right allocation id but also check whether we can fulfill other pending slot requests with an unclaimed offered slot before adding it to the list of available slots. This closes apache#5090.
tillrohrmann
force-pushed
the
fixSlotOffering
branch
from
b04dda4
to
c39ac05
Compare
tillrohrmann
added a commit
to tillrohrmann/flink
that referenced
this pull request
Dec 3, 2017
Not only check for a slot request with the right allocation id but also check whether we can fulfill other pending slot requests with an unclaimed offered slot before adding it to the list of available slots. This closes apache#5090.
GJL
reviewed
Dec 7, 2017
| @@ -383,6 +386,76 @@ public void testSlotRequestCancellationUponFailingRequest() throws Exception { | |||
| } | |||
| } | |||
|
|
|||
| /** | |||
| * Tests that unused offered slots are directly used to fulfil pending slot | |||
There was a problem hiding this comment.
nit: Inconsistent British and American English
fulfill instead of fulfil
There was a problem hiding this comment.
True, will use the american version
GJL
reviewed
Dec 7, 2017
|
|
||
| assertTrue(slotPoolGateway.offerSlot(taskManagerLocation, taskManagerGateway, slotOffer).get()); | ||
|
|
||
| // the slot offer should fulfil the second slot request |
GJL
approved these changes
Dec 7, 2017
tillrohrmann
force-pushed
the
fixSlotOffering
branch
from
c39ac05
to
257d928
Compare
tillrohrmann
added a commit
to tillrohrmann/flink
that referenced
this pull request
Dec 14, 2017
Not only check for a slot request with the right allocation id but also check whether we can fulfill other pending slot requests with an unclaimed offered slot before adding it to the list of available slots. This closes apache#5090.
The cluster entrypoints start the ResourceManager with the web interface URL. This URL is used to set the correct tracking URL in Yarn when registering the Yarn application. This closes apache#5128.
This commit introduces the SlotContext which is an abstraction for the SimpleSlot to obtain the relevant slot information to do the communication with the TaskManager without relying on the AllocatedSlot which is now only used by the SlotPool. This closes apache#5088.
Before logical slots like the SimpleSlot and SharedSlot where associated to the actually allocated slot via the AllocationID. This, however, was sub-optimal because allocated slots can be re-used to fulfill also other slot requests (logical slots). Therefore, we should bind the logical slots to the right id with the right lifecycle which is the slot request id. This closes apache#5089.
Not only check for a slot request with the right allocation id but also check whether we can fulfill other pending slot requests with an unclaimed offered slot before adding it to the list of available slots. This closes apache#5090.
tillrohrmann
force-pushed
the
fixSlotOffering
branch
from
257d928
to
331ce82
Compare
|
Thanks for the review @GJL. Merging this PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
Not only check for a slot request with the right allocation id but also check
whether we can fulfill other pending slot requests with an unclaimed offered
slot before adding it to the list of available slots in
SlotPool.This PR is based on #5089.
Verifying this change
SlotPoolTest#testFulfillingSlotRequestsWithUnusedOfferedSlotsto check that unused offered slots are directly used to fulfill other pending slot requestsDoes this pull request potentially affect one of the following parts:
@Public(Evolving): (no)Documentation
CC: @GJL