Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FLINK-19552][coordination] Consider only available input location preferences for slot profile in pipelined region scheduling #13730

Closed
wants to merge 5 commits into from
Closed

[FLINK-19552][coordination] Consider only available input location preferences for slot profile in pipelined region scheduling #13730

wants to merge 5 commits into from

Commits on Oct 27, 2020

  1. [hotfix][tests] Let SlotSharingExecutionSlotAllocatorTest extend Test… 

    …Logger
    @zhuzhurk @azagrebin
    zhuzhurk authored and azagrebin committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    0df023a View commit details
    Browse the repository at this point in the history

  2. [hotifx][runtime] Print root cause of logical slot allocation failures

    @zhuzhurk @azagrebin
    zhuzhurk authored and azagrebin committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    3532e77 View commit details
    Browse the repository at this point in the history

  3. [hotfix] Remove unused OneSlotPerExecutionSlotAllocator

    @azagrebin
    azagrebin committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    3672b50 View commit details
    Browse the repository at this point in the history

  4. [FLINK_19552][coordination] Create PreferredLocationsRetriever in Exe… 

    …cutionSlotAllocatorFactory's
    @azagrebin
    azagrebin committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    1ef876e View commit details
    Browse the repository at this point in the history

  5. [FLINK-19552][coordination] Consider only available input location pr… 

    …eferences for slot profile in pipelined region scheduling

The pipelined region scheduling strategy schedules regions once all their input blocking dependencies are ready.
The SlotSharingGroups of the region can include executions of other regions which are not scheduled yet including their dependencies.
Hence we should not wait other unavailable input dependencies to unblock the current region scheduling.

The new SlotSharingExecutionSlotAllocator creates the DefaultSyncPreferredLocationsRetriever
where the original InputsLocationsRetriever is wrapped with the AvailableInputsLocationsRetriever.
It makes the InputsLocationsRetriever return only completed input location futures, others are filtered out.
This allows to return completed future from DefaultSyncPreferredLocationsRetriever and make it synchronous and non-blocking.

This closes apache#13730.
    @azagrebin
    azagrebin committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    c313c12 View commit details
    Browse the repository at this point in the history