Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-8266] [runtime] add network memory to ResourceProfile for the input and output memory of a task #5170

Closed
wants to merge 2 commits into from

Conversation

shuai-xu
Copy link
Contributor

What is the purpose of the change

This pull request adds a network memory field to ResourceProfile. So job master can set the network memory of a task according to the number of input channels and output sub partitions.

Brief change log

(for example:)

  • The TaskInfo is stored in the blob store on job creation time as a persistent artifact
  • Deployments RPC transmits only the blob storage reference
  • TaskManagers retrieve the TaskInfo from the blob cache

Verifying this change

This change can be verified by running ResourceProfileTest:

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicabled)

@tillrohrmann
Copy link
Contributor

Thanks for the contribution @shuai-xu. There are some merge conflicts. Could you please resolve them?

@shuai-xu
Copy link
Contributor Author

@tillrohrmann sorry, the conflict is resolved now

Copy link
Contributor

@tillrohrmann tillrohrmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution @shuai-xu. Changes look good to me. Merging this PR.

tillrohrmann pushed a commit to tillrohrmann/flink that referenced this pull request Jan 25, 2018
tillrohrmann pushed a commit to tillrohrmann/flink that referenced this pull request Jan 25, 2018
@asfgit asfgit closed this in 47e6069 Jan 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants