Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-12885] [MINOR] Rename 3 fields in ShuffleWriteMetrics #10811

Closed
wants to merge 4 commits into from

Conversation

andrewor14
Copy link
Contributor

This is a small step in implementing SPARK-10620, which migrates TaskMetrics to accumulators. This patch is strictly a cleanup patch and introduces no change in functionality. It literally just renames 3 fields for consistency. Today we have:

inputMetrics.recordsRead
outputMetrics.bytesWritten
shuffleReadMetrics.localBlocksFetched
...
shuffleWriteMetrics.shuffleRecordsWritten
shuffleWriteMetrics.shuffleBytesWritten
shuffleWriteMetrics.shuffleWriteTime

The shuffle write ones are kind of redundant. We can drop the shuffle part in the method names. I added backward compatible (but deprecated) methods with the old names.

Parent PR: #10717

@SparkQA
Copy link

SparkQA commented Jan 18, 2016

Test build #49608 has finished for PR 10811 at commit 0a2d9a1.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor Author

retest this please

@andrewor14 andrewor14 changed the title [SPARK-10620] [MINOR] Rename 3 fields in ShuffleWriteMetrics [SPARK-12885] [MINOR] Rename 3 fields in ShuffleWriteMetrics Jan 18, 2016
Conflicts:
	core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala
@andrewor14
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented Jan 18, 2016

Test build #49616 has finished for PR 10811 at commit 0a2d9a1.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jan 19, 2016

Test build #49628 has finished for PR 10811 at commit 823156b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor

LGTM, so I'm going to merge this now. Thanks!

@asfgit asfgit closed this in 2b5d11f Jan 19, 2016
@andrewor14 andrewor14 deleted the rename-things branch January 19, 2016 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants