Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch #14273

Closed
wants to merge 7 commits into from

Conversation

MechCoder
Copy link
Contributor

What changes were proposed in this pull request?

Builds upon the work done by @hhbyyh in #7871 . This replaces all occurrences of TimeTracker with the more useful MultiStopWatch. More useful because it is possible to bench the total time across the worker nodes as well, for instance in the method binsToBestSplit using the DistributedStopwatch. It is also very useful to measure the optimizations in terms of time done in #13959 and should be merged before that gets reviewed. It also removes the TimeTracker since it is not being used elsewhere except the tree module.

How was this patch tested?

It was run using setLogLevel("INFO") and the following timings are printed out.

16/07/19 16:45:18 INFO RandomForest: {
binsToBestSplit: 26ms,
chooseSplits: 301ms,
findBestSplits: 307ms,
findSplitsBins: 553ms,
init: 1229ms,
total: 1572ms
}

@MechCoder
Copy link
Contributor Author

ping @jkbradley @mengxr

@SparkQA
Copy link

SparkQA commented Jul 20, 2016

Test build #62566 has finished for PR 14273 at commit e5b077d.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jul 20, 2016

Test build #62620 has finished for PR 14273 at commit abff51b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MechCoder
Copy link
Contributor Author

@jkbradley Would you be able to have a look?

@MechCoder
Copy link
Contributor Author

bump?

@jkbradley
Copy link
Member

Sorry about the delay here. Do you still have time to work on this?

@sethah
Copy link
Contributor

sethah commented Feb 28, 2017

@jkbradley I do not think @MechCoder is working on Spark for the time being.

@jkbradley
Copy link
Member

OK apologies @MechCoder for the delay. I guess we can close this issue, and someone else can open up a PR based on yours.

@MechCoder MechCoder closed this Feb 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants