-
Notifications
You must be signed in to change notification settings - Fork 28.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch #14273
Conversation
ping @jkbradley @mengxr |
Test build #62566 has finished for PR 14273 at commit
|
Test build #62620 has finished for PR 14273 at commit
|
@jkbradley Would you be able to have a look? |
bump? |
Sorry about the delay here. Do you still have time to work on this? |
@jkbradley I do not think @MechCoder is working on Spark for the time being. |
OK apologies @MechCoder for the delay. I guess we can close this issue, and someone else can open up a PR based on yours. |
What changes were proposed in this pull request?
Builds upon the work done by @hhbyyh in #7871 . This replaces all occurrences of TimeTracker with the more useful MultiStopWatch. More useful because it is possible to bench the total time across the worker nodes as well, for instance in the method
binsToBestSplit
using theDistributedStopwatch
. It is also very useful to measure the optimizations in terms of time done in #13959 and should be merged before that gets reviewed. It also removes theTimeTracker
since it is not being used elsewhere except the tree module.How was this patch tested?
It was run using
setLogLevel("INFO")
and the following timings are printed out.16/07/19 16:45:18 INFO RandomForest: {
binsToBestSplit: 26ms,
chooseSplits: 301ms,
findBestSplits: 307ms,
findSplitsBins: 553ms,
init: 1229ms,
total: 1572ms
}