New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-13358][SQL] Retrieve grep path when do benchmark #11231
Conversation
@@ -93,7 +93,8 @@ private[spark] object Benchmark { | |||
if (SystemUtils.IS_OS_MAC_OSX) { | |||
Utils.executeAndGetOutput(Seq("/usr/sbin/sysctl", "-n", "machdep.cpu.brand_string")) | |||
} else if (SystemUtils.IS_OS_LINUX) { | |||
Utils.executeAndGetOutput(Seq("/usr/bin/grep", "-m", "1", "\"model name\"", "/proc/cpuinfo")) | |||
val grepPath = Utils.executeAndGetOutput(Seq("which", "grep")) | |||
Utils.executeAndGetOutput(Seq(grepPath, "-m", "1", "model name", "/proc/cpuinfo")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
""model name"" doesn't work on my linux. "model name" works. Don't know if it is special case or general.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here.
retest this please. |
Test build #51406 has finished for PR 11231 at commit
|
@@ -93,7 +93,8 @@ private[spark] object Benchmark { | |||
if (SystemUtils.IS_OS_MAC_OSX) { | |||
Utils.executeAndGetOutput(Seq("/usr/sbin/sysctl", "-n", "machdep.cpu.brand_string")) | |||
} else if (SystemUtils.IS_OS_LINUX) { | |||
Utils.executeAndGetOutput(Seq("/usr/bin/grep", "-m", "1", "\"model name\"", "/proc/cpuinfo")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or just make it "grep" without specified path?
I guess there isn't a way to test this... Lemme try this locally. |
This works on my linux machine (Ubuntu 15.10). |
Test build #51466 has finished for PR 11231 at commit
|
ping @davies do you think this is useful? Thanks. |
LGTM, merging into master, thanks! |
JIRA: https://issues.apache.org/jira/browse/SPARK-13358
When trying to run a benchmark, I found that on my Ubuntu linux grep is not in /usr/bin/ but /bin/. So wondering if it is better to use which to retrieve grep path.
cc @davies