ACCUMULO-4718 Add CLASSPATH to accumulo-testing script#6
ACCUMULO-4718 Add CLASSPATH to accumulo-testing script#6milleruntime merged 2 commits intoapache:masterfrom
Conversation
milleruntime
force-pushed
the
ACCUMULO-4718
branch
from
383eb32 to
cb53aff
Compare
bin/accumulo-testing
Outdated
| fi | ||
| build_shade_jar | ||
| java -Dlog4j.configuration="file:$log4j_config" -cp "$at_shaded_jar" "$randomwalk_main" "$at_props" "$2" | ||
| java -Dlog4j.configuration="file:$log4j_config" -cp "$at_shaded_jar:$HADOOP_CONF_DIR" "$randomwalk_main" "$at_props" "$2" |
There was a problem hiding this comment.
One way we can decouple this from specific environment variables (which may or may not exist), is to simply remove the -cp section, and set CLASSPATH above the call to java.
Something like export CLASSPATH="$at_shaded_jar:$CLASSPATH"
That way, the caller can simply set CLASSPATH=/path/to/hadoop/confdir prior to running the accumulo-testing command.
We could also add a sanity check prior to running java to make sure that the $CLASSPATH contains a directory with core-site.xml and/or hdfs-site.xml (or whatever the Hadoop client configuration uses by default).
mikewalch
left a comment
mikewalch
left a comment
There was a problem hiding this comment.
Nice bug fix. This should fix should be made when the random walk tests are run in YARN (using rw-yarn command). I will create a ticket for that if you want to work on it.
|
Sounds good. Thanks |
|
@mikewalch As far as I can tell, I believe my changes in 52baec5 will also satisfy ACCUMULO-4723 It seemed to run fine with the quick test I ran using: accumulo-testing rw-yarn 1 Bulk.xml |
milleruntime
changed the title
…astructure Update AWS shared_state to gen S3 bucket
3 participants
The test "rw-local Bulk.xml" was throwing errors because it couldn't find the HDFS containing the files to import. The hadoop conf wasn't on the classpath so it was defaulting to the disk "file:///" file system. This doesn't seem to affect the rest of the tests. Not sure if this is the best place to include HADOOP_CONF_DIR...