add option to pass any additional classpath while launching a topology #1245
Conversation
|
👍 Thanks for this contribution |
| @@ -54,7 +54,7 @@ def heron_class(class_name, lib_jars, extra_jars=None, args=None, java_defines=N | |||
| # the java opts must be passed as part of the list | |||
| all_args = [config.get_java_path(), "-client", "-Xmx1g"] + \ | |||
| java_opts + \ | |||
| ["-cp", config.get_classpath(lib_jars + extra_jars)] | |||
| ["-cp", config.get_classpath(extra_jars + lib_jars)] | |||
There was a problem hiding this comment.
em.. why switching order here?
There was a problem hiding this comment.
This is because the user defined class path should come earlier than the system or heron class path.
cheers
/karthik
On Aug 15, 2016, at 9:36 AM, Mark Li notifications@github.com wrote:
In heron/cli/src/python/execute.py #1245 (comment):
@@ -54,7 +54,7 @@ def heron_class(class_name, lib_jars, extra_jars=None, args=None, java_defines=N
the java opts must be passed as part of the list
all_args = [config.get_java_path(), "-client", "-Xmx1g"] +
java_opts + \
["-cp", config.get_classpath(lib_jars + extra_jars)]["-cp", config.get_classpath(extra_jars + lib_jars)]—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/twitter/heron/pull/1245/files/88942cede245a9d6f1442e04388dc3189772eebc#r74790579, or mute the thread https://github.com/notifications/unsubscribe-auth/AAWcRNhbHFyTSRI-xhUmTgJWpuh1-4Uaks5qgJWcgaJpZM4Jh24I.
|
👍 |
|
|
||
| # check if the class path file exists | ||
| Log.debug('Checking class path file: %s', path) | ||
| if os.path.isfile(path): |
There was a problem hiding this comment.
The classpath entry can be a directory also. For e.g. dir of hadoop config xml files. This check fails for a directory.
|
👍 LGTM |
#1245) * add option to pass any additional classpath while launching a topology - needed for YARN, extra jars comes ahead of the lib jars, validate classpath for existence and include checking of directory for classpath
Needed for schedulers for YARN - might be useful later for others as well.