You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've set 'druid.javascript.enabled' to true in common.runtime.properties, however in the JavaScriptConfig in the mapred job on the EMR cluster "enabled" is "false".
I verified it isn't a config problem as javascript is enablewhen I run the task using the local mapred executor in the middle managers it works.
I think this has something to do with config files not being pushed to the Hadoop mapred tasks, and my reasoning is that the peon says:
2017-04-17T21:14:12,232 INFO [main] io.druid.cli.CliPeon - * druid.javascript.enabled: true
In the druid log, but the map tasks have:
2017-04-17T21:14:55,224 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.js.JavaScriptConfig] from props[druid.javascript.] as [JavaScriptConfig{enabled=false}]
I'm using 0.10.0-rc2 with EMR 2.7.3
Has anyone encountered this? Does anyone know how to work around this?
Thanks!
Gal
The text was updated successfully, but these errors were encountered:
In the Google Group @gianm correctly said I can add
"jobProperties" : {
"mapreduce.map.java.opts" : "-server -Xms512m -Xmx512m -Ddruid.javascript.enabled=true -Duser.timezone=UTC -Dfile.encoding=UTF-8",
"mapreduce.reduce.java.opts" : "-server -Xms2g -Xmx2g -Ddruid.javascript.enabled=true -Duser.timezone=UTC -Dfile.encoding=UTF-8"
}
to the jobProperties to set the system properties on the job, making it enable javascript.
Its probably worth either documenting this or adding code to have the Indexing Service pass this down to the Hadoop job.
hochbergg
changed the title
runtime.properties not propagated to indexing job on external Hadoop
runtime.properties not propagated to indexing job on external Hadoop leading to Javascript not being enabled in indexing task
Apr 18, 2017
I've set 'druid.javascript.enabled' to true in common.runtime.properties, however in the JavaScriptConfig in the mapred job on the EMR cluster "enabled" is "false".
I verified it isn't a config problem as javascript is enablewhen I run the task using the local mapred executor in the middle managers it works.
I think this has something to do with config files not being pushed to the Hadoop mapred tasks, and my reasoning is that the peon says:
2017-04-17T21:14:12,232 INFO [main] io.druid.cli.CliPeon - * druid.javascript.enabled: true
In the druid log, but the map tasks have:
2017-04-17T21:14:55,224 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.js.JavaScriptConfig] from props[druid.javascript.] as [JavaScriptConfig{enabled=false}]
I'm using 0.10.0-rc2 with EMR 2.7.3
Has anyone encountered this? Does anyone know how to work around this?
Thanks!
Gal
The text was updated successfully, but these errors were encountered: