Enable spark logging on executors

[fbertsch]

Previously, executor logs would fail with:
java.io.FileNotFoundException: /mnt/var/log/spark/spark.log (Permission denied)

Which caused the executor logs to not write.

This change will make them available, and then available in s3.

[acmiyaguchi]

This will close bug 1329779, which is related to bug 1335228.

[jezdez]

@acmiyaguchi Can we close #61 then?

[acmiyaguchi]

This looks good, I compared the test cluster logs to a random cluster in the EMR list, and it looks like logs are showing up on executors now. Looks like it's working as expected.

@jezdez It looks like I called closing those bugs too early, this solves a different problem. In the linked cluster, the spark-history-server.out.gz has the same error message:

log4j:ERROR setFile(null,true) call failed. java.io.FileNotFoundException: /mnt/var/log/spark/spark.log (Permission denied)

[fbertsch]

I'm about 99.99% sure that if we remove the "log4j.appender.FILE.File": "/mnt/var/log/spark/spark.log", line from configuration.json then the history server logs will be available. The downside to this is the spark logs won't be available in spark.log -- I'm not actually sure where they are output by default (it could just be within the spark container logs, which isn't actually an issue, since those are also saved to s3).