[HIVEMALL-30] Increase maximum Java heap size and decrease MaxPermGen to avoid OutOfMemoryError

[wangyum]

What changes were proposed in this pull request?

Increase maximum Java heap size and decrease MaxPermGen to avoid OutOfMemoryError.

How was this patch tested?

Manual tests

[coveralls]

Coverage remained the same at 35.842% when pulling 1d18c7d on wangyum:HIVEMALL-30 into ed16ca0 on apache:master.

[myui]

@maropu Could you take a look at?

[maropu]

TravisCI always pass these tests though, why did you got the error? What's your environment?

[maropu]

Oh, I got you. So, how about setting these values along with the spark one (e.g., -mx3g)? See: https://github.com/apache/spark/blob/master/pom.xml#L2066

[maropu]

okay, LGTM cc: @myui

[myui]

@maropu BTW, why mvn -q scalastyle:check test -Pspark-2.0" exited with 1 is happening?

[maropu]

@myui In running the test, a spark context and unit tests work in the same JVM. The context uses at most 1g memory by default. So, in some conditions, the unit tests run short of memory, I think. If we set 3g at the value, the context uses 1g and the tests do 2g. That sounds fine to me.

[coveralls]

Coverage remained the same at 35.842% when pulling 37da226 on wangyum:HIVEMALL-30 into ed16ca0 on apache:master.

[myui]

@maropu It seems full GC is still happening for some cases...

HivemallFeatureOpsSuite:
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

MaxPermGen=1024m might be too big and it might need more space for other stuffs. Do we need -MaxPermGen ? Less parameter is better for configuring JVM. We can use just 4GB in TravisCi while there are 3 tasks (openjdk7, oraclejdk8, oraclejdk8) in each job.

[maropu]

@myui How about 512m? Sorry, but I forgot why I set that value there though, I checked the spark configure and I found that value is 512m.

[wangyum]

I'll try to set 512m.

[coveralls]

Coverage remained the same at 35.842% when pulling 20c2eea on wangyum:HIVEMALL-30 into ed16ca0 on apache:master.

[coveralls]

Coverage remained the same at 35.842% when pulling 20c2eea on wangyum:HIVEMALL-30 into ed16ca0 on apache:master.

[myui]

@wangyum @maropu Thanks. Merged with some modifications. Configuration for spark-1.6 should also be changed.

[maropu]

okay, thanks!