Modify docker module errors and deploy docker image on openjdk

[zybing]

Modify docker module errors

1.move #!bin/bash to the first line for all shell script files.

2.add comment for the license information of xml files.

 e.g.<!--# Licensed to the Apache Software Foundation (ASF) under one or more....-->

without the <!--..-->comment, the xml file will not be parsed correctly.

3.modify the scripts/gen_base_dockerfile.sh error

in line 58 ,the sed statment doesn't work because of syntax error

4.when maven compiles storm benchmark ,the error of lacking of thrift be throwed.

To resolve the problem, add RUN apt-get update && apt-get install -y thrift-compiler to the base/base-core file

5.modify cdh-docker/Dockerfile file

the last line in cdh-docker/Dockerfile is CMD bash -C '/usr/bin/restart-hadoop-spark.sh'; 'bash' . But it is not a good solution to run docker container in Dockerfile. I delete this line and write a new runexample.sh to replace it.

6.add cdh-docker/scripts/runexample.sh file

using this file to run docker container,the content as follow:
#modify etc/hosts file to avoid connection error
cp /etc/hosts /etc/hostsbak
sed -i 's/::1/#::1/' /etc/hostsbak
cat /etc/hostsbak > /etc/hosts
rm -rf /etc/hostsbak
#run wordcount example
/usr/bin/restart-hadoop-spark.sh
${HIBENCH_HOME}/workloads/wordcount/prepare/prepare.sh
${HIBENCH_HOME}/workloads/wordcount/mapreduce/bin/run.sh

7.modify script/run_container.sh file

modify the last line. modify the run docker container method

8.modify opensource-docker module

it is similar with cdh-docker module

9.change the default maven version to 3.0.5 in hibench-docker.conf file

Deploy docker image on openjdk

1.modify base/base-core file. change the oracle-jdk installation to openjdk installation. code as follow:

# Install Java
RUN \
apt-get update && \
apt-get install -y openjdk-${JDK_VERSION}-jdk
# Define commonly used JAVA_HOME variable
ENV JAVA_HOME /usr/lib/jvm/java-${JDK_VERSION}-openjdk-amd64
ENV PATH $PATH:$JAVA_HOME/bin

2.modify cdh-docker(opensource-docker)/scripts/hadoop-env.sh file. code as follow:

export JAVA_HOME="/usr/lib/jvm/java-7-openjdk-amd64"

[zybing]

modify some error in opensource-docker module

1.modify opensource-docker/Dockerfile error

in line 67, 68. copy directory(/root) is error , change it as follow:
'COPY scripts/restart-hadoop-spark.sh /usr/bin'
'RUN chmod +x /usr/bin/restart-hadoop-spark.sh'

2.change spark vesion in hibench-docker.conf

In this mirror(http://mirror.reverse.net/pub/apache/spark/) , the minimum spark version is 1.5.2.
1.3 is not supported.

3.In base/base-core file, adopt build-all.sh to compile hibench project

/bin/build-all.sh can be used to built hibench for all known Spark and MR versions

4.set opensource docker and cdh docker different name

by this way , opensource-docker module and cdh-docker module can be built to different docker image.

[carsonwang]

These three lines are not used? Remove them?

[zybing]

originally , I think these three lines will provide some hints for users who don't have good network environment. Because downloading maven repositories is a time-consuming process. Maybe you are right, It's not meaningful enough. I can remove these.

[carsonwang]

Thanks for the work, @zybing . I just left some comments. Can we remove #!bin/bash in some files like hadoop-env.sh as the default file doesn't have this line? Please also double check the license header you added. I saw the default file uses a 2.0 version ASF licnese.

[zybing]

@carsonwang .Thanks for your advice. I have added detailed line notes below your questions. For the license header , I use the original files's license header. When I clone this repository, the licence header is already in the config and shell files. should I do some modification?

[carsonwang]

Ok, let's keep the license header for now. Can you please update the PR by removing some "#!bin/bash" as you mentioned? I'll merge it then.

[zybing]

@carsonwang . I remove the"#!/bin/bash" line from files in opensource-docker/scripts folder which don't have the line originally.

[carsonwang]

Thanks @zybing !