[Scala] Exclude example module from deploy. Update README.md

apache · Mar 24, 2016 · c1c2484 · c1c2484
1 parent a870484
commit c1c2484
Show file tree

Hide file tree

Showing 6 changed files with 102 additions and 13 deletions.
diff --git a/scala-package/README.md b/scala-package/README.md
@@ -11,6 +11,36 @@ It brings flexible and efficient GPU/CPU computing and state-of-art deep learnin
   in Scala, Java and other languages built on JVM.
 - It also enables you to construct and customize the state-of-art deep learning models in JVM languages,
   and apply them to tasks such as image classification and data science challenges.
+
+Install
+------------
+
+Technically, all you need is the `mxnet-full_2.10-{arch}-{xpu}-0.1.0.jar` in your classpath.
+It will automatically extract the native library to a tempfile and load it.
+
+Currently we provide `linux-x86_64-gpu`, `linux-x86_64-cpu` and `osx-x86_64-cpu`. Support for Windows will come soon.
+Use the following dependency in maven, change the artifactId according to your own architecture, e.g., `mxnet-full_2.10-osx-x86_64-cpu` for OSX (and cpu-only).
+
+```HTML
+<dependency>
+  <groupId>ml.dmlc.mxnet</groupId>
+  <artifactId>mxnet-full_2.10-linux-x86_64-gpu</artifactId>
+  <version>0.1.0</version>
+</dependency>
+```
+
+You can also use `mxnet-core_2.10-0.1.0.jar` and put the compiled native library somewhere in your load path.
+
+```HTML
+<dependency>
+  <groupId>ml.dmlc.mxnet</groupId>
+  <artifactId>mxnet-core_2.10</artifactId>
+  <version>0.1.0</version>
+</dependency>
+```
+
+If you have some native libraries conflict with the ones in the provided 'full' jar (e.g., you use openblas instead of atlas), this is a recommended way.
+Refer to the next section for how to build it from the very source.
 
 Build
 ------------
@@ -22,18 +52,19 @@ Then you can compile the Scala Package by
 make scalapkg
 ```
 
-Run unit/integration tests by
+(Optional) run unit/integration tests by
 
 ```bash
 make scalatest
 ```
 
-If everything goes well, you will find a jar file named like `mxnet_2.10-osx-x86_64-0.1-SNAPSHOT-full.jar` under `assembly/target`. Then you can use this jar in your own project.
+If everything goes well, you will find jars for `assembly`, `core` and `example` modules.
+Also it produces the native library in `native/{your-architecture}/target`, which you can use to cooperate with the `core` module.
 
-Also `scalapkg` target will build jars for `core` and `example` modules. If you've already downloaded and unpacked MNIST dataset to `./data/`, you can run the training example by
+Once you've downloaded and unpacked MNIST dataset to `./data/`, run the training example by
 
 ```bash
-java -Xmx4m -cp scala-package/assembly/target/*:scala-package/examples/target/mxnet-scala-examples_2.10-0.1-SNAPSHOT.jar:scala-package/examples/target/classes/lib/args4j-2.0.29.jar ml.dmlc.mxnet.examples.imclassification.TrainMnist --data-dir=./data/ --num-epochs=10 --network=mlp --cpus=0,1,2,3
+java -Xmx4m -cp scala-package/assembly/target/*:scala-package/examples/target/* ml.dmlc.mxnet.examples.imclassification.TrainMnist --data-dir=./data/ --num-epochs=10 --network=mlp --cpus=0,1,2,3
 ```
 
 Change the arguments and have fun!
@@ -123,8 +154,6 @@ println(s"Final accuracy = $acc")
 
 You can refer to [MXNet Scala Package Examples](https://github.com/javelinjs/mxnet-scala-example)
 for more information about how to integrate MXNet Scala Package into your own project.
-Currently you have to put the Jars into your project's build classpath manully.
-We will provide pre-built binary package on [Maven Repository](http://mvnrepository.com) soon.
 
 License
 -------

diff --git a/scala-package/dev/change-proj-version.sh b/scala-package/dev/change-proj-version.sh
@@ -16,6 +16,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
+# (Yizhi) This is mainly inspired by the script in apache/spark.
+# I did some modificaiton to get it with our project.
+#
 
 set -e
 

diff --git a/scala-package/dev/change-scala-version.sh b/scala-package/dev/change-scala-version.sh
@@ -16,6 +16,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
+# (Yizhi) This is mainly inspired by the script in apache/spark.
+# I did some modificaiton to get it with our project.
+#
 
 set -e
 

diff --git a/scala-package/examples/pom.xml b/scala-package/examples/pom.xml
@@ -15,6 +15,44 @@
   <name>MXNet Scala Package - Examples</name>
   <packaging>pom</packaging>
 
+  <profiles>
+    <profile>
+      <id>release</id>
+      <build>
+        <plugins>
+          <plugin>
+            <groupId>org.apache.maven.plugins</groupId>
+            <artifactId>maven-source-plugin</artifactId>
+            <configuration>
+              <skipSource>true</skipSource>
+            </configuration>
+          </plugin>
+          <plugin>
+            <groupId>org.apache.maven.plugins</groupId>
+            <artifactId>maven-javadoc-plugin</artifactId>
+            <configuration>
+              <skip>true</skip>
+            </configuration>
+          </plugin>
+          <plugin>
+            <groupId>org.apache.maven.plugins</groupId>
+            <artifactId>maven-gpg-plugin</artifactId>
+            <configuration>
+              <skip>true</skip>
+            </configuration>
+          </plugin>
+          <plugin>
+            <groupId>org.sonatype.plugins</groupId>
+            <artifactId>nexus-staging-maven-plugin</artifactId>
+            <configuration>
+              <skipNexusStagingDeployMojo>true</skipNexusStagingDeployMojo>
+            </configuration>
+          </plugin>
+        </plugins>
+      </build>
+    </profile>
+  </profiles>
+
   <build>
     <plugins>
       <plugin>

diff --git a/scala-package/pom.xml b/scala-package/pom.xml
@@ -21,7 +21,7 @@
   </licenses>
   <scm>
     <connection>scm:git:git@github.com:dmlc/mxnet.git</connection>
-    <developerConnection>scm:git:git@github.com:javelinjs/mxnet.git</developerConnection>
+    <developerConnection>scm:git:git@github.com:dmlc/mxnet.git</developerConnection>
     <url>https://github.com/dmlc/mxnet</url>
   </scm>
 
@@ -68,7 +68,6 @@
           <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-source-plugin</artifactId>
-            <version>2.2.1</version>
             <executions>
               <execution>
                 <phase>package</phase>
@@ -84,7 +83,6 @@
           <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-javadoc-plugin</artifactId>
-            <version>2.9.1</version>
             <executions>
               <execution>
                 <phase>package</phase>
@@ -106,7 +104,6 @@
           <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-gpg-plugin</artifactId>
-            <version>1.5</version>
             <executions>
               <execution>
                 <id>sign-artifacts</id>
@@ -117,11 +114,9 @@
               </execution>
             </executions>
           </plugin>
-          <!-- OSSRH Repo -->
           <plugin>
             <groupId>org.sonatype.plugins</groupId>
             <artifactId>nexus-staging-maven-plugin</artifactId>
-            <version>1.6.3</version>
             <extensions>true</extensions>
             <configuration>
               <serverId>ossrh</serverId>
@@ -302,6 +297,27 @@
             </dependency>
           </dependencies>
         </plugin>
+        <plugin>
+          <groupId>org.apache.maven.plugins</groupId>
+          <artifactId>maven-source-plugin</artifactId>
+          <version>2.2.1</version>
+        </plugin>
+        <plugin>
+          <groupId>org.apache.maven.plugins</groupId>
+          <artifactId>maven-javadoc-plugin</artifactId>
+          <version>2.9.1</version>
+        </plugin>
+        <plugin>
+          <groupId>org.apache.maven.plugins</groupId>
+          <artifactId>maven-gpg-plugin</artifactId>
+          <version>1.5</version>
+        </plugin>
+        <!-- OSSRH Repo -->
+        <plugin>
+          <groupId>org.sonatype.plugins</groupId>
+          <artifactId>nexus-staging-maven-plugin</artifactId>
+          <version>1.6.3</version>
+        </plugin>
       </plugins>
     </pluginManagement>
   </build>

diff --git a/tests/travis/run_test.sh b/tests/travis/run_test.sh
@@ -120,7 +120,7 @@ if [ ${TASK} == "scala_test" ]; then
     ln -s ${CACHE_PREFIX}/data ${PWD}/data
 
     export JAVA_HOME=$(/usr/libexec/java_home)
-    export MAVEN_OPTS="-Xmx512m -XX:MaxPermSize=128m"
+    export MAVEN_OPTS="-Xmx512m -XX:MaxPermSize=256m"
 
     make scalapkg || exit -1
     make scalatest || exit -1