Building Spring YARN Projects with Maven :: Learn how to build a Spring YARN Project with Maven
Java Shell

README.adoc

tags projects
maven
hadoop
yarn
spring-hadoop

This guide walks you through using Maven to build a simple Spring YARN project.

What you’ll build

You’ll create a simple app and then build it using Maven.

Note
In this guide we are not trying to create fully working YARN application, instead we focus on project and build model.

What you’ll need

  • About 15 minutes

  • A favorite text editor or IDE

  • JDK 6 or later

If you’re not familiar with gradle, refer to Building Java Projects with Maven.

Set up the project

First you set up a Java project for Maven to build. To keep the focus on Maven, make the project as simple as possible for now.

Here you create a ContainerApplication class.

gs-maven-yarn-container/src/main/java/hello/container/ContainerApplication.java

link:complete/gs-maven-yarn-container/src/main/java/hello/container/ContainerApplication.java[]

Here you create an AppmasterApplication class.

gs-maven-yarn-appmaster/src/main/java/hello/appmaster/AppmasterApplication.java

link:complete/gs-maven-yarn-appmaster/src/main/java/hello/appmaster/AppmasterApplication.java[]

Here you create a ClientApplication class.

gs-maven-yarn-client/src/main/java/hello/client/ClientApplication.java

link:complete/gs-maven-yarn-client/src/main/java/hello/client/ClientApplication.java[]

Here you create a application yaml configuration file for all sub-projects.

gs-maven-yarn-container/src/main/resources/application.yml

link:complete/gs-maven-yarn-container/src/main/resources/application.yml[]

gs-maven-yarn-appmaster/src/main/resources/application.yml

link:complete/gs-maven-yarn-appmaster/src/main/resources/application.yml[]

gs-maven-yarn-client/src/main/resources/application.yml

link:complete/gs-maven-yarn-client/src/main/resources/application.yml[]

Understanding Maven Usage with Spring YARN

Here you create a main parent pom.xml file.

pom.xml

link:complete/pom.xml[]

We created a project structure placing appmaster, container and client in its own sub-projects. Additionally we added a fourth project named dist which only purpose is to use assembly plugin to collect all artifacts together when project is build.

Here you create an appmaster pom.xml file.

gs-maven-yarn-appmaster/pom.xml

link:complete/gs-maven-yarn-appmaster/pom.xml[]

Here you create a container pom.xml file.

gs-maven-yarn-container/pom.xml

link:complete/gs-maven-yarn-container/pom.xml[]

Here you create a client pom.xml file.

gs-maven-yarn-client/pom.xml

link:complete/gs-maven-yarn-client/pom.xml[]

Here you create a distribution pom.xml file.

gs-maven-yarn-dist/pom.xml

link:complete/gs-maven-yarn-dist/pom.xml[]

Here you create an assembly assembly.xml file.

gs-maven-yarn-dist/assembly.xml

link:complete/gs-maven-yarn-dist/assembly.xml[]

Build Application Packages

Run build.

mvn clean package

You should see three jar files created.

gs-maven-yarn-dist/target/gs-maven-yarn-dist/gs-maven-yarn-client-0.1.0.jar
gs-maven-yarn-dist/target/gs-maven-yarn-dist/gs-maven-yarn-container-0.1.0.jar
gs-maven-yarn-dist/target/gs-maven-yarn-dist/gs-maven-yarn-appmaster-0.1.0.jar

You can run the project with java -jar gs-maven-yarn-dist/target/dist/gs-maven-yarn-client-0.1.0.jar but we haven’t added the code to actually submit the YARN application yet. So, all you will see is some logging mesages. You will build a complete application that you can submit to YARN in the other Getting Started Guides for Spring YARN.

Summary

Congratulations! You have now created a simple yet effective Maven build file for building Spring YARN projects.