This guide walks you through using Gradle to build a simple Spring YARN project.
What you’ll build
You’ll create a simple app and then build it using Gradle.
|In this guide we are not trying to create fully working YARN application, instead we focus on project and build model.|
What you’ll need
About 15 minutes
A favorite text editor or IDE
JDK 6 or later
If you’re not familiar with gradle or don’t have it installed, refer to Building Java Projects with Gradle.
Set up the project
First you set up a Java project for Gradle to build. To keep the focus on Gradle, make the project as simple as possible for now.
First, let’s create a
Next, we create an
The last Java class we need to create is a
Now we need to create an
application YAML configuration file for all
Understanding Gradle Usage with Spring YARN
We need to create a
build.gradle file where we define everything needed for the build.
We will be using the Spring Boot Gradle plugin so
dependency needs to be defined within a
We apply gradle
base plugin to all projects. Technically it’s only
needed for a root project order to have i.e.
clean task in it. But
it doesn’t matter if we add it to all projects.
Below that section we add the following — project version as
apply plugins for
idea and add
resolve all of the third party jar dependencies via Maven
Next we add
dependency to all sub-projects. Additionally we create a
task to copy created jars from a sub-projects into a distribute project
gs-yarn-testing-dist. This will make things easier to
test and run your application.
We add configure section for each sub-project. These are empty for now
but needed in further guides when more specific dependencies are
The Gradle plugin for
Spring Boot automatically creates a task to
repackage a main jar file created from a project.
Next, related to what we previously did, in project
gs-gradle-yarn-dist we add a compile dependency
for all sub-projects. This is required later guides when we create
tests to this project.
It is worth to go through what we did in above section for
Eventually we want to use this project to collect other artifacts together and create some tests. That is why we added dependencies to other projects.
test.dependsOnfor all other projects
assemblytask to make sure that Spring Boot’s gradle plugin does a repackage before any tests are run.
cleantask to wipe up our
targetdirectory. This sounds very
mavencentric but in our samples we try to support same project structure for both
gradle. Also due to limitations in Hadoop’s testing classes, some files are hard coded to be created under
jartask for this project to reduce noise. Disabling this really doesn’t matter but if there is only tests classes in this project this jar would be empty and thus unnecessary.
Last we just add a normal gradle wrapper task which is not needed for project itself but allows to use this build file without installing gradle binaries.
We need to create a
settings.gradle file where we define a name of a
root project and include sub-projects.
Build Application Packages
gradle clean build
You should see three jar files created.
gs-gradle-yarn-dist/target/gs-gradle-yarn-dist/gs-gradle-yarn-client-0.1.0.jar gs-gradle-yarn-dist/target/gs-gradle-yarn-dist/gs-gradle-yarn-container-0.1.0.jar gs-gradle-yarn-dist/target/gs-gradle-yarn-dist/gs-gradle-yarn-appmaster-0.1.0.jar
You can run the project with
java -jar gs-gradle-yarn-dist/target/dist/gs-gradle-yarn-client-0.1.0.jar
but we haven’t added the code to
actually submit the YARN application yet. So, all you will see is some logging mesages.
You will build a complete application that you can submit to YARN in the other
Getting Started Guides for Spring YARN.
Congratulations! You have now created a simple yet effective Gradle build file for building Spring YARN projects.