Authors: Dhananjay Gupta
This project simulates the DevOps environment from developers pushing their code to automated building, testing and analyzing of code. Our Java project simulates this process by downloading repositories from GitHub, uploading each repository to a local GitLab server, which triggers the build process and code analysis in the locally installed Jenkins server. Our Jenkins server configuration built, tested and analyzed project code using JaCoCo and Understand. Our implementation of the DevOps environment focused on Java Maven applications containing unit tests and JaCoCo support.
The project is implemented in Java and built with Gradle. GitHub, GitLab, and Jenkins Java APIs are used to access GitHub.com, the local GitLab server, and the local Jenkins server from the Main.java file. The JGit Java library is used for git repository management within our project. These dependencies are documented in the build.gradle file.
We were able to successfully perform a stress test of our Java application by pushing 54 Github projects through our DevOps simulation without error. From the execution of our application to the last job being built in Jenkins, it took approximately 20 minutes. 25 out of the 54 applications built successfully some of which had included JaCoCo code coverage, unit testing results, and produced Understand code analysis. The remaining 29 failed for a variety of reasons such as missing pom.xml build file, failed unit tests, missing plugins, etc. Please note that our application only pulls the first 100 repository results of our GitHub.com query due to GitHub API limitations. Despite this GitHub API limitation, during our stress test we found that 100 repositories was enough to produce a sufficient number of successful builds.
Please follow these steps:
- VirtualBox Virtual Machine
- 8 GB RAM
- Ubuntu 16.04 64-Bit
- Virtual Disk created on setup 50GB (VDI - Virtual Disk Image)
- General Settings > Advanced > Bidirectional for both sharing clipboard and drag and drop
- System Settings > Processor > 4 Cores
- System Settings > Acceleration > Enable VT-x/AMD-V, Nested Paging,
- Display Settings > Video Memory > 128mb and Enable 3D Acceleration
Initial Setup/ Configuration
Install Oracle JDK 8 (NOT 9)
- Local installation, not with docker,
- Used this tutorial https://about.gitlab.com/installation/#ubuntu
EXTERNAL_URL="http://10.0.2.15"during install. This is the IP for our virtual machine.
Local installation, not with docker,
Used this tutorial https://jenkins.io/doc/book/installing/#debian-ubuntu
Gitlab is running on
http://10.0.2.15so Jenkins needs to run on a different port (
- Stop Jenkins server:
sudo service jenkins stop
- Start Jenkins server:
sudo service jenkins start
- Stop Jenkins server:
please Follow rest of the tutorial to unlock Jenkins, please install suggested plugins
Create First Admin User:
Manage Jenkins > Plugins
- Gitlab Hook
- Test In Progress
- Workspace Cleanup
Manage Jenkins > Configure Global Security
- In section CSRF Protection, Uncheck "
Prevent Cross Site Request Forgery Exploits"
Manage Jenkins > Configure System
- Home Directory should be
- Jenkins URL should be
- Home Directory should be
Gitlab and Jenkins Integration
Used this tutorial https://www.swtestacademy.com/jenkins-gitlab-integration/
Gitlab > Settings > Access Tokens
- To add a Jenkins API Access Token, give it a name, and click on all three (api, read_user, sudo) under Scopes. Copy and store the API Token down because it will not be accessible later.
Jenkins > Manage Jenkins > Configure System > Gitlab
- Connection name: GitlabConnection
- Gitlab host URL: http://10.0.2.15/
- Credentials > Click on Add > Click on Jenkins
- Set Kind to Gitlab API Token
- Enter in the Jenkins API Token you created in Gitlab earlier
- Enter in ID and Description
- Now set Credentials to the credentials just created (Gitlab API token)
- Click on Test Connection to see if it worked
From Jenkins main server page > Click on Credentials in the Menu on the Left > Click on System (right below Credentials) > On the right, click on
Global credentials (unrestricted)> Click on Add Credentials shown on the left. Here add your Gitlab username (i.e. root) and Gitlab password (i.e. rootroot).
Install Maven, Gradle, Git
Download the software at Download, unzip the folder, add the path by editing the .bashrc file
- through home
$ gedit .bashrc
- To the bottom of the file, add line
export PATH=$PATH:/location of the unzipped scitools folder/bin/linux64example: export PATH=$PATH:/home/dhananjay/Downloads/scitools/bin/linux64
$ source .bashrcto include the changes OR logout and login again for the changes to take effect.
- run Understand:
$ understandand make sure to enter the license key and see if it is up and running.
- through home
Clone project repository from Bitbucket into directory:
Main.javafile with text editor or Intellij
- Open up Intellij
- In the welcome screen, select "Open"
- Navigate to the the viren_mody_guillermo_rojas_hernandez_dhananjay_gupta_hw1/project1 folder and select "Open"
- The "Import Project From Gradle" window should pop up.
- Select the following options:
- The gradle project path should be the viren_mody_guillermo_rojas_hernandez_dhananjay_gupta_hw1/project1 path
- Select “Use auto-import”
- Select “Create separate module per source set”
- Select “Use default gradle wrapper”
- For Gradle JVM, use the 1.8 Java version 1.8 JDK
- Project Format: “.ipr (file based)”
- Select OK
Open up Main.java file and modify the variables under the TODO comments in both the customJenkinsJobXML function and the Main function
UNDERSTAND_PATHto the location of the Understand installation, example: “/home/virenmody/Downloads/scitools/bin/linux64/”
CLONED_REPOS_BASE_PATHto the directory where you want cloned repositories installed locally, example: “/home/virenmody/ClonedRepos/”
GITLAB_URLto IP address of the GitLab server, example: "http://10.0.2.15"
GITLAB_API_ACCESS_TOKENto the access token from GitLab, example: “RDtCnzMezmjpih4u6VDm”
GITLAB_USERNAMEto username used to access Gitlab server, example: “root”
GITLAB_PASSWORDto password used to access Gitlab server, example: “rootroot”
JENKINS_URLto Jenkins server url from configuration, Example: "http://10.0.2.15:8081"
JENKINS_USERNAMEto Jenkins server username, Example: “admin”
JENKINS_PASSWORDto Jenkins server password, Example: “admin”
LOCAL_PATHto the directory you want to save cloned repos
- Select Build Program
- Select Run
The analysis report is being generated at the particular job's folder under the jenkins workspace directory as a
.txt and html files on your local machine. Kindly, visit the folders to access the reports.
Example: under the /var/lib/jenkins/workspace/"JobName". The job name is the name of the Repository being analyzed.