Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/dev 1 #34

Open
wants to merge 97 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
c42074d
jenkins pipeline error
Mar 19, 2024
e68596b
Adding unit tests to the Jenkins pipeline
Mar 19, 2024
5eff380
unit testing with jacoco
Mar 19, 2024
8142b65
Docker setup
Mar 19, 2024
3ec450e
jenkinsfile format correction
Mar 19, 2024
cc90566
more jenkinsfile correction
Mar 19, 2024
5cd88f1
Improving the Jenkinsfile
Mar 19, 2024
9050161
docker build
Mar 19, 2024
c3c1b09
more jenkins changes
Mar 19, 2024
3d80330
kubernetes deployment
Mar 19, 2024
8455994
deployment stage
Apr 16, 2024
73c3009
Correcting Typo
Apr 16, 2024
24b5960
adding sensitive info
Apr 16, 2024
7e65cb8
removed files
Apr 16, 2024
b4fc884
mutation testing
Apr 16, 2024
cc52784
changed test cases
Apr 16, 2024
a32f349
corrected more error
Apr 16, 2024
ac832c4
corrected pipeline error
Apr 16, 2024
a2dd353
Setting up sonarqube
Apr 21, 2024
40ff10a
sonarqube correcion
Apr 21, 2024
05d1fc0
fixing sonarqube
Apr 21, 2024
c8a8f55
fixing sonarqube
Apr 21, 2024
8947c14
fixing maven
Apr 21, 2024
ba05a47
Sonarqube config
Apr 21, 2024
d737421
Sonarqube config
Apr 21, 2024
37206ab
Sonarqube config test
Apr 21, 2024
eae5231
sonarqube config
Apr 21, 2024
a5ba668
Adding quality gates
Apr 27, 2024
958a393
Fixing correction
Apr 27, 2024
beb9fb3
fixing sonargate
Apr 27, 2024
fd5c3e9
formatting jenkinsfile
Apr 27, 2024
bc4dac4
formatting jenkinsfile
Apr 27, 2024
5349102
removing unused import packages
Apr 27, 2024
c0b7f3b
fixing quality gate
Apr 27, 2024
4fcd1cd
fixing quality gate talisman
Apr 27, 2024
e45fab8
fixing quality gate talisman error
Apr 27, 2024
eb2d8c0
Fixing errors
Apr 27, 2024
2160a13
Adding Dependency-check
Apr 27, 2024
3e8ce5b
Adding Dependency-check
Apr 27, 2024
58f7441
pom xml dependency-check
Apr 27, 2024
6642170
fixing pom.xml
Apr 27, 2024
dbaebcb
upgrading spring-boot-start-parent
Apr 27, 2024
4292e6c
Fixing Sonarqube error
Apr 27, 2024
c90c3a1
Fixing Sonarqube error pt2
Apr 27, 2024
f3fa02b
Fixing CVEs
May 3, 2024
db95f16
Fixing more CVE errors
May 3, 2024
ddd0600
Fixing more NumericalApplicationTests.java errors
May 3, 2024
5614a7a
Fixing test errors
May 3, 2024
52f1c29
Fixing the test app errors
May 3, 2024
51e7390
Fixing the qualityGate error
May 3, 2024
e90bf23
Fixing quality gate error
May 3, 2024
47aba90
Fixing quality gate error for responseError
May 3, 2024
8b151ad
More sonarqube fixes
May 3, 2024
05be1f2
More sonarqube gate fixes
May 3, 2024
4cf4f86
More sonarqube gate fixes again
May 3, 2024
a2a7821
More sonarqube gate fix
May 3, 2024
16be0f9
More sonarqube gate fix for code
May 3, 2024
d103e86
More sonarqube gate fix for code for error
May 3, 2024
105df9e
quality gate fix
May 3, 2024
48afe1e
quality gate fix again
May 3, 2024
ca8ac98
More quality gate fix again
May 3, 2024
74c0356
More quality gate fixes
May 3, 2024
c487e5d
fixing errors
May 3, 2024
5752918
upgrading java
May 3, 2024
5d738fd
upgrading plugins
May 3, 2024
a88ac7b
upgrading more plugins
May 3, 2024
f20ad20
upgrading more pom plugins
May 3, 2024
39c3f88
upgrading more pom.xml plugins
May 3, 2024
d06b739
upgrading more pom.xml plugins.
May 3, 2024
a631b16
upgrading more pom.xml plugins for jenkins
May 3, 2024
08e3a01
upgrading more pom.xml plugins for jenkins pipeline
May 3, 2024
92e15c0
quality gate
May 7, 2024
ce82fdd
quality gate pom xml fix
May 7, 2024
e666e95
quality gate pom xml fi 2x
May 7, 2024
7cda40a
quality gate pom fix
May 7, 2024
52395d7
quality gate pom fix testing dependency-check
May 7, 2024
d0f1483
Trivy setup
May 7, 2024
6054010
Trivy setup
May 7, 2024
696df62
Trivy setup with jenkinsfile
May 29, 2024
187dd5d
Trivy setup with jenkinsfile and talisman
May 29, 2024
7f81350
fixing formatting errors
May 29, 2024
25af53c
talisman
May 29, 2024
b986a69
more pipeline fixes
May 29, 2024
3007f32
talisman
May 29, 2024
380dea7
updating bash script
May 30, 2024
8a2cbaf
OPA config
Jun 8, 2024
0f69d6f
Editing dockerfile to pass conftest
Jun 8, 2024
2966732
creating k8s conftest config
Jun 8, 2024
0937ef7
Adding security context to the k8s manifest
Jun 8, 2024
1d8127e
Fixing non numberic user error
Jun 8, 2024
6d2e2b5
scripting kubernetes
Jun 8, 2024
f38b63a
adding kubesec
Jun 8, 2024
459b273
Adding a service account as per kubesec recommendation
Jun 8, 2024
36a44ff
Adding a ReadOnlyRootFilesystem as per kubesec recommendation
Jun 8, 2024
a6285d1
kubesec deployment
Jun 8, 2024
200c0eb
Adding volumes
Jun 8, 2024
2138d40
Running integration tests
Jun 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .idea/.gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions .idea/misc.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .idea/vcs.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 33 additions & 0 deletions .kubernetes/create_aks.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Login to Azure (Uncomment the next line if you need to login or switch accounts)
# az login

# Set Variables
$resourceGroupName = "devsecops"
$location = "northeurope"
$aksClusterName = "Devsecops-aks"
$aksVersion = "1.29.0"

# Create Resource Group if it doesn't exist
$resourceGroupExists = az group exists --name $resourceGroupName
if (-not $resourceGroupExists) {
Write-Output "Creating resource group '$resourceGroupName' in location '$location'."
az group create --name $resourceGroupName --location $location
} else {
Write-Output "Resource group '$resourceGroupName' already exists."
}

# Create AKS Cluster
Write-Output "Creating AKS cluster named '$aksClusterName' in resource group '$resourceGroupName'."
az aks create `
--resource-group $resourceGroupName `
--name $aksClusterName `
--node-count 1 `
--enable-addons monitoring `
--kubernetes-version $aksVersion `
--generate-ssh-keys `
--location $location `
--node-vm-size Standard_B2s `
--load-balancer-sku basic `
--no-wait

Write-Output "AKS cluster creation command has been executed. It may take a few minutes for the cluster to be fully operational."
8 changes: 8 additions & 0 deletions .pydevproject
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?eclipse-pydev version="1.0"?><pydev_project>
<pydev_property name="org.python.pydev.PYTHON_PROJECT_INTERPRETER">Default</pydev_property>
<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python interpreter</pydev_property>
<pydev_pathproperty name="org.python.pydev.PROJECT_SOURCE_PATH">
<path>/${PROJECT_DIR_NAME}/devops-spring-project</path>
</pydev_pathproperty>
</pydev_project>
4 changes: 4 additions & 0 deletions .talismanrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
fileignoreconfig:
- filename: Jenkinsfile
checksum: a4c9cd41b0402afd7b2324cc30a6628cbd0aade31a0d852ef2664b297e2bc117
version: ""
4 changes: 4 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"java.compile.nullAnalysis.mode": "automatic",
"java.configuration.updateBuildConfiguration": "interactive"
}
6 changes: 4 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
FROM openjdk:8-jdk-alpine
EXPOSE 8080
ARG JAR_FILE=target/*.jar
ADD ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
RUN addgroup -S pipeline && adduser -S k8s-pipeline -G pipeline
COPY ${JAR_FILE} /home/k8s-pipeline/app.jar
USER k8s-pipeline
ENTRYPOINT ["java","-jar","/home/k8s-pipeline/app.jar"]
196 changes: 189 additions & 7 deletions Jenkinsfile
Original file line number Diff line number Diff line change
@@ -1,12 +1,194 @@
pipeline {
agent any
agent any

stages {
stage('Build Artifact') {
environment {
// Set JVM options for Maven
deploymentName = "devsecops"
containerName = "devsecops-container"
serviceName = "devsecops-svc"
imageName = "manlikeabz/numeric-app:${GIT_COMMIT}"
applicationURL = "http://devsecops-abzconsultancies.eastus.cloudapp.azure.com/"
applicationURI = "/increment/99"
MAVEN_OPTS = "--add-opens java.base/java.lang=ALL-UNNAMED"
AKS_CLUSTER_NAME = 'Devsecops-aks'
NAMESPACE = 'default'
}

stages {
stage('Build Artifact') {
steps {
// Use environment variable for Maven options
sh 'mvn clean package -DskipTests=true'
archiveArtifacts artifacts: 'target/*.jar', onlyIfSuccessful: true
}
}

stage('Unit Tests - JUnit and Jacoco') {
steps {
sh "mvn test"
}
post {
always {
junit 'target/surefire-reports/*.xml'
jacoco execPattern: 'target/jacoco.exec'
}
}
}

stage('Mutation Tests - PIT') {
steps {
sh 'mvn org.pitest:pitest-maven:mutationCoverage'
}
post {
always {
pitmutation mutationStatsFile: '**/pit-reports/**/mutations.xml'
}
}
}

stage('SonarQube - SAST') {
steps {
withSonarQubeEnv('SonarQube') {
sh "mvn sonar:sonar -Dsonar.projectKey=numeric-application -Dsonar.host.url=http://devsecops-abzconsultancies.eastus.cloudapp.azure.com:9000 -Dsonar.token=squ_b0ffa602f401442384e12c06cbd73a66b51a7d2a"
// Make sure the SonarQube scanner has finished before proceeding
}
script {
// It will wait indefinitely for the SonarQube analysis to complete
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Quality gate not passed: ${qg.status}"
}
}
}
}

stage('Vulnerability Scan - Docker') {
parallel {
stage('Maven Dependency Check') {
steps {
script {
sh "mvn dependency-check:check"
}
}
post {
always {
dependencyCheckPublisher pattern: 'target/dependency-check-report.xml'
}
}
}
stage('Trivy Scan') {
steps {
script {
sh "bash trivy-docker-image-scan.sh"
//trivy image --exit-code 0 --severity HIGH,CRITICAL manlikeabz/numeric-app:${GIT_COMMIT}
}
}
}
stage('OPA Conftest') {
steps {
script {
// Run OPA Conftest against the Dockerfile
sh 'docker run --rm -v $(pwd):/project openpolicyagent/conftest test --policy opa-docker-security.rego Dockerfile'
}
}
}
}
}

stage('Docker Build and Push') {
steps {
script {
// Ensure GIT_COMMIT is populated
GIT_COMMIT = sh(script: 'git rev-parse HEAD', returnStdout: true).trim()
echo "Building and pushing Docker image for commit: ${GIT_COMMIT}"

// Docker login using credentials securely
withDockerRegistry([credentialsId: "90cf476e-ad01-40fe-86fa-4b0599ac41ff", url: ""]) {
sh "printenv"

// Docker build and push commands
sh "docker build -t manlikeabz/numeric-app:${GIT_COMMIT} ."
sh "docker push manlikeabz/numeric-app:${GIT_COMMIT}"
}
}
}
}

// stage('Vulneability Scan - Kubernetes') {
// steps {
// script {
// // Run OPA Conftest scan against the Kubernetes deployment file
// sh "docker run --rm -v $pwd:/project openpolicyagent/conftest test --policy opa-k8s-security.rego k8s_deployment_service.yaml"
// }
// }
// }

stage('Vulnerability Scan - Kubernetes') {
steps {
sh "mvn clean package -DskipTests=true"
archive 'target/*.jar' //so that they can be downloaded later
parallel(
"OPA Scan": {
script {
// Run OPA Conftest scan against the Kubernetes deployment file
sh "docker run --rm -v $(pwd):/project openpolicyagent/conftest test --policy opa-k8s-security.rego k8s_deployment_service.yaml"
}
},
"Kubesec Scan": {
sh "bsh kubesec-scan.sh"
}
}
)
}
}
}

// stage('Kubernetes Deployment - DEV') {
// steps {
// withKubeConfig([credentialsId: 'kubeconfig']) {
// sh "kubectl config use-context ${AKS_CLUSTER_NAME}"
// sh "sed -i 's#replace#manlikeabz/numeric-app:${GIT_COMMIT}#g' k8s_deployment_service.yaml"
// sh "kubectl apply -f k8s_deployment_service.yaml"
// }
// }
// }

stage('Integration Tests - DEV') {
steps {
script {
try {
withKubeConfig([credentialsId: 'kubeconfig']) {
sh "bash integration-tests.sh"
}
} catch (Exception e) {
withKubeConfig([credentialsId: 'kubeconfig']) {
sh "kubectl rollout undo deployment ${deploymentName}"
}
throw e
}
}
}

stage('K8s Deployment - Dev'){
steps{
parallel(
"Deploynment": {
withKubeConfig([credentialsId: 'kubeconfig']) {
sh "bash k8s-deployment.sh"
}
},
"Rollout Status": {
withKubeConfig([credentialsId: 'kubeconfig']) {
sh "bash k8s-deployment-rollout-status.sh"
}
}
)
}
}
}

post {
always {
// Cleanup after Docker to avoid logged in credentials hanging around
sh "docker logout"
echo 'Pipeline execution complete.'
}
}
}
}
14 changes: 14 additions & 0 deletions clusterrolebinding.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: deployment-manager-binding
namespace: default
subjects:
- kind: ServiceAccount
name: jenkins-deployer
namespace: default
roleRef:
kind: Role
name: deployment-manager
apiGroup: rbac.authorization.k8s.io

33 changes: 33 additions & 0 deletions integration-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/bash
#integration-test.sh

sleep 5s

PORT=$(kubectl -n default get svc ${serviceName} -o json | jq .spec.ports[].nodePort)
echo $PORT
echo $applicationURL:$PORT/$applicationURI

if [[ ! -z "$PORT" ]]
then
response=$(curl -s $applicationURL:$PORT$applicationURI)
http_code=$(curl -s -o /dev/null -w "%{http_code}" $applicationURL:$PORT$applicationURI)

if [[ "$response" == 100 ]];
then
echo "Increment Test Passed"
else
echo "Increment Test Failed"
exit 1;
fi;

if [[ "$http_code" == 200 ]];
then
echo "HTTP Status Code Test Passed"
else
echo "HTTP Status code is not 200"
exit 1;
fi;
else
echo "The Service does not have a NodePort"
exit 1;
fi;
11 changes: 11 additions & 0 deletions k8s-deployment-rollout-status.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

sleep 60s

if [[ $(kubectl -n default rollout status deployment/${deploymentName} --timeout 5s) != *"successfully rolled out"* ]];
then
echo "Deployment ${deploymentName} rolled out has failed"
kubectl rollout undo deployment/${deploymentName} -n default
else
echo "Deployment ${deploymentName} rollout success"
fi
15 changes: 15 additions & 0 deletions k8s-deployment.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash

sed -i "s#replace#${imageName}#g" k8s_deployment_service.yaml
# kubectl -n default get deployment ${deploymentName} &> /dev/null

# if [[ $? -ne 0]]; then
# echo "deployment ${deploymentName} not found, creating new deployment"
# kubectl apply -f k8s_deployment_service.yaml -n default
# else
# echo "deployment ${deploymentName} found, updating deployment"
# echo "image name - ${imageName}"
# kubectl set image deployment/${deploymentName} ${containerName}=${imageName} -n default
# fi

kubectl apply -f k8s_deployment_service.yaml -n default
18 changes: 18 additions & 0 deletions k8s_deployment_service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,27 @@ spec:
labels:
app: devsecops
spec:
volumes:
- name: vol
emptyDir: {}
serviceAccount: default
containers:
- image: replace
name: devsecops-container
volumeMounts:
- name: vol
mountPath: /tmp
securityContext:
runAsNonRoot: true
runAsUser: 100
readOnlyRootFilesystem: true
resources:
limits:
cpu: "1"
memory: "1Gi"
requests:
cpu: "100m"
memory: "256Mi"
---
apiVersion: v1
kind: Service
Expand Down
Loading