Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add integration tests for maven and jar deployments #1702

Merged
merged 49 commits into from
Mar 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
867b6a7
test: initial setup config tests
jhaeu Feb 20, 2024
0797d20
test: renamed and added variables, some small changes
jhaeu Feb 21, 2024
42af1f5
test: increased timeout
jhaeu Feb 21, 2024
59c347c
test: new tests for prio of config files
jhaeu Feb 21, 2024
6b1b6b7
test: increased timeouts mvn tests
jhaeu Feb 21, 2024
c0f4089
test: rename tests
jhaeu Feb 21, 2024
2240efb
test: add tests with startparam
jhaeu Feb 21, 2024
0fe5419
test: restructure to tests scripts outside containers
jhaeu Feb 22, 2024
5913fcb
test: restructure jar tests
jhaeu Feb 22, 2024
5b3a8db
test: restructure mvn tests
jhaeu Feb 22, 2024
5c4289b
test: parametrized tests with runType
jhaeu Feb 22, 2024
2d827bc
test: extracted function prepareTest
jhaeu Feb 22, 2024
ca24732
test: fixes in cli
jhaeu Feb 22, 2024
187a6f0
test: rename tests
jhaeu Feb 22, 2024
f67baee
test: fix fail fast option
jhaeu Feb 22, 2024
3c07a81
test: add runType to container name
jhaeu Feb 22, 2024
585c259
test: add tests for env variables
jhaeu Feb 22, 2024
319083b
test: moved graph building to tests
jhaeu Feb 23, 2024
67d4d07
test: rename variables
jhaeu Feb 23, 2024
21a9bb4
test: cleanup
jhaeu Feb 23, 2024
97bcde2
test: remove unnecessary .m2 in final jar docker image
jhaeu Feb 26, 2024
318dab1
test: use real local .m2
jhaeu Feb 26, 2024
24a6db1
test: fix Dockerfile-jar
jhaeu Feb 27, 2024
f82bfb1
test: integrated docker build to run-tests.sh and renamed to run.sh
jhaeu Feb 27, 2024
92326df
test: generating ors-config.yml files in tests
jhaeu Feb 27, 2024
05cbeca
test: fixes and cleanup
jhaeu Feb 28, 2024
1fc7ebe
test: workflow for integration tests
jhaeu Feb 28, 2024
d4758fd
test: workflow: add missing steps
jhaeu Feb 28, 2024
14fcce7
test: workflow: run (only) two graph building tests explicitly
jhaeu Feb 28, 2024
fe013b4
test: podman build: add explicit param ignorefile
jhaeu Feb 29, 2024
2020975
test: workflow: mvn build to fill cache, include graph builds in matrix
jhaeu Feb 29, 2024
cb658e0
test: workflow: fixed dependency
jhaeu Feb 29, 2024
8964fe7
test: workflow syntax matrix values
jhaeu Feb 29, 2024
c8503e7
test: workflow syntax matrix values
jhaeu Feb 29, 2024
8e367b9
test: workflow debugging: add verbose option
jhaeu Feb 29, 2024
5bfdc69
test: workflow: create graphs dir, colors without tput
jhaeu Feb 29, 2024
9634813
test: use if-else to get real exit status of test in any case
jhaeu Feb 29, 2024
b765019
test: workflow: use all testgroups again
jhaeu Feb 29, 2024
bf3e9cd
test: fix tests for running without precalculated graphs
jhaeu Feb 29, 2024
66ec295
test: workflow: change test patterns
jhaeu Feb 29, 2024
6cd4305
test: workflow: changed test with two program args to run with maven
jhaeu Feb 29, 2024
f6f9f06
test: remove test depending on a graph already existing
jhaeu Feb 29, 2024
691ef51
test: add test with avoid area request
jhaeu Feb 29, 2024
56f8ae8
test: cleanup
jhaeu Feb 29, 2024
dc27151
test: update description
jhaeu Mar 1, 2024
8e74b6e
test: cleanup
jhaeu Mar 1, 2024
6e50475
test: coloring
jhaeu Mar 1, 2024
978a0a5
test: cleanup
jhaeu Mar 1, 2024
9af557e
test: test building all graphs
jhaeu Mar 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 112 additions & 0 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# This workflow will build a Java project with Maven
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven

name: Integration test suite

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

jobs:
# debian-12-tomcat-war-systemd:
# name: Run Debian 12 Tomcat WAR systemd integration tests
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v2
# with:
# fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
# id: buildx
# with:
# install: true
# - name: Set up JDK 17
# uses: actions/setup-java@v2
# with:
# distribution: 'temurin'
# java-version: '17'
# - name: Cache Maven packages
# uses: actions/cache@v4
# with:
# path: ~/.m2
# key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
# restore-keys: ${{ runner.os }}-m2
# # - name: Maven dependency resolution in ~/.m2
# # run: |
# # mvn dependency:go-offline -Dmaven.repo.local=~/.m2
# # mvn clean
# - name: Run integration tests
# run: ./.integration-scenarios/debian-12-tomcat-war-systemd/run.sh -t -p
debian-12-jar-mvn-build-docker:
name: Update dependencies in maven cache
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
id: buildx
with:
install: true
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: '17'
- name: Cache Maven packages
uses: actions/cache@v4
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2
- name: Build Docker Images for jar and mvn
run: mvn clean package -DskipTests
# debian-12-jar-mvn-build-graphs:
# name: Build Graphs with mvn and jar
# needs: debian-12-jar-mvn-build-docker
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v2
# with:
# fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
# id: buildx
# with:
# install: true
# - name: Build Docker Images for jar and mvn
## run: ./.integration-scenarios/debian-12-jar-mvn/run.sh -mjt 'build-graph-*'
# run: |
# ./.integration-scenarios/debian-12-jar-mvn/tests/build-graph-bike-electric.sh jar
# ./.integration-scenarios/debian-12-jar-mvn/tests/build-graph-bike-mountain.sh mvn
debian-12-jar-mvn-run-tests:
needs: debian-12-jar-mvn-build-docker
name: Run Debian 12 Maven JAR integration tests
runs-on: ubuntu-latest
strategy:
matrix:
runtype: [ '-m' , '-j' ]
testgroup: [ build*car*, build*public*, arg-overrides*, check*, lookup-yml*, missing-config*, specify-yml* ]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
id: buildx
with:
install: true
- name: Cache Maven packages
uses: actions/cache@v4
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2
- name: Run test group with run type
run: |
echo "Scenario: ${{ matrix.scenario }}"
echo "Parameters: ${{ matrix.parameters }}"
.integration-scenarios/debian-12-jar-mvn/run.sh -b ${{ matrix.runtype }} -t ${{ matrix.testgroup }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,4 @@ ors-config-repo/
# docs stuff
docs/.vitepress/dist
docs/.vitepress/cache
/.integration-scenarios/debian-12-jar-mvn/graphs_volume/
45 changes: 45 additions & 0 deletions .integration-scenarios/debian-12-jar-mvn/Dockerfile-jar
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
FROM docker.io/debian:bookworm-slim as builder

RUN apt-get update -y && \
apt-get install -y openjdk-17-jdk-headless maven

ARG CONTAINER_CONF_DIR_USER=/root/.config/openrouteservice
ARG CONTAINER_CONF_DIR_ETC=/etc/openrouteservice
ARG CONTAINER_WORK_DIR=/home/ors/openrouteservice

WORKDIR $CONTAINER_WORK_DIR

COPY ors-api ors-api
COPY ors-engine ors-engine
COPY ors-report-aggregation ors-report-aggregation
COPY pom.xml pom.xml

RUN mvn clean package -DskipTests -PbuildFatJar -DCI=true

FROM docker.io/debian:bookworm-slim as final

RUN apt-get update -y && \
apt-get install -y openjdk-17-jdk-headless

ARG CONTAINER_CONF_DIR_USER=/root/.config/openrouteservice
ARG CONTAINER_CONF_DIR_ETC=/etc/openrouteservice
ARG CONTAINER_WORK_DIR=/home/ors/openrouteservice

WORKDIR $CONTAINER_WORK_DIR

COPY --from=builder $CONTAINER_WORK_DIR/ors-api/target/ors.jar ors.jar
COPY ors-api/src/test/files ors-api/src/test/files

RUN mkdir graphs
RUN mkdir -p "$CONTAINER_CONF_DIR_USER"
RUN mkdir -p "$CONTAINER_CONF_DIR_ETC"

RUN chmod -R 770 .
RUN chmod -R 770 "$CONTAINER_CONF_DIR_USER"
RUN chmod -R 770 "$CONTAINER_CONF_DIR_ETC"

ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64

EXPOSE 8082

ENTRYPOINT [ "java", "-jar", "ors.jar" ]
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
.metadata/
.git
.gitignore
**/*.pbf

.idea/
*.iml
.DS_Store
nohup.out
.python-version
.project

# Ignore all ors and app config files except for the sample ones
ors-api/src/main/resources/ors-config*
ors-api/src/main/resources/ors-config-sample.json

# folders generated by integration test runs
**/graphs-apitests/
ors-api/logs

# Ignore all target folders
**/target/

cgiar_provider/
cgiar_cache/
graphs/
build/
logs/
docker/
docs/
.github/
Dockerfile
envs/
.jqwik-database
!ors-engine/src/test/files/
31 changes: 31 additions & 0 deletions .integration-scenarios/debian-12-jar-mvn/Dockerfile-mvn
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
FROM docker.io/debian:bookworm-slim as final

RUN apt-get update -y && \
apt-get install -y openjdk-17-jdk-headless maven

ARG CONTAINER_CONF_DIR_USER=/root/.config/openrouteservice
ARG CONTAINER_CONF_DIR_ETC=/etc/openrouteservice
ARG CONTAINER_WORK_DIR=/home/ors/openrouteservice

WORKDIR $CONTAINER_WORK_DIR

COPY ors-api ors-api
COPY ors-engine ors-engine
COPY ors-report-aggregation ors-report-aggregation
COPY pom.xml pom.xml

RUN mkdir graphs
RUN mkdir -p "$CONTAINER_CONF_DIR_USER"
RUN mkdir -p "$CONTAINER_CONF_DIR_ETC"

RUN chmod -R 770 .
RUN chmod -R 770 "$CONTAINER_CONF_DIR_USER"
RUN chmod -R 770 "$CONTAINER_CONF_DIR_ETC"

ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64

RUN mvn clean compile

EXPOSE 8082

ENTRYPOINT [ "mvn", "spring-boot:run", "-DskiptTests", "-Dmaven.test.skip=true" ]
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
.metadata/
.git
.gitignore
**/*.pbf

.idea/
*.iml
.DS_Store
nohup.out
.python-version
.project

# Ignore all ors and app config files except for the sample ones
ors-api/src/main/resources/ors-config*
ors-api/src/main/resources/ors-config-sample.json

# folders generated by integration test runs
**/graphs-apitests/
ors-api/logs

# Ignore all target folders
**/target/

cgiar_provider/
cgiar_cache/
graphs/
build/
logs/
docker/
docs/
.github/
Dockerfile
envs/
.jqwik-database
!ors-engine/src/test/files/
30 changes: 30 additions & 0 deletions .integration-scenarios/debian-12-jar-mvn/files/.build-graph.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Called from tests/build-graph-*.sh

TESTROOT="$( cd "$(dirname "$0")"/.. >/dev/null 2>&1 ; pwd -P )"
source $TESTROOT/files/testfunctions.sh
source $TESTROOT/files/test.conf
runType=$1
callingScript=$2
enabledProfile=$3
expectedProfiles=$4
prepareTest $runType $callingScript

orsConfig=$(makeTempFile $callingScript "\
ors:
engine:
source_file: ors-api/src/test/files/heidelberg.osm.gz
profiles:
${enabledProfile}:
enabled: true")

podman run --replace --name "${CONTAINER}" -p "${HOST_PORT}":8082 \
-v "${M2_FOLDER}":/root/.m2 \
-v "${TESTROOT}/graphs_volume":"${CONTAINER_WORK_DIR}/graphs" \
-v "${orsConfig}":"${CONTAINER_WORK_DIR}/ors-config.yml" \
"local/${IMAGE}:latest" &

awaitOrsReady 60 "${HOST_PORT}"
profiles=$(requestEnabledProfiles $HOST_PORT)
cleanupTest

assertEquals "$expectedProfiles" "$profiles"
7 changes: 7 additions & 0 deletions .integration-scenarios/debian-12-jar-mvn/files/test.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

CONTAINER_WORK_DIR=/home/ors/openrouteservice
CONTAINER_CONF_DIR_USER=/root/.config/openrouteservice
CONTAINER_CONF_DIR_ETC=/etc/openrouteservice
IMAGE_NAME_JAR=test-ors-jar
IMAGE_NAME_MVN=test-ors-mvn
Loading
Loading