Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Supporting files to build/test via jenkins Changes to build/test via jenkins, which also means running all tests in a container instead of directly on the host: - Jenkinsfile controls the stages, build.sh does the build and run-tests.sh does the regression tests. - Create a new container falcosecurity/falco-tester that includes the dependencies required to run the regression tests. This is a different image than falco-builder because it doesn't need to be centos 6 based, doesn't install any compiler/etc, and installs the test running framework we use (avocado). We now use a newer version of avocado, which resulted in some small changes to how it is run and how yaml options are parsed. - Modify run_regression_tests.sh to download trace files to the build directory and only if not present. Also honor BUILD_TYPE/BUILD_DIR, which is provided via the docker run cmd. - The package tests are now moved to a separate falco_tests_package.yaml file. They will use rpm installs by default instead of debian packages. Also add the ability to install rpms in addition to debian packages. - Automate the process of creating the docker local package by: 1) Adding CMake rules to copy the Dockerfile, entrypoint to the build directory and 2) Copy test trace files and rules into the build directory. This allows running the docker build command from build/docker/local instead of the source directory. - Modify the way the container test is run a bit to use the trace files/rules copied into the container directly instead of host-mounted trace files. * Use container builder + tester for travis We'll probably be using jenkins soon, but this will allow switching back to travis later if we want. * Use download.draios.com for binutils packages That way we won't be dependent on snapshot.debian.org.
- Loading branch information
Showing
21 changed files
with
326 additions
and
98 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| add_subdirectory(local) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| add_subdirectory(traces) | ||
| add_subdirectory(rules) | ||
|
|
||
| add_custom_target(local-Dockerfile ALL | ||
| DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/Dockerfile) | ||
|
|
||
| add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/Dockerfile | ||
| COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/Dockerfile ${CMAKE_CURRENT_BINARY_DIR}/Dockerfile | ||
| DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/Dockerfile) | ||
|
|
||
| add_custom_target(local-docker-entrypoint ALL | ||
| DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/docker-entrypoint) | ||
|
|
||
| add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/docker-entrypoint | ||
| COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/docker-entrypoint.sh ${CMAKE_CURRENT_BINARY_DIR}/docker-entrypoint.sh | ||
| DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/docker-entrypoint.sh) | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Note: list of rules is created at cmake time, not build time | ||
| file(GLOB test_rule_files | ||
| "${CMAKE_CURRENT_SOURCE_DIR}/../../../test/rules/*.yaml") | ||
|
|
||
| foreach(rule_file_path ${test_rule_files}) | ||
| get_filename_component(rule_file ${rule_file_path} NAME) | ||
| add_custom_target(docker-local-rule-${rule_file} ALL | ||
| DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${rule_file}) | ||
| add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${rule_file} | ||
| COMMAND ${CMAKE_COMMAND} -E copy ${rule_file_path} ${CMAKE_CURRENT_BINARY_DIR}/${rule_file} | ||
| DEPENDS ${rule_file_path}) | ||
| endforeach() | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Note: list of traces is created at cmake time, not build time | ||
| file(GLOB test_trace_files | ||
| "${CMAKE_CURRENT_SOURCE_DIR}/../../../test/trace_files/*.scap") | ||
|
|
||
| foreach(trace_file_path ${test_trace_files}) | ||
| get_filename_component(trace_file ${trace_file_path} NAME) | ||
| add_custom_target(docker-local-trace-${trace_file} ALL | ||
| DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${trace_file}) | ||
| add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${trace_file} | ||
| COMMAND ${CMAKE_COMMAND} -E copy ${trace_file_path} ${CMAKE_CURRENT_BINARY_DIR}/${trace_file} | ||
| DEPENDS ${trace_file_path}) | ||
| endforeach() | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| FROM centos:7 | ||
|
|
||
| ENV FALCO_VERSION 0.1.1dev | ||
| ENV BUILD_TYPE Release | ||
|
|
||
| RUN yum -y install epel-release && \ | ||
| yum -y install \ | ||
| python-pip \ | ||
| docker \ | ||
| jq \ | ||
| unzip | ||
|
|
||
| RUN pip install avocado-framework avocado-framework-plugin-varianter-yaml-to-mux | ||
|
|
||
| COPY entrypoint.sh / | ||
|
|
||
| ENTRYPOINT ["/entrypoint.sh"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -euxo pipefail | ||
|
|
||
| SOURCE_DIR=/source | ||
| BUILD_DIR=/build | ||
| TASK=${1:-test} | ||
|
|
||
| if [ $TASK == "test" ]; then | ||
| echo "Building local docker image falcosecurity/falco:test from latest debian package..." | ||
| cp $BUILD_DIR/$BUILD_TYPE/falco*.deb $BUILD_DIR/$BUILD_TYPE/docker/local | ||
| cd $BUILD_DIR/$BUILD_TYPE/docker/local && docker build --build-arg FALCO_VERSION=${FALCO_VERSION} -t falcosecurity/falco:test . | ||
|
|
||
| echo "Running regression tests" | ||
| cd $SOURCE_DIR/falco/test | ||
| bash run_regression_tests.sh $BUILD_DIR/$BUILD_TYPE | ||
|
|
||
| docker rmi falcosecurity/falco:test || true | ||
| exit 0 | ||
| fi | ||
|
|
||
| if [ $TASK == "bash" ]; then | ||
| exec /bin/bash | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| void setBuildStatus(String context, String message, String state) { | ||
| step([ | ||
| $class: "GitHubCommitStatusSetter", | ||
| reposSource: [$class: "ManuallyEnteredRepositorySource", url: "https://github.com/falcosecurity/falco"], | ||
| contextSource: [$class: "ManuallyEnteredCommitContextSource", context: context], | ||
| errorHandlers: [[$class: "ChangingBuildStatusErrorHandler", result: "UNSTABLE"]], | ||
| statusResultSource: [ $class: "ConditionalStatusResultSource", results: [[$class: "AnyBuildResult", message: message, state: state]] ] | ||
| ]); | ||
| } | ||
|
|
||
| pipeline { | ||
| agent { label "agent-docker-builder" } | ||
| stages { | ||
| stage("Check out dependencies") { | ||
| steps { | ||
| dir("falco") { | ||
| checkout([$class: "GitSCM", | ||
| branches: [[name: "refs/heads/"+env.BRANCH_NAME]], | ||
| doGenerateSubmoduleConfigurations: false, | ||
| extensions: [], | ||
| submoduleCfg: [], | ||
| userRemoteConfigs: [[credentialsId: "github-jenkins-user-token", url: "https://github.com/draios/falco"]]]) | ||
| } | ||
| dir("sysdig") { | ||
| checkout([$class: "GitSCM", | ||
| branches: [[name: "dev"]], | ||
| doGenerateSubmoduleConfigurations: false, | ||
| extensions: [], | ||
| submoduleCfg: [], | ||
| userRemoteConfigs: [[credentialsId: "github-jenkins-user-token", url: "https://github.com/draios/sysdig"]]]) | ||
| } | ||
| } | ||
| } | ||
| stage("Build") { | ||
| steps { | ||
| script{ | ||
| sh("./falco/scripts/jenkins/build-pipeline/build.sh") | ||
| } | ||
| } | ||
| post { | ||
| success { | ||
| setBuildStatus("Build", "Build Successful", "SUCCESS") | ||
| } | ||
| failure { | ||
| setBuildStatus("Build", "Build Failed", "FAILURE") | ||
| } | ||
| } | ||
| } | ||
| stage("Run tests") { | ||
| steps { | ||
| script{ | ||
| sh("./falco/scripts/jenkins/build-pipeline/run-tests.sh") | ||
| } | ||
| } | ||
| post { | ||
| success { | ||
| setBuildStatus("Run tests", "All tests passed", "SUCCESS") | ||
| } | ||
| failure { | ||
| setBuildStatus("Run tests", "One or more tests failed", "FAILURE") | ||
| } | ||
| } | ||
| } | ||
| } | ||
| post { | ||
| always { | ||
| cleanWs() | ||
| } | ||
| } | ||
| } | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -xeuo pipefail | ||
|
|
||
| export FALCO_VERSION=0.1.$((2700+BUILD_NUMBER))dev | ||
|
|
||
| rm -rf ${WORKSPACE}/build | ||
| mkdir ${WORKSPACE}/build | ||
|
|
||
| docker run --user $(id -u):$(id -g) -v /etc/passwd:/etc/passwd:ro -e FALCO_VERSION=${FALCO_VERSION} -e MAKE_JOBS=4 -v ${WORKSPACE}:/source -v ${WORKSPACE}/build:/build falcosecurity/falco-builder cmake | ||
| docker run --user $(id -u):$(id -g) -v /etc/passwd:/etc/passwd:ro -e FALCO_VERSION=${FALCO_VERSION} -e MAKE_JOBS=4 -v ${WORKSPACE}:/source -v ${WORKSPACE}/build:/build falcosecurity/falco-builder package |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -xeuo pipefail | ||
|
|
||
| export FALCO_VERSION=0.1.$((2700+BUILD_NUMBER))dev | ||
|
|
||
| docker pull falcosecurity/falco-tester | ||
| docker run -v /boot:/boot:ro -v /var/run/docker.sock:/var/run/docker.sock -v /etc/passwd:/etc/passwd:ro -e FALCO_VERSION=${FALCO_VERSION} -v ${WORKSPACE}:/source -v ${WORKSPACE}/build:/build falcosecurity/falco-tester | ||
|
|
||
| exit 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| add_subdirectory(trace_files) |
Oops, something went wrong.