Skip to content

Commit

Permalink
Correct cleanup gitlab-ci
Browse files Browse the repository at this point in the history
Found the issue on regular testing hosts:
  https://gitlab.com/tarantool/tarantool/-/jobs/577884238#L7

  Fetching changes...
  00:04
   Reinitialized existing Git repository in
     /home/gitlab-runner/builds/zzyC6hh5/0/tarantool/tarantool/.git/
   Checking out 8ff7f32 as ...
   warning: failed to remove CMakeFiles/Makefile.cmake

Found the job that saved the directories with root permissions
https://gitlab.com/tarantool/tarantool/-/jobs/577768553

The issue appeared because the job that saved directories with root
permissions used the 'shell' runner to run docker container inside.
It caused the gitlab-runner to run the default workspace cleanup
outside the docker container. In opposite to it, when 'docker' runner
is used, the cleanup routine runs inside the docker container and no
fails ever exist, because the root permissions are used in the docker
container and this is the same root permissions for the host. As the
result using 'shell' runner, cleanup routine failed to remove files
created by root inside the docker container and which were shared to
global host with the same permissions, because gitlab-runner runs the
'shell' runner by regular 'gitlab-runner' user, but not by root.

To fix the issue need to run docker containers using the gitlab-runner
only in RO mode with Out-Of-Source builds in it. Either use the
'docker' runners when docker containers are needed. Anyway 'shell'
runner jobs with additional calls to docker containers can't be
control for the branches of developers, to avoid of it need to make
local cleanup routine instead of default to the working paths for each
job with 'shell' runners use.

Decided to setup gitlab-runner configuration as described in:
https://docs.gitlab.com/ce/ci/yaml/README.html#git-clean-flags

Also got the issue with left data from previous builds at the
submodule pathes, like here:
  https://gitlab.com/tarantool/tarantool/-/jobs/574199256#L3141

   Undefined symbols for architecture x86_64:
     "_u_isprint_66", referenced from:
         _yaml_emitter_is_printable in libyaml_static.a(emitter.c.o)
   ld: symbol(s) not found for architecture x86_64
   clang: error: linker command failed with exit code 1 (...)

To fix it was added the command to clean all available git submodules:
  git submodule foreach git clean -ffdx

Closes #5036

(cherry picked from commit a654760)
  • Loading branch information
avtikhon committed Jun 10, 2020
1 parent f3d0b6f commit 11acb89
Showing 1 changed file with 24 additions and 9 deletions.
33 changes: 24 additions & 9 deletions .gitlab-ci.yml
Expand Up @@ -5,6 +5,12 @@ stages:

variables:
GITLAB_MAKE: "make -f .gitlab.mk"
GIT_CLEAN_COMMAND: "git clean -ffdx -e packpack && git submodule foreach git clean -ffdx && git submodule foreach git status"
GIT_CLEAN_FLAGS: none

.shell_before_script_template: &shell_cleanup_script
before_script:
- docker run -w /source -v ${PWD}:/source -i packpack/packpack:el-7 /bin/bash -c "${GIT_CLEAN_COMMAND}"

# Jobs templates

Expand Down Expand Up @@ -43,18 +49,21 @@ variables:
stage: test
tags:
- docker_test
<<: *shell_cleanup_script

.docker_test_clang8_template: &docker_test_clang8_definition
image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-buster:latest"
stage: test
tags:
- docker_test
<<: *shell_cleanup_script

.pack_template: &pack_definition
<<: *pack_only_definition
stage: test
tags:
- deploy
<<: *shell_cleanup_script
script:
- ${GITLAB_MAKE} package

Expand All @@ -63,6 +72,7 @@ variables:
stage: test
tags:
- deploy_test
<<: *shell_cleanup_script
script:
- ${GITLAB_MAKE} package

Expand All @@ -71,6 +81,7 @@ variables:
stage: test
tags:
- deploy
<<: *shell_cleanup_script
script:
- ${GITLAB_MAKE} deploy

Expand All @@ -79,9 +90,17 @@ variables:
stage: test
tags:
- deploy_test
<<: *shell_cleanup_script
script:
- ${GITLAB_MAKE} deploy

.osx_template: &osx_definition
stage: test
before_script:
- ${GIT_CLEAN_COMMAND}
script:
- ${GITLAB_MAKE} test_osx

.vbox_template: &vbox_definition
stage: test
before_script:
Expand Down Expand Up @@ -152,28 +171,22 @@ release_asan_clang8:

osx_14_release:
<<: *release_only_definition
stage: test
tags:
- osx_14
script:
- ${GITLAB_MAKE} test_osx
<<: *osx_definition

osx_15_release:
stage: test
tags:
- osx_15
script:
- ${GITLAB_MAKE} test_osx
<<: *osx_definition

osx_15_release_lto:
<<: *release_only_definition
stage: test
tags:
- osx_15
variables:
EXTRA_ENV: 'export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;'
script:
- ${GITLAB_MAKE} test_osx
<<: *osx_definition

freebsd_12_release:
<<: *vbox_definition
Expand All @@ -185,6 +198,7 @@ freebsd_12_release:
VMS_PORT: '2232'
MAKE: 'gmake'
script:
- ${GIT_CLEAN_COMMAND}
- ${GITLAB_MAKE} vms_test_freebsd

# ####
Expand Down Expand Up @@ -485,5 +499,6 @@ static_docker_build:
stage: test
tags:
- deploy_test
<<: *shell_cleanup_script
script:
- ${GITLAB_MAKE} test_static_docker_build

0 comments on commit 11acb89

Please sign in to comment.