Skip to content
This repository has been archived by the owner on Nov 6, 2020. It is now read-only.

Updating the CI system #8765

Merged
merged 16 commits into from Aug 25, 2018
Merged

Updating the CI system #8765

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
ebc6b9c
Updating the CI system with the publication of releases and binary fi…
General-Beck Jun 2, 2018
8926b43
add missed scripts
General-Beck Jun 2, 2018
95a7b03
chmod +x scripts
General-Beck Jun 2, 2018
514e4af
fix download link for github
General-Beck Jun 2, 2018
42a0ba5
Merge branch 'master' into gitlab-next
General-Beck Aug 2, 2018
df9a1e4
rebuilding CI scripts
General-Beck Aug 2, 2018
ef997cb
small fixes
General-Beck Aug 2, 2018
0a0d797
update submodule wasm tests
General-Beck Aug 2, 2018
200a826
ci: merge master
5chdn Aug 24, 2018
44bf39d
ci: fix merge leftovers
5chdn Aug 24, 2018
46622d2
ci: remove gitlab-next from ci triggers
5chdn Aug 24, 2018
8a0c089
ci: fix git add in docs script
5chdn Aug 24, 2018
a8c2b29
ci: use nightly instead of master for publish triggers
5chdn Aug 24, 2018
c943b82
ci: remove sleep from gitlab config
5chdn Aug 24, 2018
13594c5
ci: replace ':' with '-' in gitlab targets
5chdn Aug 24, 2018
f1f8362
ci: fix recursive copy in docs script
5chdn Aug 25, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
100 changes: 47 additions & 53 deletions .gitlab-ci.yml
View file
Open in desktop
Expand Up @@ -27,13 +27,11 @@ cache:
- stable
- beta
- tags
- gitlab-next

.publishable_branches: # list of git refs for publishing builds to the "production" locations
only: &publishable_branches
- nightly # Our nightly builds from schedule, on `master`
- "v*" # Our version tags
- gitlab-next
- "v2*" # Our version tags

.collect_artifacts: &collect_artifacts
artifacts:
Expand All @@ -50,7 +48,7 @@ cache:

#### stage: test

test:rust:stable: &test
test-rust-stable: &test
stage: test
script:
- scripts/gitlab/test.sh stable
Expand All @@ -63,27 +61,27 @@ test:rust:stable: &test
only:
- master
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mean tests are not run on pull requests anymore?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only tests with <<: *optional_test included; testing with stable happens everywhere (see section test:rust:stable:); testing with beta and nightly Rust versions are only limited to master branch. It makes some sense, but not the perfect one — probably we want to be sure that our code breaks on nightly immediately, not during the next release.


test:rust:beta:
test-rust-beta:
<<: *optional_test
script:
- scripts/gitlab/test.sh beta

test:rust:nightly:
test-rust-nightly:
<<: *optional_test
script:
- scripts/gitlab/test.sh nightly

#test:rustfmt:
# <<: *optional_test
# script:
# - scripts/gitlab/rustfmt.sh
test-lint-rustfmt:
<<: *optional_test
script:
- scripts/gitlab/rustfmt.sh

#test:clippy:
# <<: *optional_test
# script:
# - scripts/gitlab/clippy.sh
test-lint-clippy:
<<: *optional_test
script:
- scripts/gitlab/clippy.sh

test:coverage:
test-coverage-kcov:
stage: test
only:
- master
Expand All @@ -96,7 +94,7 @@ test:coverage:

#### stage: build

build:linux:ubuntu:amd64: &build
build-linux-ubuntu-amd64: &build
stage: build
only: *releaseable_branches
variables:
Expand All @@ -108,31 +106,31 @@ build:linux:ubuntu:amd64: &build
- rust-stable
allow_failure: true

build:linux:ubuntu:i386:
build-linux-ubuntu-i386:
<<: *build
image: parity/rust-i686:gitlab-ci
variables:
CARGO_TARGET: i686-unknown-linux-gnu

build:linux:ubuntu:arm64:
build-linux-ubuntu-arm64:
<<: *build
image: parity/rust-arm64:gitlab-ci
variables:
CARGO_TARGET: aarch64-unknown-linux-gnu

build:linux:ubuntu:armhf:
build-linux-ubuntu-armhf:
<<: *build
image: parity/rust-armv7:gitlab-ci
variables:
CARGO_TARGET: armv7-unknown-linux-gnueabihf

build:linux:android:armhf:
build-linux-android-armhf:
<<: *build
image: parity/rust-android:gitlab-ci
variables:
CARGO_TARGET: armv7-linux-androideabi

build:macos:x86_64:
build-darwin-macos-x86_64:
<<: *build
variables:
CARGO_TARGET: x86_64-apple-darwin
Expand All @@ -142,7 +140,7 @@ build:macos:x86_64:
- osx
<<: *collect_artifacts
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the previous jobs (e.g. linux:linux:android:armhf) get their *collect_artifacts block from the *build anchor, doesn't that also work for this one?


build-windows-x86_64:
build-windows-msvc-x86_64:
stage: build
only: *releaseable_branches
cache:
Expand All @@ -162,7 +160,7 @@ build-windows-x86_64:

#### stage: package

package:linux:snap:amd64: &package_snap
package-linux-snap-amd64: &package_snap
stage: package
only: *releaseable_branches
cache: {}
Expand All @@ -171,41 +169,41 @@ package:linux:snap:amd64: &package_snap
variables:
CARGO_TARGET: x86_64-unknown-linux-gnu
dependencies:
- build:linux:ubuntu:amd64
- build-linux-ubuntu-amd64
script:
- scripts/gitlab/package-snap.sh
tags:
- rust-stable
<<: *collect_artifacts

package:linux:snap:i386:
package-linux-snap-i386:
<<: *package_snap
variables:
BUILD_ARCH: i386
CARGO_TARGET: i686-unknown-linux-gnu
dependencies:
- build:linux:ubuntu:i386
- build-linux-ubuntu-i386

package:linux:snap:arm64:
package-linux-snap-arm64:
<<: *package_snap
variables:
BUILD_ARCH: arm64
CARGO_TARGET: aarch64-unknown-linux-gnu
dependencies:
- build:linux:ubuntu:arm64
- build-linux-ubuntu-arm64

package:linux:snap:armhf:
package-linux-snap-armhf:
<<: *package_snap
variables:
BUILD_ARCH: armhf
CARGO_TARGET: armv7-unknown-linux-gnueabihf
dependencies:
- build:linux:ubuntu:armhf
- build-linux-ubuntu-armhf


#### stage: publish

publish:linux:snap:amd64: &publish_snap
publish-linux-snap-amd64: &publish_snap
stage: publish
only: *publishable_branches
image: snapcore/snapcraft:stable
Expand All @@ -215,70 +213,67 @@ publish:linux:snap:amd64: &publish_snap
variables:
BUILD_ARCH: amd64
dependencies:
- package:linux:snap:amd64
- package-linux-snap-amd64
script:
- scripts/gitlab/publish-snap.sh
tags:
- rust-stable

publish:linux:snap:i386:
publish-linux-snap-i386:
<<: *publish_snap
before_script:
- *determine_version
#- sleep 60
variables:
BUILD_ARCH: i386
dependencies:
- package:linux:snap:i386
- package-linux-snap-i386

publish:linux:snap:arm64:
publish-linux-snap-arm64:
<<: *publish_snap
before_script:
- *determine_version
#- sleep 360
variables:
BUILD_ARCH: arm64
dependencies:
- package:linux:snap:arm64
- package-linux-snap-arm64

publish:linux:snap:armhf:
publish-linux-snap-armhf:
<<: *publish_snap
before_script:
- *determine_version
#- sleep 520
variables:
BUILD_ARCH: armhf
dependencies:
- package:linux:snap:armhf
- package-linux-snap-armhf

publish:docker:parity:amd64: &publish_docker
publish-docker-parity-amd64: &publish_docker
stage: publish
only: *publishable_branches
cache: {}
dependencies:
- build:linux:ubuntu:amd64
- build-linux-ubuntu-amd64
tags:
- shell
allow_failure: true
script:
- scripts/gitlab/publish-docker.sh parity

publish:docker:parity-evm:amd64:
publish-docker-parityevm-amd64:
<<: *publish_docker
script:
- scripts/gitlab/publish-docker.sh parity-evm

publish:github:s3:
publish-github-and-s3:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my ideal world we would split this into two separate steps.
But let's merge it as it is for now.

stage: publish
only: *publishable_branches
cache: {}
dependencies:
- build:linux:ubuntu:amd64
- build:linux:ubuntu:i386
- build:linux:ubuntu:armhf
- build:linux:ubuntu:arm64
- build:macos:x86_64
- build-windows-x86_64
- build-linux-ubuntu-amd64
- build-linux-ubuntu-i386
- build-linux-ubuntu-armhf
- build-linux-ubuntu-arm64
- build-darwin-macos-x86_64
- build-windows-msvc-x86_64
before_script:
- *determine_version
script:
Expand All @@ -290,11 +285,10 @@ publish:github:s3:

####stage: docs

json:rpc:docs:
docs-rpc-json:
stage: docs
only:
- tags
- gitlab-next
cache: {}
script:
- scripts/gitlab/rpc-docs.sh
Expand Down
2 changes: 1 addition & 1 deletion scripts/gitlab/package-snap.sh
View file
Open in desktop
Expand Up @@ -3,7 +3,7 @@
set -e # fail on any error
set -u # treat unset variables as error
case ${CI_COMMIT_REF_NAME} in
master|*v2.1*|gitlab-next) export GRADE="devel";;
nightly|*v2.1*) export GRADE="devel";;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we going to publish nightlies only from the special nightly branch? Why not from the master?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

master is triggered always after we merge a pull request, so multiple times per day and nightly is only happing once per night which is sufficient. now, master does not create a snap which is fine IMHO.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@5chdn Alternatively, we can avoid triggering the actual build-and-package on master on every push (only tests), and trigger the builds only once per day with a cron-like trigger (Gitlab CI already supports those). This way, there would be no need in maintaining a separate nightly branch manually.

If you agree with me that not having to maintain separate nightly is desired, please open a separate issue for this (or let me know, and I'll do this).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are doing that. nightly is a tag.

beta|*v2.0*) export GRADE="stable";;
stable|*v1.11*) export GRADE="stable";;
*) echo "No release" exit 0;;
Expand Down
2 changes: 1 addition & 1 deletion scripts/gitlab/publish-snap.sh
View file
Open in desktop
Expand Up @@ -4,7 +4,7 @@ set -e # fail on any error
set -u # treat unset variables as error

case ${CI_COMMIT_REF_NAME} in
master|*v2.1*|gitlab-next) export CHANNEL="edge";;
nightly|*v2.1*) export CHANNEL="edge";;
beta|*v2.0*) export CHANNEL="beta";;
stable|*v1.11*) export CHANNEL="stable";;
*) echo "No release" exit 0;;
Expand Down
2 changes: 1 addition & 1 deletion scripts/gitlab/push.sh
View file
Open in desktop
Expand Up @@ -14,7 +14,7 @@ RELEASE_TABLE="$(echo "${RELEASE_TABLE//\$VERSION/${VERSION}}")"
#The text in the file CANGELOG.md before which the table with links is inserted. Must be present in this file necessarily
REPLACE_TEXT="The full list of included changes:"
case ${CI_COMMIT_REF_NAME} in
master|*v2.1*|gitlab-next) NAME="Parity "$VERSION" nightly";;
nightly|*v2.1*) NAME="Parity "$VERSION" nightly";;
beta|*v2.0*) NAME="Parity "$VERSION" beta";;
stable|*v1.11*) NAME="Parity "$VERSION" stable";;
*) echo "No release" exit 0;;
Expand Down
2 changes: 1 addition & 1 deletion scripts/gitlab/rpc-docs.sh
View file
Open in desktop
Expand Up @@ -32,7 +32,7 @@ setup_git() {
commit_files() {
echo "__________Commit files__________"
git checkout -b rpcdoc-update-${CI_COMMIT_REF_NAME}
git commit .
git add .
git commit -m "Update docs to ${CI_COMMIT_REF_NAME}"
git tag -a "${CI_COMMIT_REF_NAME}"
}
Expand Down