Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

travis-ci: deploy RPM / Deb packages #162

Merged
merged 1 commit into from Jun 29, 2020

Conversation

Totktonada
Copy link
Member

Now php-tarantool packages are deployed into two kinds of repositories:
packagecloud.io and to S3 based ones. The download.tarantool.org now
points to the latter, but we keep packagecloud.io for a while to don't
break users, which use it directly (we don't recommend it though). See
1 for more info about deprecation of our packagecloud.io repositories.

The deployment scripts allows to provide two configurations: production
and staging. The former is for deployment from the master branch and
from a git tag. The latter is for deployments from a developer branch.
It is useful when something should be verified in a specific environment
using a built package or when the deployment process itself was modified
and should be tested.

The difference between production and staging deployment process is how
duplicate package versions are handled. It give an error for production
deployment, but does not for staging. The new package is discarded in
the case for packagecloud.io, but it replaces the old package in S3
repositories.

Read comments in the deployment scripts for more details.

Fixes #117

@Totktonada Totktonada changed the title travis-ci: deploy RPM / Deb packages WIP: travis-ci: deploy RPM / Deb packages Jun 24, 2020
@Totktonada Totktonada force-pushed the Totktonada/php7-v2-deploy-rpm-deb-packages branch 7 times, most recently from a8cd34c to b7de22b Compare June 25, 2020 16:19
@Totktonada
Copy link
Member Author

Totktonada commented Jun 25, 2020

@Totktonada Totktonada force-pushed the Totktonada/php7-v2-deploy-rpm-deb-packages branch from b7de22b to 5f50bd1 Compare June 25, 2020 19:11
@Totktonada Totktonada force-pushed the Totktonada/php7-v2-deploy-rpm-deb-packages branch from 5f50bd1 to 5d16e04 Compare June 25, 2020 20:42
@Totktonada Totktonada changed the title WIP: travis-ci: deploy RPM / Deb packages travis-ci: deploy RPM / Deb packages Jun 25, 2020
@Totktonada Totktonada requested a review from avtikhon June 25, 2020 20:44
@Totktonada Totktonada force-pushed the Totktonada/php7-v2-deploy-rpm-deb-packages branch 2 times, most recently from a0e115c to 0d3a227 Compare June 26, 2020 16:01
Now php-tarantool packages are deployed into two kinds of repositories:
packagecloud.io and to S3 based ones. The download.tarantool.org now
points to the latter, but we keep packagecloud.io for a while to don't
break users, which use it directly (we don't recommend it though). See
[1] for more info about deprecation of our packagecloud.io repositories.

The deployment scripts allows to provide two configurations: production
and staging. The former is for deployment from the master branch and
from a git tag. The latter is for deployments from a developer branch.
It is useful when something should be verified in a specific environment
using a built package or when the deployment process itself was modified
and should be tested.

The difference between production and staging deployment process is how
duplicate package versions are handled. It give an error for production
deployment, but does not for staging. The new package is discarded in
the case for packagecloud.io, but it replaces the old package in S3
repositories.

Read comments in the deployment scripts for more details.

[1]: tarantool/tarantool#4947

Fixes #117
@Totktonada Totktonada force-pushed the Totktonada/php7-v2-deploy-rpm-deb-packages branch from 0d3a227 to 40100b7 Compare June 26, 2020 18:12
@Totktonada
Copy link
Member Author

Implemented automatic fetching of a repositories list to push into for both packagecloud and S3 based repositories.

Possible further features:

  • Automatically determine list of distributions (parse CI script in tarantool repository?)
  • Allow to schedule a periodic job, which will push to new repositories and will build and push for new distributions.

It seems logical to rewrite it from shell to some readable language and move it to packpack tools. I have no time for this now, to be honest.

The initial plan was to create a simple wrapper around packagecloud and update_repo.sh tools, which will deal with problems we aware of (such as this one), but the attempt to handle everything appears to be not so trivial.

Copy link

@avtikhon avtikhon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The patch LGTM, just interesting why didn't you use the travis-ci deploy internal routine.

.travis/deploy_s3.sh Show resolved Hide resolved
@Totktonada
Copy link
Member Author

just interesting why didn't you use the travis-ci deploy internal routine.

There are several reasons, but all of them comes from problems we met with Travis-CI packagecloud deploy script.

  1. We should either duplicate deploy rules or use tricky templates / condition language in order to don't miss deployments from tagged revisions. I found all those ways error-prone and prefer to use shell or any other well known language.
  2. We don't need to duplicate deploy rules for each target repository (2_2, 2_3, etc). This way is flexible enough to even implement fetching of the repositories list dynamically.
  3. packagecloud.io gives 503 http error code from time to time (quite often) when we try to push a package, so some retry strategy should be used to implement reliable deployment process.
  4. I want to skip or rewrite duplicated package for staging repository, don't give an error. Otherwise most of CI jobs on developer branches would be red.

To sum up: this way appears to be much more flexible and allows us to solve several existing problems.

@Totktonada Totktonada merged commit ad221f7 into php7-v2 Jun 29, 2020
@Totktonada Totktonada deleted the Totktonada/php7-v2-deploy-rpm-deb-packages branch June 29, 2020 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants