infra: deprecation plan for packagecloud #4947

Totktonada · 2020-04-28T10:38:24Z

https://packagecloud.io/tarantool now provides the following repositories:
1_6, 1_7, 1_9, 1_10, 2x (it is 2.1 in fact), 2_2, 2_3, 2_4.

Our S3 based infrastructure provides: 1.10, 2.1, 2.2, 2.3, 2.4, 2.5.

download.tarantool.org redirects 1.10-2.5 to S3 based repos and other repos to
packagecloud.io (it is 1.6, 1.7 and 1.9 so).

We should not create new repos on packagecloud (for 2.5+) and so it
worth to remove deployment to 2.5 from current master.

The question is what to do with 1.6-2.4 packagecloud repositories.

My proposal:

Remove 2.5 deployment from master branch in tarantool.
Prepare deployment script for Travis-CI, place it to the modulekit repository (luakit / ckit both) (I started to do it within this task).
Move 1_6, 1_7, 1_9 to S3 based repos.
Update all modules, connectors and other packages (say, small) that rely on packagecloud repo (enable it or push to it).
Remove deployment to packagecloud on all branches (1.10-2.4).
Drop packagecloud repositories at all to catch remaining problems.

Are there any objections?

Since pushing to packagecloud is performed from Travis-CI jobs, the issue #4894 is related here.

Follows up #3380.

The text was updated successfully, but these errors were encountered:

Now we have S3 based infrastructure for RPM / Deb packages and GitLab CI pipelines, which deploys packages to it. We don't plan to add 2.5+ repositories on packagecloud.io, so instead of usual change of target bucket from 2_N to 2_(N+1), the deploy stage is removed. Since all distro specific jobs are duplicated in GitLab CI pipelines and those Travis-CI jobs are needed just for deployment, it worth to remove them too. Follows up #3380. Part of #4947.

Now php-tarantool packages are deployed into two kinds of repositories: packagecloud.io and to S3 based ones. The download.tarantool.org now points to the latter, but we keep packagecloud.io for a while to don't break users, which use it directly (we don't recommend it though). See [1] for more info about deprecation of our packagecloud.io repositories. The deployment scripts allows to provide two configurations: production and staging. The former is for deployment from the master branch and from a git tag. The latter is for deployments from a developer branch. It is useful when something should be verified in a specific environment using a built package or when the deployment process itself was modified and should be tested. The difference between production and staging deployment process is how duplicate package versions are handled. It give an error for production deployment, but does not for staging. The new package is discarded in the case for packagecloud.io, but it replaces the old package in S3 repositories. Read comments in the deployment scripts for more details. [1]: tarantool/tarantool#4947 Fixes #117

Now we have S3 based infrastructure for RPM / Deb packages and GitLab CI pipelines, which deploys packages to it. We don't plan to add 2.5+ repositories on packagecloud.io, so instead of usual change of target bucket from 2_N to 2_(N+1), the deploy stage is removed. Since all distro specific jobs are duplicated in GitLab CI pipelines and those Travis-CI jobs are needed just for deployment, it worth to remove them too. Follows up #3380. Part of #4947.

@Totktonada

Removed obvious part in rpm spec for Travis-CI, due to it is no longer in use. ---- Comments from @Totktonada ---- This change is a kind of revertion of the commit d48406d ('test: add more tests to packaging testing'), which did close #4599. Here I described the story, why the change was made and why it is reverted now. We run testing during an RPM package build: it may catch some distribution specific problem. We had reduced quantity of tests and single thread tests execution to keep the testing stable and don't break packages build and deployment due to known fragile tests. Our CI had to use Travis CI, but we were in transition to GitLab CI to use our own machines and don't reach Travis CI limit with five jobs running in parallel. We moved package builds to GitLab CI, but kept build+deploy jobs on Travis CI for a while: GitLab CI was the new for us and we wanted to do this transition smoothly for users of our APT / YUM repositories. After enabling packages building on GitLab CI, we wanted to enable more tests (to catch more problems) and wanted to enable parallel execution of tests to speed up testing (and reduce amount of time a developer wait for results). We observed that if we'll enable more tests and parallel execution on Travis CI, the testing results will become much less stable and so we'll often have holes in deployed packages and red CI. So, we decided to keep the old way testing on Travis CI and perform all changes (more tests, more parallelism) only for GitLab CI. We had a guess that we have enough machine resources and will able to do some load balancing to overcome flaky fails on our own machines, but in fact we picked up another approach later (see below). That's all story behind #4599. What changes from those days? We moved deployment jobs to GitLab CI[^1] and now we completely disabled Travis CI (see #4410 and #4894). All jobs were moved either to GitLab CI or right to GitHub Actions[^2]. We revisited our approach to improve stability of testing. Attemps to do some load balancing together with attempts to keep not-so-large execution time were failed. We should increase parallelism for speed, but decrease it for stability at the same time. There is no optimal balance. So we decided to track flaky fails in the issue tracker and restart a test after a known fail (see details in [1]). This way we don't need to exclude tests and disable parallelism in order to get the stable and fast testing[^3]. At least in theory. We're on the way to verify this guess, but hopefully we'll stick with some adequate defaults that will work everywhere[^4]. To sum up, there are several reasons to remove the old workaround, which was implemented in the scope of #4599: no Travis CI, no foreseeable reasons to exclude tests and reduce parallelism depending on a CI provider. Footnotes: [^1]: This is simplification. Travis CI deployment jobs were not moved as is. GitLab CI jobs push packages to the new repositories backend (#3380). Travis CI jobs were disabled later (as part of #4947), after proofs that the new infrastructure works fine. However this is the another story. [^2]: Now we're going to use GitHub Actions for all jobs, mainly because GitLab CI is poorly integrated with GitHub pull requests (when source branch is in a forked repository). [^3]: Some work toward this direction still to be done: First, 'replication' test suite still excluded from the testing under RPM package build. It seems, we should just enable it back, it is tracked by #4798. Second, there is the issue [2] to get rid of ancient traces of the old attempts to keep the testing stable (from test-run side). It'll give us more parallelism in testing. [^4]: Of course, we perform investigations of flaky fails and fix code and testing problems it feeds to us. However it appears to be the long activity. References: [1]: tarantool/test-run#217 [2]: https://github.com/tarantool/test-run/issues/251

@Totktonada

Removed obvious part in rpm spec for Travis-CI, due to it is no longer in use. ---- Comments from @Totktonada ---- This change is a kind of revertion of the commit d48406d ('test: add more tests to packaging testing'), which did close #4599. Here I described the story, why the change was made and why it is reverted now. We run testing during an RPM package build: it may catch some distribution specific problem. We had reduced quantity of tests and single thread tests execution to keep the testing stable and don't break packages build and deployment due to known fragile tests. Our CI had to use Travis CI, but we were in transition to GitLab CI to use our own machines and don't reach Travis CI limit with five jobs running in parallel. We moved package builds to GitLab CI, but kept build+deploy jobs on Travis CI for a while: GitLab CI was the new for us and we wanted to do this transition smoothly for users of our APT / YUM repositories. After enabling packages building on GitLab CI, we wanted to enable more tests (to catch more problems) and wanted to enable parallel execution of tests to speed up testing (and reduce amount of time a developer wait for results). We observed that if we'll enable more tests and parallel execution on Travis CI, the testing results will become much less stable and so we'll often have holes in deployed packages and red CI. So, we decided to keep the old way testing on Travis CI and perform all changes (more tests, more parallelism) only for GitLab CI. We had a guess that we have enough machine resources and will able to do some load balancing to overcome flaky fails on our own machines, but in fact we picked up another approach later (see below). That's all story behind #4599. What changes from those days? We moved deployment jobs to GitLab CI[^1] and now we completely disabled Travis CI (see #4410 and #4894). All jobs were moved either to GitLab CI or right to GitHub Actions[^2]. We revisited our approach to improve stability of testing. Attemps to do some load balancing together with attempts to keep not-so-large execution time were failed. We should increase parallelism for speed, but decrease it for stability at the same time. There is no optimal balance. So we decided to track flaky fails in the issue tracker and restart a test after a known fail (see details in [1]). This way we don't need to exclude tests and disable parallelism in order to get the stable and fast testing[^3]. At least in theory. We're on the way to verify this guess, but hopefully we'll stick with some adequate defaults that will work everywhere[^4]. To sum up, there are several reasons to remove the old workaround, which was implemented in the scope of #4599: no Travis CI, no foreseeable reasons to exclude tests and reduce parallelism depending on a CI provider. Footnotes: [^1]: This is simplification. Travis CI deployment jobs were not moved as is. GitLab CI jobs push packages to the new repositories backend (#3380). Travis CI jobs were disabled later (as part of #4947), after proofs that the new infrastructure works fine. However this is the another story. [^2]: Now we're going to use GitHub Actions for all jobs, mainly because GitLab CI is poorly integrated with GitHub pull requests (when source branch is in a forked repository). [^3]: Some work toward this direction still to be done: First, 'replication' test suite still excluded from the testing under RPM package build. It seems, we should just enable it back, it is tracked by #4798. Second, there is the issue [2] to get rid of ancient traces of the old attempts to keep the testing stable (from test-run side). It'll give us more parallelism in testing. [^4]: Of course, we perform investigations of flaky fails and fix code and testing problems it feeds to us. However it appears to be the long activity. References: [1]: tarantool/test-run#217 [2]: https://github.com/tarantool/test-run/issues/251 (cherry picked from commit d9c25b7)

@Totktonada

Removed obvious part in rpm spec for Travis-CI, due to it is no longer in use. ---- Comments from @Totktonada ---- This change is a kind of revertion of the commit d48406d ('test: add more tests to packaging testing'), which did close #4599. Here I described the story, why the change was made and why it is reverted now. We run testing during an RPM package build: it may catch some distribution specific problem. We had reduced quantity of tests and single thread tests execution to keep the testing stable and don't break packages build and deployment due to known fragile tests. Our CI had to use Travis CI, but we were in transition to GitLab CI to use our own machines and don't reach Travis CI limit with five jobs running in parallel. We moved package builds to GitLab CI, but kept build+deploy jobs on Travis CI for a while: GitLab CI was the new for us and we wanted to do this transition smoothly for users of our APT / YUM repositories. After enabling packages building on GitLab CI, we wanted to enable more tests (to catch more problems) and wanted to enable parallel execution of tests to speed up testing (and reduce amount of time a developer wait for results). We observed that if we'll enable more tests and parallel execution on Travis CI, the testing results will become much less stable and so we'll often have holes in deployed packages and red CI. So, we decided to keep the old way testing on Travis CI and perform all changes (more tests, more parallelism) only for GitLab CI. We had a guess that we have enough machine resources and will able to do some load balancing to overcome flaky fails on our own machines, but in fact we picked up another approach later (see below). That's all story behind #4599. What changes from those days? We moved deployment jobs to GitLab CI[^1] and now we completely disabled Travis CI (see #4410 and #4894). All jobs were moved either to GitLab CI or right to GitHub Actions[^2]. We revisited our approach to improve stability of testing. Attemps to do some load balancing together with attempts to keep not-so-large execution time were failed. We should increase parallelism for speed, but decrease it for stability at the same time. There is no optimal balance. So we decided to track flaky fails in the issue tracker and restart a test after a known fail (see details in [1]). This way we don't need to exclude tests and disable parallelism in order to get the stable and fast testing[^3]. At least in theory. We're on the way to verify this guess, but hopefully we'll stick with some adequate defaults that will work everywhere[^4]. To sum up, there are several reasons to remove the old workaround, which was implemented in the scope of #4599: no Travis CI, no foreseeable reasons to exclude tests and reduce parallelism depending on a CI provider. Footnotes: [^1]: This is simplification. Travis CI deployment jobs were not moved as is. GitLab CI jobs push packages to the new repositories backend (#3380). Travis CI jobs were disabled later (as part of #4947), after proofs that the new infrastructure works fine. However this is the another story. [^2]: Now we're going to use GitHub Actions for all jobs, mainly because GitLab CI is poorly integrated with GitHub pull requests (when source branch is in a forked repository). [^3]: Some work toward this direction still to be done: First, 'replication' test suite still excluded from the testing under RPM package build. It seems, we should just enable it back, it is tracked by #4798. Second, there is the issue [2] to get rid of ancient traces of the old attempts to keep the testing stable (from test-run side). It'll give us more parallelism in testing. [^4]: Of course, we perform investigations of flaky fails and fix code and testing problems it feeds to us. However it appears to be the long activity. References: [1]: tarantool/test-run#217 [2]: https://github.com/tarantool/test-run/issues/251 (cherry picked from commit d9c25b7)

@Totktonada

Removed obvious part in rpm spec for Travis-CI, due to it is no longer in use. ---- Comments from @Totktonada ---- This change is a kind of revertion of the commit d48406d ('test: add more tests to packaging testing'), which did close #4599. Here I described the story, why the change was made and why it is reverted now. We run testing during an RPM package build: it may catch some distribution specific problem. We had reduced quantity of tests and single thread tests execution to keep the testing stable and don't break packages build and deployment due to known fragile tests. Our CI had to use Travis CI, but we were in transition to GitLab CI to use our own machines and don't reach Travis CI limit with five jobs running in parallel. We moved package builds to GitLab CI, but kept build+deploy jobs on Travis CI for a while: GitLab CI was the new for us and we wanted to do this transition smoothly for users of our APT / YUM repositories. After enabling packages building on GitLab CI, we wanted to enable more tests (to catch more problems) and wanted to enable parallel execution of tests to speed up testing (and reduce amount of time a developer wait for results). We observed that if we'll enable more tests and parallel execution on Travis CI, the testing results will become much less stable and so we'll often have holes in deployed packages and red CI. So, we decided to keep the old way testing on Travis CI and perform all changes (more tests, more parallelism) only for GitLab CI. We had a guess that we have enough machine resources and will able to do some load balancing to overcome flaky fails on our own machines, but in fact we picked up another approach later (see below). That's all story behind #4599. What changes from those days? We moved deployment jobs to GitLab CI[^1] and now we completely disabled Travis CI (see #4410 and #4894). All jobs were moved either to GitLab CI or right to GitHub Actions[^2]. We revisited our approach to improve stability of testing. Attemps to do some load balancing together with attempts to keep not-so-large execution time were failed. We should increase parallelism for speed, but decrease it for stability at the same time. There is no optimal balance. So we decided to track flaky fails in the issue tracker and restart a test after a known fail (see details in [1]). This way we don't need to exclude tests and disable parallelism in order to get the stable and fast testing[^3]. At least in theory. We're on the way to verify this guess, but hopefully we'll stick with some adequate defaults that will work everywhere[^4]. To sum up, there are several reasons to remove the old workaround, which was implemented in the scope of #4599: no Travis CI, no foreseeable reasons to exclude tests and reduce parallelism depending on a CI provider. Footnotes: [^1]: This is simplification. Travis CI deployment jobs were not moved as is. GitLab CI jobs push packages to the new repositories backend (#3380). Travis CI jobs were disabled later (as part of #4947), after proofs that the new infrastructure works fine. However this is the another story. [^2]: Now we're going to use GitHub Actions for all jobs, mainly because GitLab CI is poorly integrated with GitHub pull requests (when source branch is in a forked repository). [^3]: Some work toward this direction still to be done: First, 'replication' test suite still excluded from the testing under RPM package build. It seems, we should just enable it back, it is tracked by #4798. Second, there is the issue [2] to get rid of ancient traces of the old attempts to keep the testing stable (from test-run side). It'll give us more parallelism in testing. [^4]: Of course, we perform investigations of flaky fails and fix code and testing problems it feeds to us. However it appears to be the long activity. References: [1]: tarantool/test-run#217 [2]: https://github.com/tarantool/test-run/issues/251 (cherry picked from commit d9c25b7)

In short: deployment jobs often fail now and we'll recreate them on GitHub Actions in a future. In details: * The old Travis CI infrastructure (.org) will gone soon, the new one (.com) has tight limits on a free plan. * We already have testing on GitHub Actions (but without RPM / Deb uploads). * Pulls from Docker Hub (part of RPM / Deb build + deploy jobs) often ratelimited when performed from Travis CI: likely due to reusing of IP addresses. GitHub Actions is not affected by this problem. * We don't use packagecloud.io for hosting tarantool repositories anymore (see [1], [2], [3]). We'll deploy packages to our new infra in a future: it is tracked by #43. [1]: tarantool/tarantool#3380 [2]: tarantool/tarantool#5494 [3]: tarantool/tarantool#4947 Removed unused Jenkinsfile as well.

Totktonada · 2021-09-21T13:17:48Z

Now @kyukhin disabled payments for https://packagecloud.io/tarantool. Short time after that every attempt to dowload a package starts to say 'Bandwidth or Storage Limit Exceeded'. So it is effectively disabled.

Let's update repositories that depends on it on demand. I'll close this issue.

In short: deployment jobs often fail now and we'll recreate them on GitHub Actions in a future. In details: * The old Travis CI infrastructure (.org) will gone soon, the new one (.com) has tight limits on a free plan. * We already have testing on GitHub Actions (but without RPM / Deb uploads). * Pulls from Docker Hub (part of RPM / Deb build + deploy jobs) often ratelimited when performed from Travis CI: likely due to reusing of IP addresses. GitHub Actions is not affected by this problem. * We don't use packagecloud.io for hosting tarantool repositories anymore (see [1], [2], [3]). We'll deploy packages to our new infra in a future: it is tracked by #43. [1]: tarantool/tarantool#3380 [2]: tarantool/tarantool#5494 [3]: tarantool/tarantool#4947 Removed unused Jenkinsfile as well.

Totktonada · 2021-12-12T05:55:46Z

At 2021-11-30 I removed repositories for tarantool 1.10 and above from https://packagecloud.io/tarantool.

In short: deployment jobs often fail now and we'll recreate them on GitHub Actions in a future. In details: * The old Travis CI infrastructure (.org) will gone soon, the new one (.com) has tight limits on a free plan. * We already have testing on GitHub Actions (but without RPM / Deb uploads). * Pulls from Docker Hub (part of RPM / Deb build + deploy jobs) often ratelimited when performed from Travis CI: likely due to reusing of IP addresses. GitHub Actions is not affected by this problem. * We don't use packagecloud.io for hosting tarantool repositories anymore (see [1], [2], [3]). We'll deploy packages to our new infra in a future: it is tracked by tarantool#43. [1]: tarantool/tarantool#3380 [2]: tarantool/tarantool#5494 [3]: tarantool/tarantool#4947 Removed unused Jenkinsfile as well.

Totktonada added the qa Issues related to tests or testing subsystem label Apr 28, 2020

avtikhon self-assigned this May 6, 2020

Totktonada mentioned this issue Jun 24, 2020

travis-ci: deploy RPM / Deb packages tarantool/tarantool-php#162

Merged

This was referenced Jul 24, 2020

Leave Travis CI #4894

Closed

[1pt] Tagged releases are not uploaded (extra commit is needed) #3745

Closed

avtikhon added this to ON REVIEW in Quality Assurance Jul 31, 2020

avtikhon moved this from ON REVIEW to DOING in Quality Assurance Jul 31, 2020

Totktonada mentioned this issue Dec 26, 2020

need to implement packaging to packagecloud from gitlab to leave travis #4891

Closed

Totktonada mentioned this issue Apr 29, 2021

Make tarantool ecosystem CI / CD great again #6056

Open

57 tasks

Totktonada mentioned this issue May 11, 2021

Resurrect CI / CD and make it nice tarantool/vshard#272

Open

kyukhin assigned kyukhin and unassigned avtikhon Sep 1, 2021

kyukhin added the teamQ label Sep 1, 2021

Totktonada closed this as completed Sep 21, 2021

Totktonada mentioned this issue Oct 4, 2022

Publish packages of every released version #3337

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

infra: deprecation plan for packagecloud #4947

infra: deprecation plan for packagecloud #4947

Totktonada commented Apr 28, 2020

Totktonada commented Sep 21, 2021

Totktonada commented Dec 12, 2021