Deploy package repos on MRG's S3 infra #3380

kyukhin · 2018-04-27T05:43:41Z

The deployment should support mirroring.

Totktonada · 2019-09-03T14:58:20Z

The deployment should support mirroring.

I don't undestood what exactly you meant.

Motivation: We want to solve problems we have currently with packagecloud repositories. More precisely, there is the one biggest problem: we have a lot of packages (tarantool itself / modules / connectors and so on × multiple supported distros × multiple versions) and so we call to prune old package versions to keep a payment within some limit.

Pruning of old packages (even despite that it saves all last versions) makes the repositories unusable for many cases, because one is unable to hold a version of packages w/o saving them locally somewhere. I know projects (including internal ones!) with RPMs within a project repository; I also saw customers' mirrors of our packagecloud repository — users try to workaround the problem. I also saw questions how to install a specific version to, say, bisect some problem.

We have MRG infrastucture that provides S3 compatible storage that abstract us out of most problems with supporting a reliable service.

We have nice mkrepo tool that generates and updates yum and apt repositories. It support S3 compatible storages.

Our documentation shows download.tarantool.org URLs that can be switched to another backend almost seamlessly. The only problem here is that packagecloud does not give us private repository keys, so we need to either install our new keys within tarantool package for some time period (and only then switch) or leave instructions how to update keys manually after we'll switch download.tarantool.org repos (see #3736).

We discussed a bit how to achieve the goal with @avtikhon. He looking for other tools instead of mkrepo, but I don't see any reason to do so. Hope he'll clarify his points here. I'll show mine points below as answers for questions that were arisen around this task in near days.

From my side, strengths of mkrepo:

Simple and our own (easy to support).
Support both apt and yum.
Support S3 natively as storage (doesn't require a full local copy).

Pros of creating our own repositories instead of mirroring packagecloud ones:

Simpler infrastructure.
Less fault points in a deployment process.
We don't depend on timings when packagecloud will enable support of new distros (they however make this very fast when it is requested explicitly).

I see several cons of proposed using of reprepro (and see no pros):

First of all: no (working) project website, no public repository.
- I saw some mirros on github like this which does not look as official nor as supported.
- Hey, a lot of C code; looks much more complex that our pythonic mkrepo.
Debian specific (no yum repos support); more on that:
- What with yum repos? Some other tool is needed?
Requires to have a local mirror; more on that:
- A cumulative size of repos will growth in linear way (now it is constant because of pruning).
- We should support a (more or less) reliable storage for that mirror; that is what we want to avoid by using MRG's S3.
We don't control behaviour of that tool; more on that:
- Say, I saw comments that the tool allows to handle only one version of a package. If it is so, then it just contradicts to our goals.
- I saw comments that it does not report an error in the case when you are trying to push a version of a package that is older then an existing one.

Alexander (@avtikhon) also met several small problems with mkrepo. We'll resolve them; it should not affect our decision. If there is suspect that mkrepo is not enough qualitative, then it should be said explicitly. At the first glance it looks much more maintainable then reprepro and it seems that it fit our needs much more better.

As Gentoo user I wondered that we need to separate repositories per <major>.<minor> tarantool versions (say, 1.10, 2.1, 2.2, 2.3). As I understood, this is workaround for two problems:

A maintainer is unable to mark different versions of a package as stable / unstable within one repository.
A user is unable to mask a specific version range to, say, install tarantool of 2.2 series, but receive updates for it.

Brief googling does not give solutions for those problems for apt-get and yum. It seems that package managers in most popular distros are not very flexible.

So we need to provide separate repositories anyway.

avtikhon · 2019-09-03T15:32:51Z

'reprepro' tool is the officially suggested tool by Debian:
https://wiki.debian.org/DebianRepository/Setup#reprepro

reprepro
(formerly known as mirrorer)
Goals: Local Debian package repository storing files in a pool/ directory.
Pros:
Strict checking of what comes in, no database server needed.
Cons:
Lots of different options and commands, difficult to use for beginners.
Package: reprepro

Distributions: stable, testing, unstable and squeeze-backports

Dependencies: unstable/utils/reprepro

Automatic repositories: Yes
Incoming mechanism: Yes
Pools: Yes
GPG signing: Yes
Inclusion of .deb without .changes: Yes
Several versions of each package: not in the same Packages instance
Manual: Yes

HOWTOs:
Short HOWTO (official)

Setting up signed Debian repository with reprepro

Setting up your own automatic Debian repository see this article on reprepro. The link is rather outdated, but still contains some useful information.

Setting up reprepro with apache2 and sign key (in italian language) http://www.fabreg.it/reprepro-repository-debian-43/

Creating your own Signed APT Repository and Debian Packages by Jon Cowie

Positive:

It means that we don't need to follow the changes in Debian packages formats.
No need to support the local tool 'mkrepo'.

Other:

Yes, we need separate repositories for YUM and DEB and it is right way.
Local repositories won't take much space, also it will provide the local backup and ability to
check/work with local copy.

mkrepo - "Supports S3 natively as storage (doesn't require a full local copy)."

It means that we have some tool that supports unknown way to use S3, while there is the AWS
official tool 's3fs' for mounting the S3 buckets, which is already known as buggy - does it mean that
mkrepo is better ?

Totktonada · 2019-09-03T16:52:44Z

So, the options are:

Use the simple (and controlled by us) mkrepo tool that do exactly what we want and support everything what we want.
Glue three different tools that that doing something more or less what we want, each more complex then mkrepo, non-controlled by us, require extra infra (local mirror) — so it is hard to run just, say, on Travis-CI.

(Note also the reprepro just doesn't fit if it supports storing only a last versions of a package — I already wrote about that above.)

For me the choose is obvious. However this is not my task, so either do it as you want (but don't ask me for review) or ask Kirill (@kyukhin) to decide.

Totktonada · 2019-09-03T18:45:26Z

Mounting S3 storage via FUSE eliminates the need of a local mirror. Other points remain valid.

avtikhon · 2019-09-04T07:49:22Z

Actually found that s3fs really works and mounts the external S3 to the local path, the only issue found that it fails with path names including dots, which produces the 404 "not found" respond from S3.

Added ability to store packages additionaly at MCS S3. Closes #3380

Added ability to store packages additionaly at MCS S3. The target idea was to add the new way of packages creation at MCS S3, which temporary duplicates the packaging at PackageCloud by Packpack tool. Also it was needed to pack the binaries in the native way of the each packing OS style. It were created two separate scripts, each of it for different packing style: - DEB for Ubuntu and Debian, script tools/pub_packs_s3_deb.sh - RPM for CentOS and Fedora, script tools/pub_packs_s3_rpm.sh Common parts of the scripts are: - create new meta files for the new binaries - copy new binaries to MCS S3 - get previous metafiles from MCS S3 and merge the new meta data for the new binaries - update the meta files at MCS S3 Different parts: - DEB script based on reprepro external tool which needs the prepared path with needed file structure - .gitlab.mk file updated for it, also it works separately only on OS versions level - it means that meta data it updates for all Distritbutions together. - RPM script based on createrepo external tool which doesn't need prepared path with file structure and works with the given path with binaries, also it works separately for OS/Distribution level - it means that meta data it updates for all Distributions separately. Closes #3380

Added ability to store packages additionaly at MCS S3. The target idea was to add the new way of packages creation at MCS S3, which temporary duplicates the packaging at PackageCloud by Packpack tool. Also it was needed to pack the binaries in the native way of the each packing OS style. It was created standalone script for adding the packages binaries/sources to MCS S3 at DEB either RPM repositories: 'tools/add_pack_s3_repo.sh' Common parts of the script are: - create new meta files for the new binaries - copy new binaries to MCS S3 - get previous metafiles from MCS S3 and merge the new meta data for the new binaries - update the meta files at MCS S3 Different parts: - DEB script part based on reprepro external tool, also it works separately only on OS versions level - it means that meta data it updates for all Distritbutions together. - RPM script part based on createrepo external tool, also it works separately for OS/Release level - it means that meta data it updates for all Releases separately. Closes #3380

Created Dockerfile based script to check packages available at the S3 repository. Set this script to be run just after the package creation. Follows up #3380

Found that modules may have only binaries packages w/o sources packages. Script changed to be able to work with only binaries either sources packages. Follow-up #3380

Added cleanup functionality for the meta files. Script may have the following situations: - package files removed at S3, but it still registered: Script stores and registers the new packages at S3 and removes all the other registered blocks for the sames files in meta files. - package files already exists at S3 with the same hashes: Script passes it with warning message. - package files already exists at S3 with the old hashes: Script fails w/o force flag, otherwise it stores and registers the new packages at S3 and removes all the other registered blocks for the sames files in meta files. Added '-s|skip_errors' option flag to skip errors on changed packages to avoid of exits on script run. Follow-up #3380

Found that modules may have only binaries packages w/o sources packages. Script changed to be able to work with only binaries either sources packages. Follow-up #3380

Added instructions on 'product' option with examples. Follow-up #3380

Created Dockerfile based script to check packages available at the S3 repository. Set this script to be run just after the package creation. Follows up #3380

Now we have S3 based infrastructure for RPM / Deb packages and GitLab CI pipelines, which deploys packages to it. We don't plan to add 2.5+ repositories on packagecloud.io, so instead of usual change of target bucket from 2_N to 2_(N+1), the deploy stage is removed. Since all distro specific jobs are duplicated in GitLab CI pipelines and those Travis-CI jobs are needed just for deployment, it worth to remove them too. Follows up #3380. Part of #4947.

@Totktonada

Removed obvious part in rpm spec for Travis-CI, due to it is no longer in use. ---- Comments from @Totktonada ---- This change is a kind of revertion of the commit d48406d ('test: add more tests to packaging testing'), which did close #4599. Here I described the story, why the change was made and why it is reverted now. We run testing during an RPM package build: it may catch some distribution specific problem. We had reduced quantity of tests and single thread tests execution to keep the testing stable and don't break packages build and deployment due to known fragile tests. Our CI had to use Travis CI, but we were in transition to GitLab CI to use our own machines and don't reach Travis CI limit with five jobs running in parallel. We moved package builds to GitLab CI, but kept build+deploy jobs on Travis CI for a while: GitLab CI was the new for us and we wanted to do this transition smoothly for users of our APT / YUM repositories. After enabling packages building on GitLab CI, we wanted to enable more tests (to catch more problems) and wanted to enable parallel execution of tests to speed up testing (and reduce amount of time a developer wait for results). We observed that if we'll enable more tests and parallel execution on Travis CI, the testing results will become much less stable and so we'll often have holes in deployed packages and red CI. So, we decided to keep the old way testing on Travis CI and perform all changes (more tests, more parallelism) only for GitLab CI. We had a guess that we have enough machine resources and will able to do some load balancing to overcome flaky fails on our own machines, but in fact we picked up another approach later (see below). That's all story behind #4599. What changes from those days? We moved deployment jobs to GitLab CI[^1] and now we completely disabled Travis CI (see #4410 and #4894). All jobs were moved either to GitLab CI or right to GitHub Actions[^2]. We revisited our approach to improve stability of testing. Attemps to do some load balancing together with attempts to keep not-so-large execution time were failed. We should increase parallelism for speed, but decrease it for stability at the same time. There is no optimal balance. So we decided to track flaky fails in the issue tracker and restart a test after a known fail (see details in [1]). This way we don't need to exclude tests and disable parallelism in order to get the stable and fast testing[^3]. At least in theory. We're on the way to verify this guess, but hopefully we'll stick with some adequate defaults that will work everywhere[^4]. To sum up, there are several reasons to remove the old workaround, which was implemented in the scope of #4599: no Travis CI, no foreseeable reasons to exclude tests and reduce parallelism depending on a CI provider. Footnotes: [^1]: This is simplification. Travis CI deployment jobs were not moved as is. GitLab CI jobs push packages to the new repositories backend (#3380). Travis CI jobs were disabled later (as part of #4947), after proofs that the new infrastructure works fine. However this is the another story. [^2]: Now we're going to use GitHub Actions for all jobs, mainly because GitLab CI is poorly integrated with GitHub pull requests (when source branch is in a forked repository). [^3]: Some work toward this direction still to be done: First, 'replication' test suite still excluded from the testing under RPM package build. It seems, we should just enable it back, it is tracked by #4798. Second, there is the issue [2] to get rid of ancient traces of the old attempts to keep the testing stable (from test-run side). It'll give us more parallelism in testing. [^4]: Of course, we perform investigations of flaky fails and fix code and testing problems it feeds to us. However it appears to be the long activity. References: [1]: tarantool/test-run#217 [2]: https://github.com/tarantool/test-run/issues/251

@Totktonada

Removed obvious part in rpm spec for Travis-CI, due to it is no longer in use. ---- Comments from @Totktonada ---- This change is a kind of revertion of the commit d48406d ('test: add more tests to packaging testing'), which did close #4599. Here I described the story, why the change was made and why it is reverted now. We run testing during an RPM package build: it may catch some distribution specific problem. We had reduced quantity of tests and single thread tests execution to keep the testing stable and don't break packages build and deployment due to known fragile tests. Our CI had to use Travis CI, but we were in transition to GitLab CI to use our own machines and don't reach Travis CI limit with five jobs running in parallel. We moved package builds to GitLab CI, but kept build+deploy jobs on Travis CI for a while: GitLab CI was the new for us and we wanted to do this transition smoothly for users of our APT / YUM repositories. After enabling packages building on GitLab CI, we wanted to enable more tests (to catch more problems) and wanted to enable parallel execution of tests to speed up testing (and reduce amount of time a developer wait for results). We observed that if we'll enable more tests and parallel execution on Travis CI, the testing results will become much less stable and so we'll often have holes in deployed packages and red CI. So, we decided to keep the old way testing on Travis CI and perform all changes (more tests, more parallelism) only for GitLab CI. We had a guess that we have enough machine resources and will able to do some load balancing to overcome flaky fails on our own machines, but in fact we picked up another approach later (see below). That's all story behind #4599. What changes from those days? We moved deployment jobs to GitLab CI[^1] and now we completely disabled Travis CI (see #4410 and #4894). All jobs were moved either to GitLab CI or right to GitHub Actions[^2]. We revisited our approach to improve stability of testing. Attemps to do some load balancing together with attempts to keep not-so-large execution time were failed. We should increase parallelism for speed, but decrease it for stability at the same time. There is no optimal balance. So we decided to track flaky fails in the issue tracker and restart a test after a known fail (see details in [1]). This way we don't need to exclude tests and disable parallelism in order to get the stable and fast testing[^3]. At least in theory. We're on the way to verify this guess, but hopefully we'll stick with some adequate defaults that will work everywhere[^4]. To sum up, there are several reasons to remove the old workaround, which was implemented in the scope of #4599: no Travis CI, no foreseeable reasons to exclude tests and reduce parallelism depending on a CI provider. Footnotes: [^1]: This is simplification. Travis CI deployment jobs were not moved as is. GitLab CI jobs push packages to the new repositories backend (#3380). Travis CI jobs were disabled later (as part of #4947), after proofs that the new infrastructure works fine. However this is the another story. [^2]: Now we're going to use GitHub Actions for all jobs, mainly because GitLab CI is poorly integrated with GitHub pull requests (when source branch is in a forked repository). [^3]: Some work toward this direction still to be done: First, 'replication' test suite still excluded from the testing under RPM package build. It seems, we should just enable it back, it is tracked by #4798. Second, there is the issue [2] to get rid of ancient traces of the old attempts to keep the testing stable (from test-run side). It'll give us more parallelism in testing. [^4]: Of course, we perform investigations of flaky fails and fix code and testing problems it feeds to us. However it appears to be the long activity. References: [1]: tarantool/test-run#217 [2]: https://github.com/tarantool/test-run/issues/251 (cherry picked from commit d9c25b7)

@Totktonada

Removed obvious part in rpm spec for Travis-CI, due to it is no longer in use. ---- Comments from @Totktonada ---- This change is a kind of revertion of the commit d48406d ('test: add more tests to packaging testing'), which did close #4599. Here I described the story, why the change was made and why it is reverted now. We run testing during an RPM package build: it may catch some distribution specific problem. We had reduced quantity of tests and single thread tests execution to keep the testing stable and don't break packages build and deployment due to known fragile tests. Our CI had to use Travis CI, but we were in transition to GitLab CI to use our own machines and don't reach Travis CI limit with five jobs running in parallel. We moved package builds to GitLab CI, but kept build+deploy jobs on Travis CI for a while: GitLab CI was the new for us and we wanted to do this transition smoothly for users of our APT / YUM repositories. After enabling packages building on GitLab CI, we wanted to enable more tests (to catch more problems) and wanted to enable parallel execution of tests to speed up testing (and reduce amount of time a developer wait for results). We observed that if we'll enable more tests and parallel execution on Travis CI, the testing results will become much less stable and so we'll often have holes in deployed packages and red CI. So, we decided to keep the old way testing on Travis CI and perform all changes (more tests, more parallelism) only for GitLab CI. We had a guess that we have enough machine resources and will able to do some load balancing to overcome flaky fails on our own machines, but in fact we picked up another approach later (see below). That's all story behind #4599. What changes from those days? We moved deployment jobs to GitLab CI[^1] and now we completely disabled Travis CI (see #4410 and #4894). All jobs were moved either to GitLab CI or right to GitHub Actions[^2]. We revisited our approach to improve stability of testing. Attemps to do some load balancing together with attempts to keep not-so-large execution time were failed. We should increase parallelism for speed, but decrease it for stability at the same time. There is no optimal balance. So we decided to track flaky fails in the issue tracker and restart a test after a known fail (see details in [1]). This way we don't need to exclude tests and disable parallelism in order to get the stable and fast testing[^3]. At least in theory. We're on the way to verify this guess, but hopefully we'll stick with some adequate defaults that will work everywhere[^4]. To sum up, there are several reasons to remove the old workaround, which was implemented in the scope of #4599: no Travis CI, no foreseeable reasons to exclude tests and reduce parallelism depending on a CI provider. Footnotes: [^1]: This is simplification. Travis CI deployment jobs were not moved as is. GitLab CI jobs push packages to the new repositories backend (#3380). Travis CI jobs were disabled later (as part of #4947), after proofs that the new infrastructure works fine. However this is the another story. [^2]: Now we're going to use GitHub Actions for all jobs, mainly because GitLab CI is poorly integrated with GitHub pull requests (when source branch is in a forked repository). [^3]: Some work toward this direction still to be done: First, 'replication' test suite still excluded from the testing under RPM package build. It seems, we should just enable it back, it is tracked by #4798. Second, there is the issue [2] to get rid of ancient traces of the old attempts to keep the testing stable (from test-run side). It'll give us more parallelism in testing. [^4]: Of course, we perform investigations of flaky fails and fix code and testing problems it feeds to us. However it appears to be the long activity. References: [1]: tarantool/test-run#217 [2]: https://github.com/tarantool/test-run/issues/251 (cherry picked from commit d9c25b7)

@Totktonada

Removed obvious part in rpm spec for Travis-CI, due to it is no longer in use. ---- Comments from @Totktonada ---- This change is a kind of revertion of the commit d48406d ('test: add more tests to packaging testing'), which did close #4599. Here I described the story, why the change was made and why it is reverted now. We run testing during an RPM package build: it may catch some distribution specific problem. We had reduced quantity of tests and single thread tests execution to keep the testing stable and don't break packages build and deployment due to known fragile tests. Our CI had to use Travis CI, but we were in transition to GitLab CI to use our own machines and don't reach Travis CI limit with five jobs running in parallel. We moved package builds to GitLab CI, but kept build+deploy jobs on Travis CI for a while: GitLab CI was the new for us and we wanted to do this transition smoothly for users of our APT / YUM repositories. After enabling packages building on GitLab CI, we wanted to enable more tests (to catch more problems) and wanted to enable parallel execution of tests to speed up testing (and reduce amount of time a developer wait for results). We observed that if we'll enable more tests and parallel execution on Travis CI, the testing results will become much less stable and so we'll often have holes in deployed packages and red CI. So, we decided to keep the old way testing on Travis CI and perform all changes (more tests, more parallelism) only for GitLab CI. We had a guess that we have enough machine resources and will able to do some load balancing to overcome flaky fails on our own machines, but in fact we picked up another approach later (see below). That's all story behind #4599. What changes from those days? We moved deployment jobs to GitLab CI[^1] and now we completely disabled Travis CI (see #4410 and #4894). All jobs were moved either to GitLab CI or right to GitHub Actions[^2]. We revisited our approach to improve stability of testing. Attemps to do some load balancing together with attempts to keep not-so-large execution time were failed. We should increase parallelism for speed, but decrease it for stability at the same time. There is no optimal balance. So we decided to track flaky fails in the issue tracker and restart a test after a known fail (see details in [1]). This way we don't need to exclude tests and disable parallelism in order to get the stable and fast testing[^3]. At least in theory. We're on the way to verify this guess, but hopefully we'll stick with some adequate defaults that will work everywhere[^4]. To sum up, there are several reasons to remove the old workaround, which was implemented in the scope of #4599: no Travis CI, no foreseeable reasons to exclude tests and reduce parallelism depending on a CI provider. Footnotes: [^1]: This is simplification. Travis CI deployment jobs were not moved as is. GitLab CI jobs push packages to the new repositories backend (#3380). Travis CI jobs were disabled later (as part of #4947), after proofs that the new infrastructure works fine. However this is the another story. [^2]: Now we're going to use GitHub Actions for all jobs, mainly because GitLab CI is poorly integrated with GitHub pull requests (when source branch is in a forked repository). [^3]: Some work toward this direction still to be done: First, 'replication' test suite still excluded from the testing under RPM package build. It seems, we should just enable it back, it is tracked by #4798. Second, there is the issue [2] to get rid of ancient traces of the old attempts to keep the testing stable (from test-run side). It'll give us more parallelism in testing. [^4]: Of course, we perform investigations of flaky fails and fix code and testing problems it feeds to us. However it appears to be the long activity. References: [1]: tarantool/test-run#217 [2]: https://github.com/tarantool/test-run/issues/251 (cherry picked from commit d9c25b7)

In short: deployment jobs often fail now and we'll recreate them on GitHub Actions in a future. In details: * The old Travis CI infrastructure (.org) will gone soon, the new one (.com) has tight limits on a free plan. * We already have testing on GitHub Actions (but without RPM / Deb uploads). * Pulls from Docker Hub (part of RPM / Deb build + deploy jobs) often ratelimited when performed from Travis CI: likely due to reusing of IP addresses. GitHub Actions is not affected by this problem. * We don't use packagecloud.io for hosting tarantool repositories anymore (see [1], [2], [3]). We'll deploy packages to our new infra in a future: it is tracked by #43. [1]: tarantool/tarantool#3380 [2]: tarantool/tarantool#5494 [3]: tarantool/tarantool#4947 Removed unused Jenkinsfile as well.

In short: deployment jobs often fail now and we'll recreate them on GitHub Actions in a future. In details: * The old Travis CI infrastructure (.org) will gone soon, the new one (.com) has tight limits on a free plan. * We already have testing on GitHub Actions (but without RPM / Deb uploads). * Pulls from Docker Hub (part of RPM / Deb build + deploy jobs) often ratelimited when performed from Travis CI: likely due to reusing of IP addresses. GitHub Actions is not affected by this problem. * We don't use packagecloud.io for hosting tarantool repositories anymore (see [1], [2], [3]). We'll deploy packages to our new infra in a future: it is tracked by tarantool#43. [1]: tarantool/tarantool#3380 [2]: tarantool/tarantool#5494 [3]: tarantool/tarantool#4947 Removed unused Jenkinsfile as well.

kyukhin added the qa Issues related to tests or testing subsystem label Apr 27, 2018

kyukhin added this to the wishlist milestone Apr 27, 2018

kyukhin assigned LaserPhaser Apr 27, 2018

Totktonada unassigned LaserPhaser Sep 2, 2019

avtikhon added a commit that referenced this issue Oct 3, 2019

gitlab-ci: implement packing into MCS S3

300cb49

Added ability to store packages additionaly at MCS S3. Closes #3380

avtikhon added a commit that referenced this issue Oct 3, 2019

gitlab-ci: implement packing into MCS S3

d14e555

Added ability to store packages additionaly at MCS S3. Closes #3380

avtikhon added a commit that referenced this issue Oct 4, 2019

gitlab-ci: implement packing into MCS S3

d576322

Added ability to store packages additionaly at MCS S3. Closes #3380

avtikhon added a commit that referenced this issue Oct 4, 2019

gitlab-ci: implement packing into MCS S3

2bc734c

Added ability to store packages additionaly at MCS S3. Closes #3380

avtikhon added a commit that referenced this issue Oct 4, 2019

gitlab-ci: implement packing into MCS S3

669b0e7

Added ability to store packages additionaly at MCS S3. Closes #3380

avtikhon added a commit that referenced this issue Mar 23, 2020

Add S3 package checker

2fc1049

Created Dockerfile based script to check packages available at the S3 repository. Set this script to be run just after the package creation. Follows up #3380

avtikhon added a commit that referenced this issue Mar 27, 2020

Enable script for saving packages in S3 for modules

8df61be

Found that modules may have only binaries packages w/o sources packages. Script changed to be able to work with only binaries either sources packages. Follow-up #3380

avtikhon added a commit that referenced this issue Mar 27, 2020

Enable script for saving packages in S3 for modules

0c91056

Found that modules may have only binaries packages w/o sources packages. Script changed to be able to work with only binaries either sources packages. Follow-up #3380

avtikhon added a commit that referenced this issue Mar 27, 2020

Enable script for saving packages in S3 for modules

a17b8dc

Found that modules may have only binaries packages w/o sources packages. Script changed to be able to work with only binaries either sources packages. Follow-up #3380

avtikhon added a commit that referenced this issue Mar 29, 2020

Add help instruction on 'product' option

4738747

Added instructions on 'product' option with examples. Follow-up #3380

avtikhon added a commit that referenced this issue Apr 11, 2020

Add S3 package checker

023f327

Created Dockerfile based script to check packages available at the S3 repository. Set this script to be run just after the package creation. Follows up #3380

avtikhon added a commit that referenced this issue Apr 11, 2020

Add S3 package checker

a3480ae

Created Dockerfile based script to check packages available at the S3 repository. Set this script to be run just after the package creation. Follows up #3380

avtikhon added a commit that referenced this issue Apr 11, 2020

Add S3 package checker

b5c8eb3

Created Dockerfile based script to check packages available at the S3 repository. Set this script to be run just after the package creation. Follows up #3380

avtikhon added a commit that referenced this issue Apr 11, 2020

Add S3 package checker

32f7637

Created Dockerfile based script to check packages available at the S3 repository. Set this script to be run just after the package creation. Follows up #3380

avtikhon added a commit that referenced this issue Apr 11, 2020

Add S3 package checker

1aa1d82

Created Dockerfile based script to check packages available at the S3 repository. Set this script to be run just after the package creation. Follows up #3380

kyukhin removed this from the wishlist milestone Apr 21, 2020

Totktonada mentioned this issue Apr 28, 2020

infra: deprecation plan for packagecloud #4947

Closed

Totktonada mentioned this issue Dec 26, 2020

need to implement packaging to packagecloud from gitlab to leave travis #4891

Closed

This was referenced Feb 13, 2021

MVP tarantool/rws#1

Merged

modules: RPM / Deb deploy infrastructure for S3 based repositories #5494

Closed

Totktonada mentioned this issue Apr 29, 2021

Make tarantool ecosystem CI / CD great again #6056

Open

57 tasks

Totktonada mentioned this issue May 11, 2021

Resurrect CI / CD and make it nice tarantool/vshard#272

Open

Totktonada mentioned this issue Oct 4, 2022

Publish packages of every released version #3337

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deploy package repos on MRG's S3 infra #3380

Deploy package repos on MRG's S3 infra #3380

kyukhin commented Apr 27, 2018

Totktonada commented Sep 3, 2019

avtikhon commented Sep 3, 2019 •

edited

Loading

Totktonada commented Sep 3, 2019

Totktonada commented Sep 3, 2019

avtikhon commented Sep 4, 2019 •

edited

Loading

Deploy package repos on MRG's S3 infra #3380

Deploy package repos on MRG's S3 infra #3380

Comments

kyukhin commented Apr 27, 2018

Totktonada commented Sep 3, 2019

avtikhon commented Sep 3, 2019 • edited Loading

Totktonada commented Sep 3, 2019

Totktonada commented Sep 3, 2019

avtikhon commented Sep 4, 2019 • edited Loading

avtikhon commented Sep 3, 2019 •

edited

Loading

avtikhon commented Sep 4, 2019 •

edited

Loading