Skip to content

Update packaging scripts to support all packages on a single branch#62

Merged
JelteF merged 3 commits intodevelopfrom
simplify-branching
Apr 29, 2020
Merged

Update packaging scripts to support all packages on a single branch#62
JelteF merged 3 commits intodevelopfrom
simplify-branching

Conversation

@JelteF
Copy link
Copy Markdown
Contributor

@JelteF JelteF commented Apr 1, 2020

This is needed to support: citusdata/packaging#441


# Based on the repo, update the package related variables
if ( $DISTRO_VERSION eq "redhat" || $DISTRO_VERSION eq "microsoft") {
if ( $DISTRO_VERSION eq "redhat" || $DISTRO_VERSION eq "microsoft" || $DISTRO_VERSION eq "all") {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should document the allowed $DISTRO_VERSION values after we merge this.

We may either introduce some documentation like we have in citus_package

__END__
=head1 NAME
citus_package - easily create OS packages for Citus projects
=head1 SYNOPSIS
B<citus_package> [I<options>] I<project> I<build_type>
=head1 DESCRIPTION
Packages a Citus project for one or more platforms and places the results in
platform-specific directories within the working directory. B<citus_package>
uses Docker under the hood to ensure repeatable builds, so a working Docker
installation is the only prerequisite.
Given a Citus I<project> and I<build_type>, B<citus_package> will build one
package for a single platform, specified using the B<--platform> option. This
option can be provided multiple times in order to build a package for many
platforms at once. In addition, if the special value I<local> is provided as
the project name, B<citus_package> will use packaging files from the current
directory rather than fetching any particular project's files from GitHub.
The I<build_type> argument has two special values: I<release> and I<nightly>. A
release build is based on the latest release version (extracted from the build
files contained within the C<citusdata/packaging> GitHub repository), pulling
code from the corresponding git tag, which must be have a GitHub-verified
signature. A nightly build is based on the latest commit to the "active" branch
for a given project, which is usually C<master>, but can differ by project.
All other I<build_type> values are passed directly to GitHub, which is free to
interpret them how it sees fit, e.g. branch names, tags, or commit identifiers.
B<citus_package> uses the GitHub API to gather information about the project
it is building. As such, a valid C<GITHUB_TOKEN> environment variable must be
set. See the L</ENVIRONMENT> section for details.
=head1 OPTIONS
=over 4
=item B<-p> I<os/release>, B<--platform=>I<os/release>
Platform: required. Provide more than once for multi-platform builds
=back
=head1 ENVIRONMENT
For B<citus_package> to do its job, the C<GITHUB_TOKEN> environment variable
must be populated with a valid GitHub personal access token. It is recommended
that you add a line to your shell profile to ensure this variable is always
correctly set.
To generate a new access token, ensure you're logged into GitHub, then navigate
to your account settings. Choose "Personal access tokens" from the sidebar,
press the "Generate new token" button and name your token (like "packaging").
Ensure the top-level C<repo> and C<user> boxes are checked and press the
"Generate token" button.
B<Do not leave the next page until you've copied your new token!> Paste it
into your e.g. C<.bash_profile> or C<.zshrc> to ensure your shells will have
access to your new token.
B<citus_package> will attempt to sign any Linux packages it builds if the
C<PACKAGING_SECRET_KEY> and C<PACKAGING_PASSPHRASE> environment variables are
set. C<PACKAGING_SECRET_KEY> must be a base64-encoded PGP private key. The
passphrase needed to decrypt this key must be specified in the other variable,
C<PACKAGING_PASSPHRASE>.
If L<gpg(1)> is installed, B<citus_package> will search its keychain for the
Citus Data private key so that only the C<PACKAGING_PASSPHRASE> will be needed
to sign packages.
=head1 COMMON PROJECTS
=over 4
=item I<citus> Citus (Open-Source)
=item I<enterprise> Citus Enterprise
=item I<hll> HyperLogLog
=item I<rebalancer> Shard Rebalancer
=back
=head1 SUPPORTED PLATFORMS
=over 4
=item I<debian/buster> Debian 10 "Buster"
=item I<debian/stretch> Debian 9 "Stretch"
=item I<debian/jessie> Debian 8 "Jessie"
=item I<debian/wheezy> Debian 7 "Wheezy"
=item I<el/8> Enterprise Linux 8.0 (CentOS, RedHat, Amazon Linux)
=item I<el/7> Enterprise Linux 7.0 (CentOS, RedHat, Amazon Linux)
=item I<el/6> Enterprise Linux 6.0 (CentOS, RedHat, Amazon Linux)
=item I<fedora/28> Fedora 28
=item I<fedora/27> Fedora 27
=item I<fedora/26> Fedora 26
=item I<ol/8> Oracle Linux 8.0
=item I<ol/7> Oracle Linux 7.0
=item I<ol/6> Oracle Linux 6.0
=item I<ubuntu/bionic> Ubuntu 18.04 LTS (Bionic Beaver)
=item I<ubuntu/xenial> Ubuntu 16.04 LTS (Xenial Xerus)
=item I<ubuntu/trusty> Ubuntu 14.04 LTS (Trusty Tahr)
=item I<pgxn> PostgreSQL Extension Network (PGXN)
=back
=head1 TODO
Eventually support a different output folder.
=head1 CAVEATS
Projects other than Citus may not use the PGXN platform. When building a PGXN
Citus package, only the I<release> build type may be used.
=head1 SEE ALSO
=over 4
=item L<GitHub Personal Access Tokens|https://github.com/settings/tokens>
=item L<Citus Packaging on GitHub|https://github.com/citusdata/packaging>
=item L<Citus Packaging on Docker Hub|https://hub.docker.com/r/citus/packaging>
=item L<Docker|https://www.docker.com>
=back
=head1 AUTHOR
Jason Petersen L<E<lt>jason@citusdata.comE<gt>>

This idea may be outside the scope of this PR, just sharing my opinion

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or before introducing that, we should at least introduce this "all" parameter to our word doc

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, once we merged all the PRs I think we should update our worddoc to use the "all" parameter.

Copy link
Copy Markdown
Member

@hanefi hanefi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with some minor comment

I am ok with introducing new envvar parameters that will be used in the CI jobs, however we need to document the changes in allowed $DISTRO_VERSION values somewhere. Preferably in Citus Packaging Documentation.

@JelteF JelteF force-pushed the simplify-branching branch from 4f5a2bc to ae4b732 Compare April 29, 2020 07:53
@JelteF JelteF changed the title Update packaging scripts to support Update packaging scripts to support all packages on a single branch Apr 29, 2020
@JelteF JelteF force-pushed the simplify-branching branch from ae4b732 to 8d5c08e Compare April 29, 2020 08:27
@JelteF JelteF force-pushed the simplify-branching branch from 8d5c08e to a20cc5a Compare April 29, 2020 09:15
@onurctirtir onurctirtir self-requested a review April 29, 2020 13:12
@JelteF JelteF merged commit 7efbaea into develop Apr 29, 2020
@JelteF JelteF deleted the simplify-branching branch April 29, 2020 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants