Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: Travis -> GitHub Actions; Create Release Binaries and Container Images #444

Merged
merged 27 commits into from
Mar 12, 2024
Merged

CI: Travis -> GitHub Actions; Create Release Binaries and Container Images #444

merged 27 commits into from
Mar 12, 2024

Conversation

sheurich
Copy link
Contributor

@sheurich sheurich commented Feb 26, 2024 •
edited

Checks
Tests

As a follow-on to #439 this change request implements the first step of building a new CI system for Pebble. As this is a testbed environment for Boulder, some of these changes may prove to be useful in the latter context.

Per #434 (comment), GitHub Actions is a reasonable choice for a new CI implementation to supersede Travis. Much, but not all, of the existing functionality is present in this initial set of changes.

Items currently implemented include:

  • Binary production of pebble and challtestsrv for platforms:
    • AMD64
      • darwin
      • linux
      • windows
    • ARM64
      • darwin
      • linux
      • windows
  • Multiplatform docker images published on the GitHub Container Registry for platforms:
    • linux/amd64
    • linux/arm64
    • windows/amd64
  • Golang linting.
  • Golang coverage.
  • Local test scripts for Go and Docker:
    • ./build.sh
    • ./docker.sh
    • ./test.sh

This PR also adds a -version flag to Pebble, which is set during release build to the Git commit ref.

This change builds on earlier work done in #379 and #442.

@sheurich sheurich changed the title Ci overhaul CI Overhaul Feb 26, 2024
@ldez
Copy link
Contributor

ldez commented Feb 26, 2024 •
edited

I think we can merge our 2 PRs #442
My PR is the basics, and your PR is the release part.

ldez
ldez reviewed Feb 26, 2024
View reviewed changes
.github/workflows/ci.yml Outdated Show resolved Hide resolved
@sheurich
Copy link
Contributor Author

I think we can merge our 2 PRs #442

@ldez Yes let's do that. I can take a look tomorrow.

@ldez
Copy link
Contributor

ldez commented Feb 26, 2024 •
edited

I mean I think my PR can be merged and your PR will just need to be "cleaned" to remove the duplication with mine.
By "merge" I meant "complementary", I know it was not clear.

@sheurich
Copy link
Contributor Author

@ldez glad you found #440! I spent some time troubleshooting at the CI level but hadn't checked Pebble itself yet.

@sheurich sheurich force-pushed the ci-overhaul branch 2 times, most recently from 60b576e to 5812b5f Compare February 27, 2024 16:26
ldez
ldez reviewed Feb 27, 2024
View reviewed changes
.github/workflows/ci.yml Outdated Show resolved Hide resolved
ldez
ldez reviewed Feb 27, 2024
View reviewed changes
.github/workflows/ci.yml Outdated Show resolved Hide resolved
ldez
ldez reviewed Feb 27, 2024
View reviewed changes
.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/checks.yml Outdated Show resolved Hide resolved
.github/workflows/checks.yml Show resolved Hide resolved
@mcpherrinm mcpherrinm mentioned this pull request Feb 28, 2024
Closed
@wgreenberg wgreenberg mentioned this pull request Feb 28, 2024
Closed
@sheurich sheurich force-pushed the ci-overhaul branch 3 times, most recently from 2605a05 to 6f2f2c5 Compare February 28, 2024 23:16
@sheurich
Copy link
Contributor Author

@ldez I think all of the changes have been made and would appreciate another review. The PR description hasn't yet been updated to match the actual changes.

ldez
ldez reviewed Feb 28, 2024
View reviewed changes
.github/workflows/build.yml Outdated Show resolved Hide resolved
@sheurich sheurich force-pushed the ci-overhaul branch 2 times, most recently from 0f8556c to 8de4178 Compare February 29, 2024 16:21
@sheurich
Copy link
Contributor Author

This will be ready for review soon, pending a bug fix.

@sheurich
Copy link
Contributor Author

I think this should be ready for review @mcpherrinm

@mcpherrinm
Copy link
Contributor

It doesn’t have to be in this PR, but we are generally standardizing on either distroless or debian-slim as the base for docker images instead of alpine. We can make that change later, or it would be helpful if you want to make it here while we’re changing the docker files.

I’ll review more fully later today

@sheurich
Copy link
Contributor Author

sheurich commented Mar 1, 2024 •
edited

It doesn’t have to be in this PR, but we are generally standardizing on either distroless or debian-slim as the base for docker images instead of alpine.

I had assumed that Alpine was chosen over Debian as a derivative of the strategy outlined in the Pebble README:

In places where the ACME specification allows customization/CA choice Pebble aims to make choices different from Boulder.

It's no problem to change to Distroless for release. I will add that update to this PR.

@sheurich
Copy link
Contributor Author

sheurich commented Mar 1, 2024

Since cgo is not needed by pebble or pebble-challtestsrv, the release image can be FROM scratch. This WFM but are there other requirements which need glibc or other items from distroless?

@mcpherrinm
Copy link
Contributor

Scratch is fine by me

@sheurich
Copy link
Contributor Author

sheurich commented Mar 2, 2024

@mcpherrinm the scratch change is implemented and a few other bugs have been squashed.

@sheurich sheurich changed the title CI Overhaul CI: Travis -> GitHub Actions; Create Release Binaries and Container Images Mar 2, 2024
@sheurich sheurich mentioned this pull request Mar 3, 2024
Merged
@sheurich
add local build script
359ddb1
@sheurich
add local docker build/deploy/test script
cb9faa0
@sheurich
add pebble -version flag; smoketest in release; add local build.sh …
d9f9aaf 
…and docker.sh
@sheurich
fix: set version ldflags from github.sha in release workflow
cd03dec
@sheurich
release fixes
c6b572a
@sheurich
use ghcr.io registry in compose
d157204
@sheurich
fix base docker tag
8154447
@sheurich
fix: add test configuration files to docker image
60c58d4
@sheurich
cleanup unused dockerfiles
8fc5cbf
@sheurich
add eggsampler/acme-based integration test
68307f6
@sheurich
cleanup docker.sh GOCACHE
2bf1264
@sheurich
simplify local docker testing
57e58e0
@sheurich
use scratch for linux container base image
2af6cb1
@sheurich
update release version string
556b878
@sheurich
fix: remove binary from compose command
3008fdd 
fix: run release workflow on release publish

fix: run eggsampler/acme tests with path needed for TLS certs

fix: start pebble-challtestsrv

fix: use separate jobs for each test type
@sheurich
display pebble version in container for release
5d4a227
@sheurich
display pebble version in container for release
7f95d56
@sheurich
create a release and upload release assets
86aaee7
@sheurich
fix: remove excess permissions for release workflow
8ead44c
@sheurich
remove unused shell scripts
09d6943
@aarongable aarongable requested review from a team and pgporada and removed request for a team March 12, 2024 22:23
@mcpherrinm mcpherrinm merged commit 235473c into letsencrypt:main Mar 12, 2024
14 checks passed
@mcpherrinm
Copy link
Contributor

Merged and tagged as v2.5.0

@pgporada pgporada mentioned this pull request May 24, 2024
Merged
pgporada added a commit that referenced this pull request May 31, 2024
@pgporada
Document exposing API and management ports when not using docker-comp…
ad9e5bb 
…ose.yaml (#465)

#444 removed exposing the API
and management ports directly in the `Dockerfile` in favor of the
`docker-compose.yml`. Not everyone will be using `docker compose` so
let's document how it should be done.

Fixes #452
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ldez ldez ldez left review comments

@mcpherrinm mcpherrinm mcpherrinm approved these changes

@aarongable aarongable aarongable approved these changes

@pgporada pgporada Awaiting requested review from pgporada pgporada was automatically assigned from letsencrypt/boulder-developers

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@sheurich @ldez @mcpherrinm @aarongable