Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Look into replacing Boulder tests w/ Pebble tests (or removing it entirely) #9898

Closed
wgreenberg opened this issue Feb 8, 2024 · 4 comments · Fixed by #9918
Closed

Look into replacing Boulder tests w/ Pebble tests (or removing it entirely) #9898

wgreenberg opened this issue Feb 8, 2024 · 4 comments · Fixed by #9918
Assignees
@wgreenberg

Comments

@wgreenberg
Copy link
Collaborator

Our boulder tests have recently been breaking due to a change in boulder, but pebble (https://github.com/letsencrypt/pebble/) is the recommended test util for testing against ACME servers. Can we convert our boulder tests to use pebble, or if we deem them unnecessary, remove them entirely?
@wgreenberg wgreenberg self-assigned this Feb 8, 2024
@wgreenberg wgreenberg mentioned this issue Feb 9, 2024
Closed
@jvanasco
Copy link

jvanasco commented Feb 9, 2024

FYI, I implemented pebble tests for our client in a very simple but inelegant way.

I use a decorator to spin up a version of pebble for each test (https://github.com/aptise/peter_sslers/blob/main/tests/_utils.py#L288-L334) using psutil.

I also manipulate the hosts file on the github setup (https://github.com/aptise/peter_sslers/blob/main/.github/workflows/core-tests.yml#L48-L51)

This has made unit and integrated tests very easy. There is a bit of an overhead to spin up pebble for each test, but the entire pebble database is localized to the test.

@wgreenberg
Copy link
Collaborator Author

oh nice! that's a handy setup. though actually, certbot's already got a harness for running pebble tests (i didn't do a great job explaining this in the issue description), we just also have a parallel set of tests that use boulder which we run in certain circumstances.

@wgreenberg
Copy link
Collaborator Author

With letsencrypt/pebble#433 landed, this is just blocked on the next Pebble release, which itself is blocked pending their CI overhaul (letsencrypt/pebble#444).

@wgreenberg
Copy link
Collaborator Author

New pebble release is out, so this should be good to go. I'll open a PR shortly

@wgreenberg wgreenberg mentioned this issue Mar 26, 2024
Merged
@bmw bmw closed this as completed in #9918 May 2, 2024
bmw added a commit that referenced this issue May 2, 2024
@wgreenberg @bmw
Replace boulder tests with pebble (#9918)
873f979 
Pebble 2.5.1 supports OCSP stapling, so we can finally replace all boulder tests/harnesses with the much simpler pebble setup.

Closes #9898

* Remove unused `--acme-server` argument

Since this argument is never set and always defaults to 'pebble', just
remove it to simplify assumptions about which test server's being used.

* Remove boulder option from integration tests

Now that pebble supports all of our test cases, we can move off of
the much more complicated boulder test harness.

* pebble_artifacts: bump to latest pebble release

* pebble_artifacts: fix download path

* certbot-ci: unzip pebble assets

* CI: rip out windows tests/jobs

* tox.ini: rm outdated Windows comment

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* ci: rm redundant integration test

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* acme_server: raise error if proxy and http-01 port are both set

* acme_server: rm vestigial preterimate commands stuff

---------

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wgreenberg wgreenberg

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Replace boulder tests with pebble certbot/certbot
2 participants
@jvanasco @wgreenberg