-
-
Notifications
You must be signed in to change notification settings - Fork 6.7k
runtests: generate certs dynamically, bump to EC-256, tidy up #16824
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Fallouts:
|
This is working. A handful of local issues aside, it was reasonably smooth to implement. Downside is that it makes running tests require POSIX shell, also with CMake. It's also to be seen how annoying the cert generation is when running local tests. We might want to make the |
Excellent! |
edit: Possibly caused by an upstream update to vcpkg libiconv: microsoft/vcpkg#44424 (comment) Odd, unrelated-looking, Android 21 CMake build fallout popping in and out:
https://github.com/curl/curl/actions/runs/14058024728/job/39361854217?pr=16824#step:11:81 (gone after a rebuild) |
Hah, the question now is how to tell https://github.com/curl/curl/actions/runs/14059113100/job/39365322136?pr=16824#step:39:22 |
This seems complete. Now integrated into the test run targets. Also bumped keys to EC-256, and cut down on log output to make it faster. Now it's 3-4 seconds on the slowest MSVC/Cygwin jobs, which is fine. The step could be dropped for non-stunnel jobs to save idle work (though it'd probably be better to have stunnel everywhere for coverage, e.g. on Cygwin). Bash dependency still stands. For stunnel-enabled runs, bash mush be available to avoid failing test cases. If we get reports of this combo causing confusion, we can add dedicated logic and an early error message for example. Or the non-bash alternative if that case turns out to be important.
edit: dependency handling and local use may also need updates to limit re-runs only when necessary. I haven't tested. |
Update: not ready yet. Local tests kept regenerating certs. Trying to fix things now. Also moving certs to the build directory from the source, which seems to be natural place for them, when generated dynamically. This allows dropping the (Though generating them is quite quick, and the log is just a few lines now.) autotools is still to be tested. |
Automake, cmake, local, repeated builds are covered now. This is ready to merge. Likely fit for RC3 or the release. |
vcpkg requires Android 28 by default after a recent update that's being deployed onto CI runs (with `libiconv:arm64-android@1.18#1`). Revert to bare, no-ssl, no-psl configuration for Android 21 jobs to make them work again. Bug: #16824 (comment) Ref: microsoft/vcpkg#44424 (comment) Closes #16832
To avoid regenerating certs always.
It'd much nicer to use a multi-target rule, but that'd require GNU Make 4.3 for "Grouped explicit targets" with `&:`.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think getting rid of those big "magic" files from git is a huge benefit, let's do this!
This is merged now! Does anybody know how to generate |
To remove POSIX shell as an extra dependency for runtests. Follow-up to 44341e7 curl#16824
The obvious just hit me: We might rewrite As for SRP, the GnuTLS We may create a generatotr script, but running it dynamically will |
Yes I think we should to that. SRP also does not work with TLS 1.3 and was never used much. |
To remove POSIX shell as an extra dependency for runtests. Follow-up to 44341e7 curl#16824
Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.
Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.
As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.
Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.
Details:
separate checker tool.
confusion when updating the revocation database and counter.
scripts
subdir, merge two scripts into one,auto-generate root cert, allow generating multiple leafs at once.
-x
echo, text dumps, most other output. To avoid lognoise and make it quicker in CI.
build-certs
rule to not depend onSRPFILES
(
srp-verifier-*
).EXCLUDE_FROM_ALL
for the cert subdir. It makesthe Visual Studio generator miss to create the
clean-certs
,build-certs
targets. No target depend on them, so they don't executeimplicitly anyway. Fixes:
VERBATIM USES_TERMINAL
tobuild-certs
target.Follow-up to 556f722 #16593
Follow-up to fa461b4 #14486
w/o ws https://github.com/curl/curl/pull/16824/files?w=1
TODO:
clean-certs
step from CI workflows.