Tests fail when building with embedded CA bundle due to additional output

[rainerjung]

I did this

building curl with "--with-ca-embed=" lets the following tests fail:

TESTFAIL: These test cases failed: 24 220 360 371 387 418 421 433 469 470 728 749 750 794 978 994 996 1007 1067 1148 1188 1268 1278 1457 1471 1472 1489 1491
This is due to the following two additional output lines, that are not expected by the test definitions:

Note: Using embedded CA bundle (NNN bytes)[CR][LF]
or
Note: Using embedded CA bundle (NNN bytes)[LF]

and

Note: Using embedded CA bundle, for proxies (NNN bytes)[CR][LF]
or
Note: Using embedded CA bundle, for proxies (NNN bytes)[LF]

I checked the docs about writing tests but didn't find a way to ignore such lines in a sort of global configuration. As a work around, I patched the tests, which were failing for my build with ither

<stripfile>
s/^Note: Using embedded CA bundle.*\n//
</stripfile>

or in some cases more specific

<stripfile2>
s/^Note: Using embedded CA bundle.*\n//
</stripfile2>

i will attach the patch I used. Of course there could be other tests failing, which were skipped for my build. So a global ignore list would be nice.

I expected the following

_No failure.

curl/libcurl version

curl 8.20.0

operating system

EL Linux 8

[rainerjung]

tests-tolerate-ca-embed.patch

[vszakats]

Thanks for the report and patch.

Agreed that a global ignore would be ideal here. runtests can detect
such build via the CAcert feature string, and then strip these line
from all curl stderr output.

I'd be happy to hear if this patch works for you: #21973

[rainerjung]

Thanks, great, that the small change fixes most of the test failures much more robust than in the individual test descriptions.

Unfortunately four tests still fail due to or conditions, which are not fixed by this patch. All others check and are fixed.

I will attach the patch fixing the remaining four ones with individual changes to the test descriptions. But I will also check, whether I can add a fix along your proposal to runtests.pl, which applies to and as well.

[rainerjung]

tests-tolerate-file-ca-embed.patch

[vszakats]

Pushed an update to also auto-apply the filter to redirected output files.

Does it help?

[rainerjung]

Thanks. I worked in parallel on a slightly different approach. I'll attach the (trivial) patch. If you like it, I can make a pull request, but you can also just pick it up. If you don't like it, I can test your latest version, which by inspection already looks good.

[rainerjung]

runtests-global-skip.patch

[vszakats]

LGTM, I merge it into the open PR.