Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GHA: add NetBSD, OpenBSD, FreeBSD/arm64 and OmniOS jobs #13583

Closed
wants to merge 91 commits into from

Conversation

vszakats
Copy link
Member

@vszakats vszakats commented May 10, 2024

Add these jobs to GHA:

  • NetBSD, cmake-unity, clang, OpenSSL, x86_64, with tests, w/o python,
    no parallelism (was flaky sometimes)
  • OpenBSD, cmake-unity, clang, LibreSSL, x86_64, with tests,
    with python, -j8, TFTP results ignored due to TFTP tests fail on OpenBSD (ci/GHA) #13623.
  • FreeBSD, cmake-unity and autotools, clang, OpenSSL, arm64
    (Tests disabled for arm64, because they are slow. It's available for
    x86_64 with python, -j12.)
    Configuration matches our existing Cirrus CI one.
  • OmniOS, autotools, gcc, OpenSSL, x86_64, with tests, -j12.

All build with websockets and examples.

Closes #13583


@vszakats vszakats added the CI Continuous Integration label May 10, 2024
@vszakats vszakats marked this pull request as draft May 10, 2024 13:30
@vszakats
Copy link
Member Author

vszakats commented May 10, 2024

unity_0_c.c(CMakeFiles/curl.dir/Unity/unity_0_c.c.o:(create_dir_hierarchy)): warning: strcpy() is almost always misused, please use strlcpy()

Ref: https://github.com/curl/curl/actions/runs/9033005218/job/24822356703?pr=13583#step:3:421

edit: The complete list:

curl_trc.c:99 (curl/lib/curl_trc.c:99)(unity_0_c.c.o:(Curl_failf) in archive ../../lib/libcurlu-d.a): warning: strcpy() is almost always misused, please use strlcpy()
first.c:101 (curl/tests/libtest/first.c:101)(CMakeFiles/lib1156.dir/first.c.o:(memory_tracking_init)): warning: strcpy() is almost always misused, please use strlcpy()
getpart.c:470 (curl/tests/server/getpart.c:470)(CMakeFiles/fake_ntlm.dir/getpart.c.o:(getpart)): warning: strcpy() is almost always misused, please use strlcpy()
hsts-preload.c:60 (curl/docs/examples/hsts-preload.c:60)(CMakeFiles/curl-example-hsts-preload.dir/hsts-preload.c.o:(hstsread)): warning: strcpy() is almost always misused, please use strlcpy()
tool_dirhie.c:147 (curl/src/tool_dirhie.c:147)(CMakeFiles/curl.dir/Unity/unity_0_c.c.o:(create_dir_hierarchy)): warning: strcpy() is almost always misused, please use strlcpy()

Ref: https://github.com/curl/curl/actions/runs/9301034519/job/25598321217?pr=13832

@bagder
Copy link
Member

bagder commented May 10, 2024

warning: strcpy() is almost always misused, please use strlcpy()

The build looked green anyway so I think this is a warning we can just ignore?

@vszakats
Copy link
Member Author

It seems one of those OpenBSD-specific extra security warnings. Could explain why it doesn't trip on -Werror.
If the code looks okay, it's safe to ignore. If it has an easy fix to silence, it'd be nice to do that for a clean log.

@bagder
Copy link
Member

bagder commented May 10, 2024

If the code looks okay, it's safe to ignore.

Right. strlcpy is not a portable function anyway so we cannot easily switch to it universally. Our existing uses of strcpy() should be fairly safe, and we are working on minimizing the use of this function.

If it has an easy fix to silence, it'd be nice to do that for a clean log.

Agreed.

vszakats added a commit to vszakats/curl that referenced this pull request May 10, 2024
```
../../lib/ldap.c: In function 'ldap_do':
  ../../lib/ldap.c:380:11: error: unused variable 'ldap_ca' [-Werror=unused-variable]
    380 |     char *ldap_ca = conn->ssl_config.CAfile;
        |           ^~~~~~~
  ../../lib/ldap.c:379:9: error: unused variable 'ldap_option' [-Werror=unused-variable]
    379 |     int ldap_option;
        |         ^~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9033564377/job/24824192730#step:3:6059

Ref: curl#13583
Closes #xxxxx
@vszakats
Copy link
Member Author

Running tests in these env is risky, and it's done at this point for FreeBSDs and OmniOS. The latter had this fallout:

TESTFAIL: These test cases failed: 1119 1167

Ref: https://github.com/curl/curl/actions/runs/9034968328/job/24828726396#step:3:10013

Not much clue here:
1119: https://github.com/curl/curl/actions/runs/9034968328/job/24828726396#step:3:9406
1167: https://github.com/curl/curl/actions/runs/9034968328/job/24828726396#step:3:9453

Both are static test scripts.

@vszakats
Copy link
Member Author

Both are static test scripts.

Both depending on a C preprocessor cpp. Likely this is where it fail on this OS.

@dfandrich
Copy link
Contributor

We already have a FreeBSD 14.0 on Cirrus. Is there any reason to have this one, too?

vszakats added a commit that referenced this pull request May 10, 2024
```
../../lib/ldap.c: In function 'ldap_do':
  ../../lib/ldap.c:380:11: error: unused variable 'ldap_ca' [-Werror=unused-variable]
    380 |     char *ldap_ca = conn->ssl_config.CAfile;
        |           ^~~~~~~
  ../../lib/ldap.c:379:9: error: unused variable 'ldap_option' [-Werror=unused-variable]
    379 |     int ldap_option;
        |         ^~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9033564377/job/24824192730#step:3:6059

Ref: #13583
Closes #13588
@vszakats
Copy link
Member Author

vszakats commented May 10, 2024

This method supports other BSDs and OmniOS, along with arm64 support. Also multiple versions of these OSes. The configuration is more familiar, being GHA. Also since it's GHA, it's possible to restart jobs in case of failure for team members (without special Cirrus CI access) (edit: turns out Cirrus CI can easily be restarted from the GitHub UI).

It demonstrates that these platforms can be built on GHA, which I think we haven't done yet in curl.

This is based on these two projects, with more info about capabilities under their pages:

We can keep Cirrus CI as-is, and drop the x86_64 job from this workflow. Cirrus CI seems to complete faster. [DONE]

vszakats added a commit to vszakats/curl that referenced this pull request May 11, 2024
`ENABLE_DEBUG=ON` enables curl-level debug features. Building and
running the curl test suite requires setting this option.

Before this patch, while building the `testdeps` target with debug
features on with the default ('Release') CMake config, the build stopped
with this linker error:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
A failure has been detected in another branch of the parallel make
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

It happened because `dedotdotify` compiled as a static function into
`libcurlu` due to the undefined `DEBUGBUILD` macro. Then `unit1395`
failed to link it.

Even though the build requires the `DEBUGBUILD` macro, our CMake logic
defined it when building with CMake's 'Debug' configuration only. That
configuration is not required to build or run the test suite, nor to use
curl-level debug features.

This patch fixes this by always defining `DEBUGBUILD` when setting
`ENABLE_DEBUG=ON`. Decoupling this custom option from the selected CMake
Release/Debug configuration.

This change may also allow dropping Debug mode in AppVeyor CMake builds,
possibly making them build faster and run smoother.

Also delete the internal variable `ENABLE_CURLDEBUG` which was a copy of
`ENABLE_DEBUG`. Use the latter everywhere instead.

Ref: curl#13583
Closes #xxxxx
@vszakats
Copy link
Member Author

vszakats commented May 11, 2024

All clear (with !1119 !1167 for OmniOS), except OpenBSD, where these fail (out of 2 runs):

TESTFAIL: These test cases failed:
  271 283 284 285 286 332 1007 1009 1093 1094 1099 1167 1242 1243 2002 2003 
  271 283 284 285 286 332 1007 1049 1093 1094 1099 1167 1242 1243 2002 2003 

Ref: https://github.com/curl/curl/actions/runs/9042288591/job/24855020568?pr=13583#step:3:9209

edit: failures are consistent for 2 runs. They seem to be TFTP related, except 1167, which was failing for OmniOS too.

edit: Speaking of TFTP and OpenBSD, it seems the result contains the content twice:

 271: protocol FAILED:
--- log/check-expected	Sat May 11 17:39:48 2024
+++ log/check-generated	Sat May 11 17:39:48 2024
@@ -3,3 +3,8 @@
 tsize = 0[LF]
 blksize = 512[LF]
 filename = /271[LF]
+opcode = 1[LF]
+mode = octet[LF]
+tsize = 0[LF]
+blksize = 512[LF]
+filename = /271[LF]
== Contents of files in the log/ dir after test 271
=== Start of file check-expected
 opcode = 1[LF]
 mode = octet[LF]
 tsize = 0[LF]
 blksize = 512[LF]
 filename = /271[LF]
=== End of file check-expected
=== Start of file check-generated
 opcode = 1[LF]
 mode = octet[LF]
 tsize = 0[LF]
 blksize = 512[LF]
 filename = /271[LF]
 opcode = 1[LF]
 mode = octet[LF]
 tsize = 0[LF]
 blksize = 512[LF]
 filename = /271[LF]
=== End of file check-generated
=== Start of file commands.log
 ../src/curl -q --output log/curl271.out  --include --trace-ascii log/trace271 --trace-time tftp://127.0.0.1:14411//271 > log/stdout271 2> log/stderr271
=== End of file commands.log
=== Start of file curl271.out
 a chunk of
 data
 returned
  to client
=== End of file curl271.out
=== Start of file server.cmd
 Testnum 271
=== End of file server.cmd
=== Start of file server.input
 opcode = 1
 mode = octet
 tsize = 0
 blksize = 512
 timeout = 6
 filename = /271
 opcode = 1
 mode = octet
 tsize = 0
 blksize = 512
 timeout = 6
 filename = /271
=== End of file server.input
=== Start of file stderr271
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
 
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 100    36    0    36    0     0  28731      0 --:--:-- --:--:-- --:--:-- 28731
 
 100    36    0    36    0     0  26354      0 --:--:-- --:--:-- --:--:-- 26354
=== End of file stderr271
=== Start of file tftp_server.log
 17:39:47.382385 Wrote pid 4524 to log/server/tftp_server.pid
 17:39:47.382974 Wrote port 14411 to log/server/tftp_server.port
 17:39:47.382987 Running IPv4 version on port UDP/14411
 17:39:48.371327 trying to get file: verifiedserver mode 1
 17:39:48.371372 Are-we-friendly question received
 17:39:48.371436 write
 17:39:48.371487 read
 17:39:48.371678 read: 4
 17:39:48.371924 end of one transfer
 17:39:48.372321 trying to get file: verifiedserver mode 1
 17:39:48.372336 Are-we-friendly question received
 17:39:48.372373 write
 17:39:48.372398 read
 17:39:48.372434 read: -1
 17:39:48.372441 read: fail
 17:39:48.372569 end of one transfer
 17:39:48.397807 trying to get file: /271 mode 1
 17:39:48.397862 requested test number 271 part 0
 17:39:48.397984 file opened and all is good
 17:39:48.397994 write
 17:39:48.398011 read
 17:39:48.398492 read: 4
 17:39:48.398664 end of one transfer
 17:39:48.398990 trying to get file: /271 mode 1
 17:39:48.398999 requested test number 271 part 0
 17:39:48.399131 file opened and all is good
 17:39:48.399202 write
 17:39:48.399222 read
 17:39:48.399274 read: -1
 17:39:48.399289 read: fail
 17:39:48.399449 end of one transfer
=== End of file tftp_server.log
=== Start of file trace271
 17:39:48.396257 == Info: !!! WARNING !!
 17:39:48.396665 == Info: This is a debug build of libcurl, do not use in production.
 17:39:48.396729 == Info: STATE: INIT => SETUP handle 0x164fa4fe008; line 1971
 17:39:48.396732 == Info: STATE: SETUP => CONNECT handle 0x164fa4fe008; line 1987
 17:39:48.396865 == Info: Added connection 0. The cache now contains 1 members
 17:39:48.396918 == Info: STATE: CONNECT => CONNECTING handle 0x164fa4fe008; line 2023
 17:39:48.397202 == Info:   Trying 127.0.0.1:14411...
 17:39:48.397208 == Info: Connected to 127.0.0.1 (127.0.0.1) port 14411
 17:39:48.397212 == Info: STATE: CONNECTING => PROTOCONNECT handle 0x164fa4fe008; line 2131
 17:39:48.397223 == Info: set timeouts for state 0; Total  300000, retry 6 maxtry 50
 17:39:48.397245 == Info: STATE: PROTOCONNECT => DO handle 0x164fa4fe008; line 2160
 17:39:48.397248 == Info: TFTP_STATE_START
 17:39:48.397279 == Info: TFTP_STATE_START
 17:39:48.397293 == Info: STATE: DO => DOING handle 0x164fa4fe008; line 2243
 17:39:48.398067 <= Recv data, 36 bytes (0x24)
 0000: a chunk of.data.returned. to client.
 17:39:48.398434 == Info: TFTP_STATE_START
 17:39:48.398440 == Info: Connected for receive
 17:39:48.398444 == Info: set timeouts for state 1; Total  0, retry 72 maxtry 50
 17:39:48.398463 == Info: DO phase is complete
 17:39:48.398465 == Info: STATE: DOING => DID handle 0x164fa4fe008; line 2326
 17:39:48.398469 == Info: STATE: DID => DONE handle 0x164fa4fe008; line 2382
 17:39:48.398472 == Info: multi_done[DONE]: status: 0 prem: 0 done: 0
 17:39:48.398823 == Info: multi_done, not reusing connection=0, forbid=0, close=1, premature=0, conn_multiplex=0
 17:39:48.398865 == Info: The cache now contains 0 members
 17:39:48.398869 == Info: Curl_disconnect(conn #0, dead=0)
 17:39:48.398887 == Info: Closing connection
=== End of file trace271

It seems the transfer is executed twice.

@vszakats vszakats changed the title ci/GHA: add BSD and OmniOS builds GHA: add BSD and OmniOS jobs May 11, 2024
@vszakats vszakats changed the title GHA: add BSD and OmniOS jobs GHA: add NetBSD, OpenBSD, FreeBSD/arm64 and OmniOS jobs May 11, 2024
@vszakats vszakats marked this pull request as ready for review May 11, 2024 18:18
@vszakats vszakats marked this pull request as draft May 11, 2024 18:19
@vszakats vszakats marked this pull request as ready for review May 11, 2024 18:52
vszakats added a commit to vszakats/curl that referenced this pull request May 12, 2024
`ENABLE_DEBUG=ON` enables curl-level debug features. Building and
running the curl test suite requires setting this option.

Before this patch, while building the `testdeps` target with debug
features on with the default ('Release') CMake config, the build stopped
with this linker error:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
A failure has been detected in another branch of the parallel make
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

It happened because `dedotdotify` compiled as a static function into
`libcurlu` due to the undefined `DEBUGBUILD` macro. Then `unit1395`
failed to link it.

Even though the build requires the `DEBUGBUILD` macro, our CMake logic
defined it when building with CMake's 'Debug' configuration only. That
configuration is not required to build or run the test suite, nor to use
curl-level debug features.

This patch fixes this by always defining `DEBUGBUILD` when setting
`ENABLE_DEBUG=ON`. Decoupling this custom option from the selected CMake
Release/Debug configuration.

This change may also allow dropping Debug mode in AppVeyor CMake builds,
possibly making them build faster and run smoother.

Also delete the internal variable `ENABLE_CURLDEBUG` which was a copy of
`ENABLE_DEBUG`. Use the latter everywhere instead.

Ref: curl#13583
Closes #xxxxx
vszakats added a commit to vszakats/curl that referenced this pull request May 12, 2024
`ENABLE_DEBUG=ON` enables curl-level debug features. Building and
running the curl test suite requires setting this option.

Before this patch, while building the `testdeps` target with debug
features on with the default ('Release') CMake config, the build stopped
with this linker error:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
A failure has been detected in another branch of the parallel make
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

It happened because `dedotdotify` compiled as a static function into
`libcurlu` due to the undefined `DEBUGBUILD` macro. Then `unit1395`
failed to link it.

Even though the build requires the `DEBUGBUILD` macro, our CMake logic
defined it when building with CMake's 'Debug' configuration only. That
configuration is not required to build or run the test suite, nor to use
curl-level debug features.

This patch fixes this by always defining `DEBUGBUILD` when setting
`ENABLE_DEBUG=ON`. Decoupling this custom option from the selected CMake
Release/Debug configuration.

This change may also allow dropping Debug mode in AppVeyor CMake builds,
possibly making them build faster and run smoother.

Ref: curl#13583
Closes curl#13592
vszakats added a commit to vszakats/curl that referenced this pull request May 13, 2024
`ENABLE_DEBUG=ON` enables curl-level debug features. Building and
running the curl test suite requires setting this option.

Before this patch, while building the `testdeps` target with debug
features on with the default ('Release') CMake config, the build stopped
with this linker error:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
A failure has been detected in another branch of the parallel make
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

It happened because `dedotdotify` compiled as a static function into
`libcurlu` due to the undefined `DEBUGBUILD` macro. Then `unit1395`
failed to link it.

Even though the build requires the `DEBUGBUILD` macro, our CMake logic
defined it when building with CMake's 'Debug' configuration only. That
configuration is not required to build or run the test suite, nor to use
curl-level debug features.

This patch fixes this by always defining `DEBUGBUILD` when setting
`ENABLE_DEBUG=ON`. Decoupling this custom option from the selected CMake
Release/Debug configuration.

This change may also allow dropping Debug mode in AppVeyor CMake builds,
possibly making them build faster and run smoother.

Ref: curl#13583
Closes curl#13592
@vszakats
Copy link
Member Author

vszakats commented May 17, 2024

These new tests are ready to merge. They also might need tweaking if they turn out to be unstable or too much load.

The autotools job for FreeBSD is slow, but left it there to match Cirrus CI (but using arm64 here). The Cirrus CI build configuration is also matched, in case we'd want to migrate it (its runner is a little flaky) to GHA.

For NetBSD and OpenBSD it's not possible to uninstall system curl (it's required by cmake), so I used the CURL env to point to the built binary, but I haven't confirmed that tests are indeed running our curl, not the system one. (I also didn't verify this for platforms which allow to delete system curl). It'd be nice to display which curl binary is used for tests. Maybe it's already done, and I missed it.

There were various small issues that I stopped pursuing, e.g. I failed to enable GSSAPI/heimdal on OpenBSD and OmniOS. Also libidn2 on OmniOS. It's also kind of weird how autotools picks up dependencies automatically on some platforms, but requires the --with-* on others. In many cases dependency detection defaults are not in sync between autotools and CMake; maybe subject to future PR(s).

Tests requiring python (and packages), stunnel, nghttp2 server are not run. These components are installed for FreeBSD (based on Cirrus CI), but all tests are skipped on that platform due to emulated CPU performance (but they do work, or at least they did at one point while working on this). For the remaining platforms it's a fiddly task and did not end up finishing installing them. (It's also a maintenance burden due to versioned package names.)

Then there is the TFTP failures on OpenBSD. Tests are run now, but results ignored as a workaround.

vszakats added a commit to vszakats/curl that referenced this pull request May 18, 2024
`ENABLE_DEBUG=ON` enables curl-level debug features. Building and
running the curl test suite requires setting this option.

Before this patch, while building the `testdeps` target with debug
features on with the default ('Release') CMake config, the build stopped
with this linker error:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
A failure has been detected in another branch of the parallel make
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

It happened because `dedotdotify` compiled as a static function into
`libcurlu` due to the undefined `DEBUGBUILD` macro. Then `unit1395`
failed to link it.

Even though the build requires the `DEBUGBUILD` macro, our CMake logic
defined it when building with CMake's 'Debug' configuration only. That
configuration is not required to build or run the test suite, nor to use
curl-level debug features.

This patch fixes this by always defining `DEBUGBUILD` when setting
`ENABLE_DEBUG=ON`. Decoupling this custom option from the selected CMake
Release/Debug configuration.

This change may also allow dropping Debug mode in AppVeyor CMake builds,
possibly making them build faster and run smoother.

Ref: curl#13583
Closes curl#13592
@vszakats
Copy link
Member Author

I plan to merge this today. If the new jobs turn out to be more pain than gain, I'm perfectly fine with tuning and/or deleting any/all of them. If it turns out to be working well, we might drop Cirrus CI in favour of a new GHA job in this workflow.

@vszakats vszakats closed this in 90e644f May 19, 2024
@vszakats vszakats deleted the gha-non-native branch May 19, 2024 21:10
@vszakats
Copy link
Member Author

vszakats commented May 19, 2024

Post-merge update: Right after merging, one PR failed in OmniOS running MQTT tests, and an FTP test in another job. This hasn't happened throughout the almost 100 runs while working on the PR.

MQTT / OpenBSD: → b3f23f2

TESTFAIL: These test cases failed: 3017

Ref: https://github.com/curl/curl/actions/runs/9223543272/job/25376999226?pr=13759#step:3:16326
Ref: https://github.com/curl/curl/actions/runs/9230183764/job/25397883193?pr=13766#step:3:16345

FTP / OpenBSD:

TESTFAIL: These test cases failed: 707

Ref: https://github.com/curl/curl/actions/runs/9497439182/job/26174007666?pr=13936#step:3:7217

FTP / OpenBSD:

TESTFAIL: These test cases failed: 1137

Ref: https://github.com/curl/curl/actions/runs/9735232161/job/26864218394?pr=14065#step:3:7253

MQTT / OmniOS:

TESTFAIL: These test cases failed: 1190 1198 3017

Ref: https://github.com/curl/curl/actions/runs/9258522297/job/25468730731?pr=13694#step:3:10251

MQTT / OmniOS:

TESTFAIL: These test cases failed: 1194 2200 2203 2205

Ref: https://github.com/curl/curl/actions/runs/9150523540/job/25155409832#step:3:10233

FTP / OmniOS:

TESTFAIL: These test cases failed: 1096

Ref: https://github.com/curl/curl/actions/runs/9150702711/job/25155793948#step:3:10247

FTP / OmniOS:

TESTFAIL: These test cases failed: 381

Ref: https://github.com/curl/curl/actions/runs/9163863822/job/25193897640#step:3:10230

FTP / OmniOS:

TESTFAIL: These test cases failed: 340

Ref: https://github.com/curl/curl/actions/runs/9233804752/job/25406671742?pr=13771#step:3:10245

MQTT / OpenBSD:

TESTFAIL: These test cases failed: 1193

Ref: https://github.com/curl/curl/actions/runs/9864507611/job/27239560519?pr=14136#step:3:6818

after cross-platform-actions/action v0.25.0:

FTP / OpenBSD:

TESTFAIL: These test cases failed: 1153

Ref: https://github.com/curl/curl/actions/runs/9929875896/job/27427964347?pr=14185#step:3:7159

FTP / OpenBSD:

TESTFAIL: These test cases failed: 1062

https://github.com/curl/curl/actions/runs/9936817577/job/27445863668?pr=14187#step:3:8661

I keep monitoring.

vszakats added a commit to vszakats/curl that referenced this pull request May 20, 2024
`ENABLE_DEBUG=ON` enables curl-level debug features. Building and
running the curl test suite requires setting this option.

Before this patch, while building the `testdeps` target with debug
features on with the default ('Release') CMake config, the build stopped
with this linker error:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
A failure has been detected in another branch of the parallel make
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

It happened because `dedotdotify` compiled as a static function into
`libcurlu` due to the undefined `DEBUGBUILD` macro. Then `unit1395`
failed to link it.

Even though the build requires the `DEBUGBUILD` macro, our CMake logic
defined it when building with CMake's 'Debug' configuration only. That
configuration is not required to build or run the test suite, nor to use
curl-level debug features.

This patch fixes this by always defining `DEBUGBUILD` when setting
`ENABLE_DEBUG=ON`. Decoupling this custom option from the selected CMake
Release/Debug configuration.

This change may also allow dropping Debug mode in AppVeyor CMake builds,
possibly making them build faster and run smoother.

Ref: curl#13583
Closes curl#13592
vszakats added a commit to vszakats/curl that referenced this pull request May 24, 2024
We do it in Cirrus CI, but for some platform it's not possible to delete
it and tests work anyway.

The test runner also runs `../src/curl` by default, which is always the
one freshly built. The runner may also need a system curl to talk to
APIs when needed.

Also stop setting `CURL` env. This isn't picked up by the runners,
and works out of the box anyway.

Follow-up to 90e644f curl#13583
Closes #xxxxx
vszakats added a commit to vszakats/curl that referenced this pull request May 24, 2024
We do it in Cirrus CI, but for some platform it's not possible to delete
it and tests work anyway.

The test runner also runs `../src/curl` by default, which is always the
one freshly built. The runner may also need a system curl to talk to
APIs when needed.

Also:

- stop setting `CURL` env. This isn't picked up by the runners,
  and works out of the box anyway.

- quote an option just in case.

Follow-up to 90e644f curl#13583
Closes curl#13765
vszakats added a commit to vszakats/curl that referenced this pull request May 24, 2024
We do it in Cirrus CI, but for some platform it's not possible to delete
it and tests work anyway.

The test runner also runs `../src/curl` by default, which is always the
one freshly built. The runner may also need a system curl to talk to
APIs when needed.

Also:
- stop setting `CURL` env. This isn't picked up by the runners,
  and works out of the box anyway.
- quote an option just in case.

Follow-up to 90e644f curl#13583
Closes curl#13765
vszakats added a commit that referenced this pull request May 24, 2024
We do it in Cirrus CI, but for some platforms it's not possible to
delete it and tests work anyway.

The test runner also runs `../src/curl` by default, which is always the
one freshly built. The runner may also need the system curl to talk to
APIs when needed.

Also:
- stop setting `CURL` env. This isn't picked up by the runners,
  and works out of the box anyway.
- quote an option just in case.

Follow-up to 90e644f #13583
Closes #13765
vszakats added a commit to vszakats/curl that referenced this pull request May 27, 2024
`ENABLE_DEBUG=ON` enables curl-level debug features. Building and
running the curl test suite requires setting this option.

Before this patch, while building the `testdeps` target with debug
features on with the default ('Release') CMake config, the build stopped
with this linker error:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
A failure has been detected in another branch of the parallel make
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

It happened because `dedotdotify` compiled as a static function into
`libcurlu` due to the undefined `DEBUGBUILD` macro. Then `unit1395`
failed to link it.

Even though the build requires the `DEBUGBUILD` macro, our CMake logic
defined it when building with CMake's 'Debug' configuration only. That
configuration is not required to build or run the test suite, nor to use
curl-level debug features.

This patch fixes this by always defining `DEBUGBUILD` when setting
`ENABLE_DEBUG=ON`. Decoupling this custom option from the selected CMake
Release/Debug configuration.

This change may also allow dropping Debug mode in AppVeyor CMake builds,
possibly making them build faster and run smoother.

Ref: curl#13583
Closes curl#13592
vszakats added a commit that referenced this pull request May 27, 2024
Before this patch `ENABLE_DEBUG=ON` always enabled the TrackMemory
(aka `ENABLE_CURLDEBUG=ON`) feature, but required the `Debug` CMake
configration to actually enable curl debug features
(aka `-DDEBUGBUILD`).

Curl debug features do not require compiling with C debug options. This
also made enabling debug features unintuitive and complicated to use.
Due to other issues (subject to PR #13694) it also caused an error in
default (and `Release`/`MinSizeRel`/`RelWithDebInfo`) configs, when
building the `testdeps` target:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

Fix it by always defining `DEBUGBUILD` when setting `ENABLE_DEBUG=ON`.
Decoupling this option from the selected CMake configuration.

Note that after this patch `ENABLE_DEBUG=ON` unconditionally enables
curl debug features. These features are insecure and unsuited for
production. Make sure to omit this option when building for production
in default, `Release` (and other not-`Debug`) modes.

Also delete a workaround no longer necessary in GHA CI jobs.

Ref: 1a62b6e (2015-03-03)
Ref: #13583
Closes #13592
vszakats added a commit that referenced this pull request May 27, 2024
MQTT / OmniOS:
```
TESTFAIL: These test cases failed: 1190 1198 3017
```
Ref: https://github.com/curl/curl/actions/runs/9258522297/job/25468730731?pr=13694#step:3:10251

MQTT / OmniOS:
```
TESTFAIL: These test cases failed: 1194 2200 2203 2205
```
Ref: https://github.com/curl/curl/actions/runs/9150523540/job/25155409832#step:3:10233

FTP / OmniOS:
```
TESTFAIL: These test cases failed: 1096
```
Ref: https://github.com/curl/curl/actions/runs/9150702711/job/25155793948#step:3:10247

FTP / OmniOS:
```
TESTFAIL: These test cases failed: 381
```
Ref: https://github.com/curl/curl/actions/runs/9163863822/job/25193897640#step:3:10230

FTP / OmniOS:
```
TESTFAIL: These test cases failed: 340
```
Ref: https://github.com/curl/curl/actions/runs/9233804752/job/25406671742?pr=13771#step:3:10245

Ref: #13583 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration tests
Development

Successfully merging this pull request may close these issues.

None yet

3 participants