Skip to content

libtests: stop building the sames source multiple times #17598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 23 commits into from
Closed

libtests: stop building the sames source multiple times #17598

wants to merge 23 commits into from

Conversation

vszakats
Copy link
Member

@vszakats vszakats commented Jun 11, 2025 •
edited
Loading

After this patch there is no more double/multiple compile of the same
libtest source under a different libtest ID. Each libtest is compiled
once, and changing behavior at runtime based on test ID.

  • drop recently added physical clones for two prevously multi-compiled
    tests:

    • merge lib587 into lib554 again, branch at runtime.
    • merge lib645 into lib643 again, branch at runtime.

  • replace existing dynamic branching to use testnum instead of
    a manually rolled testno based on an extra command-line argument.
    lib1571, lib1576.

  • mk-bundle.pl: stop defining LIB* macros. No longer used.

  • libtests: drop all LIB*_C guards.

  • Make these tests branch at runtime, stop building copies:

    • lib585, based on lib500
    • lib565, based on lib510
    • lib529, based on lib525
    • lib527, lib532, based on lib526
    • lib545, based on lib544
    • lib548, based on lib547
    • lib696, based on lib556
    • lib584, based on lib589
    • lib1539, based on lib1514
    • lib1543, based on lib1518
    • lib1917, based on lib1916
    • lib1946, based on lib1940
    • lib671, 672, 673, based on lib670

Follow-up to 02dd471 #17591
Follow-up to 6897aeb #17468

w/o ws https://github.com/curl/curl/pull/17598/files?w=1

@vszakats vszakats marked this pull request as draft June 11, 2025 14:38
@vszakats vszakats mentioned this pull request Jun 11, 2025
Closed
12 tasks
vszakats added 19 commits June 11, 2025 16:46
@vszakats
lib1571: use testnum (was testno)
9c41358
@vszakats
lib1571 cleanup
447d7f2
@vszakats
lib1576: use testnum (was testno)
b6b1e0e
@vszakats
lib1576: fixup, 1580 was reusing 1576 but pretending to be 1578
66e34ed
@vszakats
lib585 convert to dynamic, based on lib500
c1c0c3a
@vszakats
more tests based on lib500
db31610
@vszakats
lib565 convert to dynamic, based on lib510
dc497b7
@vszakats
lib529 convert to dynamic, based on lib525
f9aa2a1
@vszakats
lib527, lib532 convert to dynamic, based on lib526
ccfe18b
@vszakats
lib545 convert to dynamic, based on lib544
fbeeb27
@vszakats
lib548 convert to dynamic, based on lib547
2fded26
@vszakats
lib696 convert to dynamic, based on lib556
dc447b8
@vszakats
lib584 convert to dynamic, based on lib589
7915323
@vszakats
lib1539 convert to dynamic, based on lib1514
538dfe1
@vszakats
lib1543 convert to dynamic, based on lib1518
b627dd7
@vszakats
lib1917 convert to dynamic, based on lib1916
5822da8
@vszakats
lib1946 convert to dynamic, based on lib1940
b65c3e2
@vszakats
lib671, 672, 673 convert to dynamic, based on lib670
769a3ef
@vszakats
mk-bundle.pl: stop defining LIB* macros. no longer used
86f93ea
vszakats added 2 commits June 11, 2025 18:32
@vszakats
whitespace
673089e
@vszakats
lib547: try asan fixup
ba75754 
```
==28724==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f028b500060 at pc 0x55edc7ea2e85 bp 0x7fff7d70a4d0 sp 0x7fff7d70a4c8
 READ of size 4 at 0x7f028b500060 thread T0
 /usr/bin/llvm-symbolizer-18: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: no version information available (required by /usr/bin/llvm-symbolizer-18)
 /usr/bin/llvm-symbolizer-18: symbol lookup error: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: undefined symbol: __asan_option_detect_stack_use_after_return
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't write to symbolizer at fd 9
 /usr/bin/llvm-symbolizer-18: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: no version information available (required by /usr/bin/llvm-symbolizer-18)
 /usr/bin/llvm-symbolizer-18: symbol lookup error: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: undefined symbol: __asan_option_detect_stack_use_after_return
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
     #0 0x55edc7ea2e84  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x1b6e84) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
     #1 0x7f028f6111dd  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x8111dd) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #2 0x7f028f38b8c4  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x58b8c4) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #3 0x7f028f38d60a  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x58d60a) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #4 0x7f028f3796a5  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x5796a5) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #5 0x7f028f11c442  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x31c442) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #6 0x7f028f11bd9c  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x31bd9c) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #7 0x7f028f37891b  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x57891b) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #8 0x7f028f2bb7f3  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x4bb7f3) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #9 0x7f028f28b01d  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x48b01d) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #10 0x7f028f5e3ce4  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x7e3ce4) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#11 0x7f028f5de4c1  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x7de4c1) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#12 0x7f028f340504  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x540504) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#13 0x7f028f33dba1  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x53dba1) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#14 0x7f028f4f02bd  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x6f02bd) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#15 0x7f028f212a09  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x412a09) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#16 0x7f028f211bae  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x411bae) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#17 0x55edc7ea2c73  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x1b6c73) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
     curl#18 0x55edc7f7b832  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x28f832) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
     curl#19 0x7f028dc2a1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 42c84c92e6f98126b3e2230ebfdead22c235b667)
     curl#20 0x7f028dc2a28a  (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 42c84c92e6f98126b3e2230ebfdead22c235b667)
     curl#21 0x55edc7da3a34  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0xb7a34) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
 Address 0x7f028b500060 is located in stack of thread T0 at offset 32 in frame
 ==28724==WARNING: Can't read from symbolizer at fd 6
     #0 0x55edc7ea2717  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x1b6717) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
   This frame has 1 object(s):
     [32, 36) 'counter' (line 91) <== Memory access at offset 32 is inside this variable
 HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
       (longjmp and C++ exceptions *are* supported)
 ==28724==WARNING: Can't read from symbolizer at fd 6
 SUMMARY: AddressSanitizer: stack-use-after-scope (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x1b6e84) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
 Shadow bytes around the buggy address:
   0x7f028b4ffd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x7f028b4ffe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x7f028b4ffe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x7f028b4fff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x7f028b4fff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 =>0x7f028b500000: f5 f5 f5 f5 f5 f5 f5 f5 f1 f1 f1 f1[f8]f3 f3 f3
   0x7f028b500080: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
   0x7f028b500100: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
   0x7f028b500180: f1 f1 f1 f1 04 f2 f8 f3 f5 f5 f5 f5 f5 f5 f5 f5
   0x7f028b500200: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
   0x7f028b500280: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
 Shadow byte legend (one shadow byte represents 8 application bytes):
   Addressable:           00
   Partially addressable: 01 02 03 04 05 06 07
   Heap left redzone:       fa
   Freed heap region:       fd
   Stack left redzone:      f1
   Stack mid redzone:       f2
   Stack right redzone:     f3
   Stack after return:      f5
   Stack use after scope:   f8
   Global redzone:          f9
   Global init order:       f6
   Poisoned by user:        f7
   Container overflow:      fc
   Array cookie:            ac
   Intra object redzone:    bb
   ASan internal:           fe
   Left alloca redzone:     ca
   Right alloca redzone:    cb
 ==28724==ABORTING
```
https://github.com/curl/curl/actions/runs/15590177615/job/43906987845?pr=17598#step:39:1276
@testclutch

This comment was marked as outdated.

Sign in to view
@vszakats vszakats marked this pull request as ready for review June 11, 2025 16:52
@vszakats vszakats changed the title libtests: try to avoid build-time clone of libtests libtests: stop building the same source multiple times Jun 11, 2025
@vszakats vszakats changed the title libtests: stop building the same source multiple times libtests: stop building the sames source multiple times Jun 11, 2025
@vszakats vszakats closed this in c9460d6 Jun 11, 2025
@vszakats vszakats deleted the t-unclone branch June 11, 2025 17:15
bagder added a commit that referenced this pull request Jun 11, 2025
@bagder
tests: tidy ups
369b5ca 
What remains after #17598
denandz pushed a commit to denandz/curl that referenced this pull request Jun 21, 2025
@vszakats @denandz
libtests: stop building the sames source multiple times
7228aee 
After this patch there is no more double/multiple compile of the same
libtest source under a different libtest ID. Each libtest is compiled
once, and changing behavior at runtime based on test ID.

- drop recently added physical clones for two prevously multi-compiled
  tests:
  - merge lib587 into lib554 again, branch at runtime.
  - merge lib645 into lib643 again, branch at runtime.

- replace existing dynamic branching to use `testnum` instead of
  a manually rolled `testno` based on an extra command-line argument.
  lib1571, lib1576.

- mk-bundle.pl: stop defining `LIB*` macros. No longer used.

- libtests: drop all `LIB*_C` guards.

- Make these tests branch at runtime, stop building copies:
  - lib585, based on lib500
  - lib565, based on lib510
  - lib529, based on lib525
  - lib527, lib532, based on lib526
  - lib545, based on lib544
  - lib548, based on lib547
  - lib696, based on lib556
  - lib584, based on lib589
  - lib1539, based on lib1514
  - lib1543, based on lib1518
  - lib1917, based on lib1916
  - lib1946, based on lib1940
  - lib671, 672, 673, based on lib670

Follow-up to 02dd471 curl#17591
Follow-up to 6897aeb curl#17468

Closes curl#17598
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
tests tidy-up
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vszakats @testclutch