Skip to content

libtests: stop building the sames source multiple times#17598

Closed
vszakats wants to merge 23 commits intocurl:masterfrom
vszakats:t-unclone
Closed

libtests: stop building the sames source multiple times#17598
vszakats wants to merge 23 commits intocurl:masterfrom
vszakats:t-unclone

Conversation

@vszakats
Copy link
Copy Markdown
Member

@vszakats vszakats commented Jun 11, 2025 •
edited
Loading

After this patch there is no more double/multiple compile of the same
libtest source under a different libtest ID. Each libtest is compiled
once, and changing behavior at runtime based on test ID.

  • drop recently added physical clones for two prevously multi-compiled
    tests:

    • merge lib587 into lib554 again, branch at runtime.
    • merge lib645 into lib643 again, branch at runtime.

  • replace existing dynamic branching to use testnum instead of
    a manually rolled testno based on an extra command-line argument.
    lib1571, lib1576.

  • mk-bundle.pl: stop defining LIB* macros. No longer used.

  • libtests: drop all LIB*_C guards.

  • Make these tests branch at runtime, stop building copies:

    • lib585, based on lib500
    • lib565, based on lib510
    • lib529, based on lib525
    • lib527, lib532, based on lib526
    • lib545, based on lib544
    • lib548, based on lib547
    • lib696, based on lib556
    • lib584, based on lib589
    • lib1539, based on lib1514
    • lib1543, based on lib1518
    • lib1917, based on lib1916
    • lib1946, based on lib1940
    • lib671, 672, 673, based on lib670

Follow-up to 02dd471 #17591
Follow-up to 6897aeb #17468

w/o ws https://github.com/curl/curl/pull/17598/files?w=1

@vszakats vszakats marked this pull request as draft June 11, 2025 14:38
@vszakats vszakats mentioned this pull request Jun 11, 2025
Closed
12 tasks
vszakats added 19 commits June 11, 2025 16:46
@vszakats
lib1571: use testnum (was testno)
9c41358
@vszakats
lib1571 cleanup
447d7f2
@vszakats
lib1576: use testnum (was testno)
b6b1e0e
@vszakats
lib1576: fixup, 1580 was reusing 1576 but pretending to be 1578
66e34ed
@vszakats
lib585 convert to dynamic, based on lib500
c1c0c3a
@vszakats
more tests based on lib500
db31610
@vszakats
lib565 convert to dynamic, based on lib510
dc497b7
@vszakats
lib529 convert to dynamic, based on lib525
f9aa2a1
@vszakats
lib527, lib532 convert to dynamic, based on lib526
ccfe18b
@vszakats
lib545 convert to dynamic, based on lib544
fbeeb27
@vszakats
lib548 convert to dynamic, based on lib547
2fded26
@vszakats
lib696 convert to dynamic, based on lib556
dc447b8
@vszakats
lib584 convert to dynamic, based on lib589
7915323
@vszakats
lib1539 convert to dynamic, based on lib1514
538dfe1
@vszakats
lib1543 convert to dynamic, based on lib1518
b627dd7
@vszakats
lib1917 convert to dynamic, based on lib1916
5822da8
@vszakats
lib1946 convert to dynamic, based on lib1940
b65c3e2
@vszakats
lib671, 672, 673 convert to dynamic, based on lib670
769a3ef
@vszakats
mk-bundle.pl: stop defining LIB* macros. no longer used
86f93ea
vszakats added 2 commits June 11, 2025 18:32
@vszakats
whitespace
673089e
@vszakats
lib547: try asan fixup
ba75754 
```
==28724==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7f028b500060 at pc 0x55edc7ea2e85 bp 0x7fff7d70a4d0 sp 0x7fff7d70a4c8
 READ of size 4 at 0x7f028b500060 thread T0
 /usr/bin/llvm-symbolizer-18: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: no version information available (required by /usr/bin/llvm-symbolizer-18)
 /usr/bin/llvm-symbolizer-18: symbol lookup error: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: undefined symbol: __asan_option_detect_stack_use_after_return
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't write to symbolizer at fd 9
 /usr/bin/llvm-symbolizer-18: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: no version information available (required by /usr/bin/llvm-symbolizer-18)
 /usr/bin/llvm-symbolizer-18: symbol lookup error: /home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4: undefined symbol: __asan_option_detect_stack_use_after_return
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
 ==28724==WARNING: Can't read from symbolizer at fd 6
     #0 0x55edc7ea2e84  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x1b6e84) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
     #1 0x7f028f6111dd  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x8111dd) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #2 0x7f028f38b8c4  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x58b8c4) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #3 0x7f028f38d60a  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x58d60a) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #4 0x7f028f3796a5  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x5796a5) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #5 0x7f028f11c442  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x31c442) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #6 0x7f028f11bd9c  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x31bd9c) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #7 0x7f028f37891b  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x57891b) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #8 0x7f028f2bb7f3  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x4bb7f3) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #9 0x7f028f28b01d  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x48b01d) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #10 0x7f028f5e3ce4  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x7e3ce4) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #11 0x7f028f5de4c1  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x7de4c1) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     #12 0x7f028f340504  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x540504) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#13 0x7f028f33dba1  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x53dba1) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#14 0x7f028f4f02bd  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x6f02bd) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#15 0x7f028f212a09  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x412a09) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#16 0x7f028f211bae  (/home/runner/work/curl/curl/bld/lib/.libs/libcurl.so.4+0x411bae) (BuildId: 8b44169afc81107024b77ec9a2942da0f6e28bfa)
     curl#17 0x55edc7ea2c73  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x1b6c73) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
     curl#18 0x55edc7f7b832  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x28f832) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
     curl#19 0x7f028dc2a1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 42c84c92e6f98126b3e2230ebfdead22c235b667)
     curl#20 0x7f028dc2a28a  (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 42c84c92e6f98126b3e2230ebfdead22c235b667)
     curl#21 0x55edc7da3a34  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0xb7a34) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
 Address 0x7f028b500060 is located in stack of thread T0 at offset 32 in frame
 ==28724==WARNING: Can't read from symbolizer at fd 6
     #0 0x55edc7ea2717  (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x1b6717) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
   This frame has 1 object(s):
     [32, 36) 'counter' (line 91) <== Memory access at offset 32 is inside this variable
 HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
       (longjmp and C++ exceptions *are* supported)
 ==28724==WARNING: Can't read from symbolizer at fd 6
 SUMMARY: AddressSanitizer: stack-use-after-scope (/home/runner/work/curl/curl/bld/tests/libtest/.libs/libtests+0x1b6e84) (BuildId: 7d9c526eeac6e37274bab929807a3ed93d8fe67c)
 Shadow bytes around the buggy address:
   0x7f028b4ffd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x7f028b4ffe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x7f028b4ffe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x7f028b4fff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x7f028b4fff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 =>0x7f028b500000: f5 f5 f5 f5 f5 f5 f5 f5 f1 f1 f1 f1[f8]f3 f3 f3
   0x7f028b500080: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
   0x7f028b500100: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
   0x7f028b500180: f1 f1 f1 f1 04 f2 f8 f3 f5 f5 f5 f5 f5 f5 f5 f5
   0x7f028b500200: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
   0x7f028b500280: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
 Shadow byte legend (one shadow byte represents 8 application bytes):
   Addressable:           00
   Partially addressable: 01 02 03 04 05 06 07
   Heap left redzone:       fa
   Freed heap region:       fd
   Stack left redzone:      f1
   Stack mid redzone:       f2
   Stack right redzone:     f3
   Stack after return:      f5
   Stack use after scope:   f8
   Global redzone:          f9
   Global init order:       f6
   Poisoned by user:        f7
   Container overflow:      fc
   Array cookie:            ac
   Intra object redzone:    bb
   ASan internal:           fe
   Left alloca redzone:     ca
   Right alloca redzone:    cb
 ==28724==ABORTING
```
https://github.com/curl/curl/actions/runs/15590177615/job/43906987845?pr=17598#step:39:1276
@testclutch

This comment was marked as outdated.

Sign in to view
@vszakats vszakats marked this pull request as ready for review June 11, 2025 16:52
@vszakats vszakats changed the title libtests: try to avoid build-time clone of libtests libtests: stop building the same source multiple times Jun 11, 2025
@vszakats vszakats changed the title libtests: stop building the same source multiple times libtests: stop building the sames source multiple times Jun 11, 2025
@vszakats vszakats closed this in c9460d6 Jun 11, 2025
@vszakats vszakats deleted the t-unclone branch June 11, 2025 17:15
bagder added a commit that referenced this pull request Jun 11, 2025
@bagder
tests: tidy ups
369b5ca 
What remains after #17598
denandz pushed a commit to denandz/curl that referenced this pull request Jun 21, 2025
@vszakats @denandz
libtests: stop building the sames source multiple times
7228aee 
After this patch there is no more double/multiple compile of the same
libtest source under a different libtest ID. Each libtest is compiled
once, and changing behavior at runtime based on test ID.

- drop recently added physical clones for two prevously multi-compiled
  tests:
  - merge lib587 into lib554 again, branch at runtime.
  - merge lib645 into lib643 again, branch at runtime.

- replace existing dynamic branching to use `testnum` instead of
  a manually rolled `testno` based on an extra command-line argument.
  lib1571, lib1576.

- mk-bundle.pl: stop defining `LIB*` macros. No longer used.

- libtests: drop all `LIB*_C` guards.

- Make these tests branch at runtime, stop building copies:
  - lib585, based on lib500
  - lib565, based on lib510
  - lib529, based on lib525
  - lib527, lib532, based on lib526
  - lib545, based on lib544
  - lib548, based on lib547
  - lib696, based on lib556
  - lib584, based on lib589
  - lib1539, based on lib1514
  - lib1543, based on lib1518
  - lib1917, based on lib1916
  - lib1946, based on lib1940
  - lib671, 672, 673, based on lib670

Follow-up to 02dd471 curl#17591
Follow-up to 6897aeb curl#17468

Closes curl#17598
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

tests tidy-up

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vszakats @testclutch