Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmake: add and test "unity" builds #1034

Closed
wants to merge 1 commit into from
Closed

cmake: add and test "unity" builds #1034

wants to merge 1 commit into from

Conversation

vszakats
Copy link
Member

@vszakats vszakats commented May 7, 2023 •
edited

"Unity" (aka "jumbo", aka "amalgamation" builds concatenate source files
before compiling. It has these benefits for example: faster builds,
improved code optimization, cleaner code. Let's support and test this.

  • enable unity builds for some existing CI builds to test this build
    scenario.
  • tune UNITY_BUILD_BATCH_SIZE size.
  • disable unity build for example and test programs (they use one source
    each already).

You can enable it by passing -DCMAKE_UNITY_BUILD=ON to cmake.
Supported by CMake 3.16 and newer.

Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html

Closes #1034

@vszakats
cmake: add and test unity builds
98b93db 
"Unity" builds concatenate sources before compiling. Has some useful
effects (faster builds, improves code optimization, cleaner code).
Let's support and test this.

- enable UNITY_BUILD for a few existing CI builds to test this build scenario.
- tune `UNITY_BUILD_BATCH_SIZE` size.
- disable UNITY_BUILD for example and test programs
  (they use one source each).

Supported by CMake 3.16 and newer.

Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html

Closes libssh2#1034
@vszakats vszakats added the build label May 7, 2023
@vszakats vszakats changed the title cmake: add and test unity builds cmake: add and test "unity" builds May 7, 2023
@vszakats vszakats closed this in 7129ea9 May 7, 2023
@vszakats vszakats deleted the cmake-unity branch May 7, 2023 10:05
@vszakats vszakats mentioned this pull request May 7, 2023
Closed
vszakats added a commit to curl/curl-for-win that referenced this pull request May 7, 2023
@vszakats
libssh2-cmake.sh: enable CMAKE_UNITY_BUILD with next version [ci skip]
9f3ffa6 
Ref:
libssh2/libssh2@7129ea9
libssh2/libssh2#1034
@vszakats vszakats mentioned this pull request May 9, 2023
Closed
vszakats added a commit to vszakats/curl that referenced this pull request May 10, 2023
@vszakats
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
103a254 
per target as a single C source.

Can be enabled by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.

It makes builds (much) faster and allows for better optimization.

Fix related issues:

- add missing include guard to `easy_lock.h`.

- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.

- add an `#undef` after use.

- add a missing `#undef` before use.

- move internal definitions from `ftp.h` to `ftp.c`.

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
vszakats added a commit to vszakats/curl that referenced this pull request May 19, 2023
@vszakats
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
60ebcba 
per target as a single C source.

Can be enabled by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.

It makes builds (much) faster and allows for better optimization.

Fix related issues:

- add missing include guard to `easy_lock.h`.

- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.

- add an `#undef` after use.

- add a missing `#undef` before use.

- move internal definitions from `ftp.h` to `ftp.c`.

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
vszakats added a commit to vszakats/curl that referenced this pull request May 20, 2023
@vszakats
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
46ccabd 
per target as a single C source.

Can be enabled by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.

It makes builds (much) faster and allows for better optimization.

Fix related issues:

- add missing include guard to `easy_lock.h`.

- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.

- add an `#undef` after use.

- add a missing `#undef` before use.

- move internal definitions from `ftp.h` to `ftp.c`.

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
vszakats added a commit to vszakats/curl that referenced this pull request May 25, 2023
@vszakats
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
bb2f4b9 
per target as a single C source.

Can be enabled by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.

It makes builds (much) faster and allows for better optimization.

Fix related issues:

- add missing include guard to `easy_lock.h`.

- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.

- add an `#undef` after use.

- add a missing `#undef` before use.

- move internal definitions from `ftp.h` to `ftp.c`.

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
vszakats added a commit to vszakats/curl that referenced this pull request May 25, 2023
@vszakats
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
916adef 
per target as a single C source.

Can be enabled by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.

It makes builds (much) faster and allows for better optimization.

Fix related issues:

- add missing include guard to `easy_lock.h`.

- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.

- add an `#undef` after use.

- add a missing `#undef` before use.

- move internal definitions from `ftp.h` to `ftp.c`.

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
vszakats added a commit to vszakats/curl that referenced this pull request May 26, 2023
@vszakats
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
8a4c134 
per target as a single C source.

Can be enabled by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.

It makes builds (much) faster and allows for better optimization.

Fix related issues:

- add missing include guard to `easy_lock.h`.

- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.

- add an `#undef` after use.

- add a missing `#undef` before use.

- move internal definitions from `ftp.h` to `ftp.c`.

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
vszakats added a commit to vszakats/curl that referenced this pull request Jun 4, 2023
@vszakats
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
15ebe35 
per target as a single C source.

Can be enabled by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.

It makes builds (much) faster and allows for better optimization.

Fix related issues:

- add missing include guard to `easy_lock.h`.

- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.

- add an `#undef` after use.

- add a missing `#undef` before use.

- move internal definitions from `ftp.h` to `ftp.c`.

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
vszakats added a commit to curl/curl that referenced this pull request Jun 7, 2023
@vszakats
cmake: add support for "unity" builds
3f8fc25 
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
bch pushed a commit to bch/curl that referenced this pull request Jul 19, 2023
@vszakats @bch
cmake: add support for "unity" builds
1e3293e 
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
ptitSeb pushed a commit to wasix-org/curl that referenced this pull request Sep 25, 2023
@vszakats @ptitSeb
cmake: add support for "unity" builds
97a2d4e 
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: libssh2/libssh2#1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes curl#11095
vszakats added a commit to vszakats/libssh2 that referenced this pull request Apr 14, 2024
@vszakats
ci: enable Unity mode for most CMake builds
db8e0ec 
Ref: 7129ea9 libssh2#1034
Closes #xxxx
@vszakats vszakats mentioned this pull request Apr 14, 2024
Closed
vszakats added a commit that referenced this pull request Apr 14, 2024
@vszakats
ci: enable Unity mode for most CMake builds
1bfae57 
Ref: 7129ea9 #1034
Closes #1367
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
build
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@vszakats