Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CMake build improvements #441

Open
wants to merge 93 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
2a52d46
InitCommit - Update CMakeLists.txt
stefankiesz Feb 28, 2024
f3e3af1
Don't include LWS KVS source files
stefankiesz Feb 28, 2024
b837e0f
Remove redundant line from libcurl-CMakeLists.txt
stefankiesz Feb 28, 2024
998ea4e
Cleanup? libwebsockets-CMakeLists.txt
stefankiesz Feb 28, 2024
cdad372
Working!
stefankiesz Feb 28, 2024
62b31af
Add comments, remove duplicated lines
stefankiesz Feb 28, 2024
2cd7c83
Link top level opensoure libs directory
stefankiesz Feb 28, 2024
394a4a6
Switch more dirs from current to top level
stefankiesz Feb 28, 2024
9c6de41
Remove one of the lib links
stefankiesz Feb 28, 2024
175ead2
Link curl, ssl, and crypto as private
stefankiesz Feb 28, 2024
dbba69f
Some extra reference files to be deleted later
stefankiesz Feb 28, 2024
2ebb24d
Remove curl test code from test main
stefankiesz Feb 28, 2024
0afd3da
Setup choosing between mbedtls and openssl libs
stefankiesz Feb 29, 2024
73dde64
curl->libcurl for windows
stefankiesz Feb 29, 2024
c7d43b0
Add comments
stefankiesz Mar 1, 2024
bb766aa
Trying to fix Windows curl
stefankiesz Mar 8, 2024
65c7c00
Merge branch 'develop' into cmake-cleanup
stefankiesz Mar 8, 2024
c797aa4
Update CMakeLists.txt
stefankiesz Mar 8, 2024
55a4d1b
Update CMakeLists.txt
stefankiesz Mar 9, 2024
9127680
Testing Windows fixes
stefankiesz Mar 9, 2024
f032a13
more
stefankiesz Mar 9, 2024
fe16da2
Undo Windows fix that wasn't it
stefankiesz Mar 14, 2024
e6fa97d
Bring in LWS building, not tested
stefankiesz Mar 14, 2024
8881061
Allow for turning off of sample builds
stefankiesz Mar 14, 2024
2fb90e0
Build local curl for windows
stefankiesz Mar 14, 2024
275c578
Change openssl libs names
stefankiesz Mar 14, 2024
c3be70d
Same for curl
stefankiesz Mar 14, 2024
ff9f4ec
Remove unused variables
stefankiesz Mar 14, 2024
27a2610
Print CMAKE_BUILD_TYPE
stefankiesz Mar 14, 2024
3a71f9f
Add open-source/bin to link_directories
stefankiesz Mar 14, 2024
54d116d
Set whole build to static for Windows
stefankiesz Mar 15, 2024
0e19b5e
Fix syntax
stefankiesz Mar 15, 2024
dd65e53
Shorten path for windows CI build
stefankiesz Mar 15, 2024
0247b51
cd into new path
stefankiesz Mar 15, 2024
78e2970
Trying with "curl-d"
stefankiesz Mar 15, 2024
9330277
more
stefankiesz Mar 15, 2024
98cf950
more
stefankiesz Mar 15, 2024
fa02b3f
Print link dirs
stefankiesz Mar 15, 2024
a0946bb
Brute force linking working
stefankiesz Mar 16, 2024
2fe1471
Separate into conditional
stefankiesz Mar 16, 2024
709e3ba
Switch order of linked libs to fix static build
stefankiesz Mar 17, 2024
6bc18f3
Only build with crypto if building common curl
stefankiesz Mar 18, 2024
e7fbe19
Undo
stefankiesz Mar 18, 2024
49d8af8
more
stefankiesz Mar 18, 2024
c41dacf
Set OPEN_SRC_INCLUDE_DIRS regardless of BUILD_DEPENDENCIES
stefankiesz Mar 18, 2024
37ad49d
Testing with find_package removed
stefankiesz Mar 18, 2024
bb2004b
Link "libwebsockets"
stefankiesz Mar 18, 2024
9ba15b2
more testing
stefankiesz Mar 18, 2024
78836b5
more
stefankiesz Mar 18, 2024
51eebb4
Add in the crypto libs for commonLWS
stefankiesz Mar 18, 2024
4113240
more
stefankiesz Mar 19, 2024
9e26a85
more
stefankiesz Mar 19, 2024
cb83157
more
stefankiesz Mar 19, 2024
0295cea
Reverting changes to libwebsockets-cmakelists.txt
stefankiesz Mar 19, 2024
f04658d
more
stefankiesz Mar 19, 2024
3bec97c
more
stefankiesz Mar 19, 2024
0464ba2
Add logs
stefankiesz Mar 19, 2024
e7df7c0
develop
stefankiesz Mar 19, 2024
0d2ab42
more
stefankiesz Mar 19, 2024
8d6a48c
Removing this line to verify this was the issue
stefankiesz Mar 20, 2024
f47cdbe
Revert to originally working version with some changes kept
stefankiesz Mar 20, 2024
77adbf3
Try fix EVP_sha256 undefined
stefankiesz Mar 20, 2024
a758022
works, remove hardcode openssl
stefankiesz Mar 20, 2024
0a60e7b
Should all work now...
stefankiesz Mar 21, 2024
60647df
Trying to fix Windows runtime errors
stefankiesz Mar 21, 2024
9e5140a
Revert "Trying to fix Windows runtime errors"
stefankiesz Mar 21, 2024
d3afb1c
Add comments, cleanup
stefankiesz Mar 21, 2024
be81fba
More commenting and cleanup
stefankiesz Mar 22, 2024
4d27f7a
comment
stefankiesz Mar 22, 2024
e69543f
Cleanup setting of linker paths for Windows
stefankiesz Mar 23, 2024
93993b0
Remove unlinking of openssl from CI tests
stefankiesz Mar 23, 2024
129262a
Specify openssl build path when from built from source, tested static…
stefankiesz Mar 23, 2024
f952306
Switch local openssl test to local everything
stefankiesz Mar 23, 2024
a392dc0
Unset DYLD_LIBRARY_PATH
stefankiesz Mar 23, 2024
69c3758
Print DYLD_LIBRARY_PATH
stefankiesz Mar 23, 2024
5dd013b
Set openssl root directory for curl to find
stefankiesz Mar 23, 2024
e7b922e
Trying more
stefankiesz Mar 23, 2024
18f5bde
more
stefankiesz Mar 23, 2024
85f6d8b
more
stefankiesz Mar 23, 2024
3946c18
more
stefankiesz Mar 23, 2024
1b61699
more
stefankiesz Mar 23, 2024
2ba56c9
more
stefankiesz Mar 24, 2024
269630c
more
stefankiesz Mar 24, 2024
a5a535f
printing all openssl find_package vars
stefankiesz Mar 24, 2024
38424d9
Switch my curl fork
stefankiesz Mar 24, 2024
9b9be56
Isolate job, verify unlink openssl works
stefankiesz Mar 24, 2024
6432c50
Try setting dyld lib path
stefankiesz Mar 24, 2024
eb20224
Revert "Remove unlinking of openssl from CI tests"
stefankiesz Mar 25, 2024
3278951
Remove setting of openssldir
stefankiesz Mar 25, 2024
0b27c82
Tested BUILD_DEPS off for common curl
stefankiesz Mar 25, 2024
a9541a1
Update CI tests
stefankiesz Mar 25, 2024
aa8ea9e
Install GTest
stefankiesz Mar 25, 2024
58da8fe
Add tests
stefankiesz Mar 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/build_windows.bat
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
REM NOTE: May need to replace "Enterprise" with "Community" int the below command
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: int -> in

call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
mkdir build
cd build
Expand Down
91 changes: 71 additions & 20 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:
- master
jobs:
clang-format-check:
runs-on: macos-11
runs-on: macos-latest
steps:
- name: Clone repository
uses: actions/checkout@v3
Expand All @@ -24,20 +24,19 @@ jobs:
bash scripts/check-clang.sh

mac-os-build-gcc:
runs-on: macos-12
runs-on: macos-latest
permissions:
id-token: write
contents: read
env:
CC: /usr/local/bin/gcc-13
CXX: /usr/local/bin/g++-13
CC: gcc-13
CXX: g++-13
AWS_KVS_LOG_LEVEL: 2
steps:
- name: Clone repository
uses: actions/checkout@v3
- name: Build repository
run: |
brew install pkgconfig
brew unlink openssl # it seems the libcurl is trying to access this openssl despite explicitly setting it to our build
mkdir build && cd build
cmake .. --trace -DBUILD_TEST=TRUE
Expand All @@ -55,7 +54,7 @@ jobs:
./tst/producer_test

mac-os-build-clang:
runs-on: macos-11
runs-on: macos-latest
env:
AWS_KVS_LOG_LEVEL: 2
permissions:
Expand All @@ -66,7 +65,6 @@ jobs:
uses: actions/checkout@v3
- name: Build repository
run: |
brew install pkgconfig
brew unlink openssl # it seems the libcurl is trying to access this openssl despite explicitly setting it to our build
mkdir build && cd build
cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE
Expand Down Expand Up @@ -95,10 +93,9 @@ jobs:
uses: actions/checkout@v3
- name: Build repository
run: |
brew install pkgconfig
brew unlink openssl # it seems the libcurl is trying to access this openssl despite explicitly setting it to our build
mkdir build && cd build
sh -c 'cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE -DCMAKE_C_COMPILER=$(brew --prefix llvm@15)/bin/clang -DCMAKE_CXX_COMPILER=$(brew --prefix llvm@15)/bin/clang++;cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE -DCMAKE_C_COMPILER=$(brew --prefix llvm@15)/bin/clang -DCMAKE_CXX_COMPILER=$(brew --prefix llvm@15)/bin/clang++'
cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE -DCMAKE_C_COMPILER=$(brew --prefix llvm@15)/bin/clang -DCMAKE_CXX_COMPILER=$(brew --prefix llvm@15)/bin/clang++
make
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1-node16
Expand All @@ -112,14 +109,14 @@ jobs:
cd build
./tst/producer_test

mac-os-build-gcc-local-openssl:
runs-on: macos-11
mac-os-build-gcc-system-openssl:
runs-on: macos-latest
permissions:
id-token: write
contents: read
env:
CC: /usr/local/bin/gcc-13
CXX: /usr/local/bin/g++-13
CC: gcc-13
CXX: g++-13
AWS_KVS_LOG_LEVEL: 2
LDFLAGS: -L/usr/local/opt/openssl@3/lib
CPPFLAGS: -I/usr/local/opt/openssl@3/include
Expand All @@ -131,7 +128,7 @@ jobs:
run: |
brew info openssl
mkdir build && cd build
cmake .. -DBUILD_TEST=TRUE -DLOCAL_OPENSSL_BUILD=ON
cmake .. -DBUILD_TEST=TRUE -DBUILD_CRYPTO=FALSE
make
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1-node16
Expand All @@ -145,7 +142,7 @@ jobs:
cd build
./tst/producer_test

mac-os-build-clang-local-openssl:
mac-os-build-clang-system-openssl:
runs-on: macos-latest
env:
AWS_KVS_LOG_LEVEL: 2
Expand All @@ -160,9 +157,10 @@ jobs:
uses: actions/checkout@v3
- name: Build repository
run: |
brew install googletest
brew info openssl
mkdir build && cd build
cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE -DLOCAL_OPENSSL_BUILD=ON
cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE -DBUILD_CRYPTO=FALSE
make
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1-node16
Expand All @@ -176,6 +174,52 @@ jobs:
cd build
./tst/producer_test

# Producer is not compatible with latest CURL, so build only, don't run tests.
mac-os-build-gcc-system-deps:
runs-on: macos-latest
permissions:
id-token: write
contents: read
env:
CC: gcc-13
CXX: g++-13
AWS_KVS_LOG_LEVEL: 2
LDFLAGS: -L/usr/local/opt/openssl@3/lib
CPPFLAGS: -I/usr/local/opt/openssl@3/include
OPENSSL_ROOT_DIR: /usr/local/opt/openssl@3/
steps:
- name: Clone repository
uses: actions/checkout@v3
- name: Build repository
run: |
brew install googletest
brew info openssl
mkdir build && cd build
cmake .. -DBUILD_TEST=TRUE -DBUILD_DEPENDENCIES=FALSE
make

# Producer is not compatible with latest CURL, so build only, don't run tests.
mac-os-build-clang-system-deps:
runs-on: macos-latest
env:
AWS_KVS_LOG_LEVEL: 2
LDFLAGS: -L/usr/local/opt/openssl@3/lib
CPPFLAGS: -I/usr/local/opt/openssl@3/include
OPENSSL_ROOT_DIR: /usr/local/opt/openssl@3/
permissions:
id-token: write
contents: read
steps:
- name: Clone repository
uses: actions/checkout@v3
- name: Build repository
run: |
brew install googletest
brew info openssl
mkdir build && cd build
cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE -DBUILD_DEPENDENCIES=FALSE
make

ubuntu-os-gcc-build-lws-mbedtls:
runs-on: ubuntu-20.04
env:
Expand Down Expand Up @@ -391,13 +435,18 @@ jobs:
steps:
- name: Clone repository
uses: actions/checkout@v3
- name: Move cloned repo # to shorten path length for Windows OS
run: |
git config --system core.longpaths true
mkdir C:\amazon-kinesis-video-streams-producer-c
Move-Item -Path "D:\a\amazon-kinesis-video-streams-producer-c\amazon-kinesis-video-streams-producer-c\*" -Destination "C:\amazon-kinesis-video-streams-producer-c"
- name: Install dependencies
run: |
choco install nasm strawberryperl
- name: Build repository
run: |
$env:Path += ';C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Strawberry\c\bin;C:\Program Files\NASM;D:\a\amazon-kinesis-video-streams-producer-c\amazon-kinesis-video-streams-producer-c\open-source\lib;D:\a\amazon-kinesis-video-streams-producer-c\amazon-kinesis-video-streams-producer-c\open-source\bin'
git config --system core.longpaths true
$env:Path += ';C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Strawberry\c\bin;C:\Program Files\NASM;C:\amazon-kinesis-video-streams-producer-c\open-source\lib;C:\amazon-kinesis-video-streams-producer-c\open-source\bin'
cd C:\amazon-kinesis-video-streams-producer-c
.github/build_windows.bat
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1-node16
Expand All @@ -408,8 +457,8 @@ jobs:
role-duration-seconds: 10800
- name: Run tests
run: |
$env:Path += ';C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Strawberry\c\bin;C:\Program Files\NASM;D:\a\amazon-kinesis-video-streams-producer-c\amazon-kinesis-video-streams-producer-c\open-source\lib;D:\a\amazon-kinesis-video-streams-producer-c\amazon-kinesis-video-streams-producer-c\open-source\bin'
& "D:\a\amazon-kinesis-video-streams-producer-c\amazon-kinesis-video-streams-producer-c\build\tst\producer_test.exe" --gtest_filter="-ProducerFunctionalityTest.pressure_on_buffer_duration_fail_new_connection_at_token_rotation"
$env:Path += ';C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Strawberry\c\bin;C:\Program Files\NASM;C:\amazon-kinesis-video-streams-producer-c\open-source\lib;C:\amazon-kinesis-video-streams-producer-c\open-source\bin'
& "C:\amazon-kinesis-video-streams-producer-c\build\tst\producer_test.exe" --gtest_filter="-ProducerFunctionalityTest.pressure_on_buffer_duration_fail_new_connection_at_token_rotation"

arm64-cross-compilation:
runs-on: ubuntu-20.04
Expand All @@ -430,6 +479,7 @@ jobs:
cmake .. -DBUILD_TEST=TRUE -DBUILD_OPENSSL_PLATFORM=linux-generic64
make
file libcproducer.so

linux-aarch64-cross-compilation:
runs-on: ubuntu-20.04
env:
Expand All @@ -449,6 +499,7 @@ jobs:
cmake .. -DBUILD_TEST=TRUE -DBUILD_OPENSSL_PLATFORM=linux-aarch64
make
file libcproducer.so

arm32-cross-compilation:
runs-on: ubuntu-20.04
env:
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ cmake-build-release/
open-source
outputs
dependency/
cmake-build-producerc/
cmake-build-producerc/
.vscode/settings.json
2 changes: 0 additions & 2 deletions CMake/Dependencies/libcurl-CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ cmake_minimum_required(VERSION 3.6.3)

project(libcurl-download LANGUAGES C)

find_program(MAKE_EXE NAMES make)

if (DEFINED CMAKE_OSX_SYSROOT AND NOT CMAKE_OSX_SYSROOT STREQUAL "")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -isysroot${CMAKE_OSX_SYSROOT}")
endif()
Expand Down
4 changes: 2 additions & 2 deletions CMake/Dependencies/libopenssl-CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ else()
endif()

if (DEFINED BUILD_OPENSSL_PLATFORM AND NOT BUILD_OPENSSL_PLATFORM STREQUAL OFF)
SET(CONFIGURE_COMMAND ${CMAKE_CURRENT_BINARY_DIR}/build/src/project_libopenssl/Configure ${OPENSSL_EXTRA} --prefix=${OPEN_SRC_INSTALL_PREFIX} --openssldir=${OPEN_SRC_INSTALL_PREFIX} ${BUILD_OPENSSL_PLATFORM} -Wno-nullability-completeness -Wno-expansion-to-defined)
SET(CONFIGURE_COMMAND ${CMAKE_CURRENT_BINARY_DIR}/build/src/project_libopenssl/Configure ${OPENSSL_EXTRA} --prefix=${OPEN_SRC_INSTALL_PREFIX} ${BUILD_OPENSSL_PLATFORM} -Wno-nullability-completeness -Wno-expansion-to-defined)
else()
SET(CONFIGURE_COMMAND ${CMAKE_CURRENT_BINARY_DIR}/build/src/project_libopenssl/config ${OPENSSL_EXTRA} --prefix=${OPEN_SRC_INSTALL_PREFIX} --openssldir=${OPEN_SRC_INSTALL_PREFIX} -Wno-nullability-completeness -Wno-expansion-to-defined)
SET(CONFIGURE_COMMAND ${CMAKE_CURRENT_BINARY_DIR}/build/src/project_libopenssl/config ${OPENSSL_EXTRA} --prefix=${OPEN_SRC_INSTALL_PREFIX} -Wno-nullability-completeness -Wno-expansion-to-defined)
endif()
endif()

Expand Down
2 changes: 1 addition & 1 deletion CMake/Dependencies/libwebsockets-CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,4 @@ ExternalProject_Add(project_libwebsockets
-DOPENSSL_ROOT_DIR=${OPENSSL_DIR}
BUILD_ALWAYS TRUE
TEST_COMMAND ""
)
)
Loading
Loading