Skip to content

Download certs into demos/certificates with an option to disable downloading #1126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Aug 14, 2020
Merged

Download certs into demos/certificates with an option to disable downloading #1126

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
3a5d394
Add option to turn off downloading of certificates and create demos/c…
yourslab Aug 13, 2020
f310caa
Merge branch 'development' into cmake/download-certs-option
yourslab Aug 13, 2020
79c4882
Remove Mosquitto certificate from list of certs to download
yourslab Aug 13, 2020
e9d7031
Merge branch 'cmake/download-certs-option' of github.com:yourslab/aws…
yourslab Aug 13, 2020
80fd8f0
Address PR comments
yourslab Aug 14, 2020
e72910e
Add CMake command line options to use for configuring demos
yourslab Aug 14, 2020
6167e28
Update README.md to contain extra flag for ROOT_CA_CERT_PATH
yourslab Aug 14, 2020
46079ac
Merge branch 'development' into cmake/download-certs-option
yourslab Aug 14, 2020
170cc64
Set prefix to PWD if any path flags are relative
yourslab Aug 14, 2020
c0ed226
Merge branch 'cmake/download-certs-option' of github.com:yourslab/aws…
yourslab Aug 14, 2020
e3fbbe0
Merge branch 'development' into cmake/download-certs-option
yourslab Aug 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ option( BUILD_TESTS
option( BUILD_CLONE_SUBMODULES
"Set this to ON to automatically clone any required Git submodules. When OFF, submodules must be manually cloned."
ON )
option( DOWNLOAD_CERTS
"Set this to ON to automatically download certificates needed to run the demo. When OFF, certificates must be manually downloaded."
ON )

# Unity test framework does not export the correct symbols for DLLs.
set( ALLOW_SHARED_LIBRARIES ON )
Expand Down
7 changes: 5 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,12 @@ It is required to setup an AWS account and access the AWS IoT Console for runnin

### Configuring the mutual auth demos

- You can pass the following configuration settings as command line options in order to run the mutual auth demos: `cmake .. -DAWS_IOT_ENDPOINT="aws-iot-endpoint" -DCLIENT_CERT_PATH="certificate-path" -DCLIENT_PRIVATE_KEY_PATH="private-key-path"`
- You can pass the following configuration settings as command line options in order to run the mutual auth demos:
```bash
cmake .. -DAWS_IOT_ENDPOINT="aws-iot-endpoint" -DROOT_CA_CERT_PATH="root-ca-path" -DCLIENT_CERT_PATH="certificate-path" -DCLIENT_PRIVATE_KEY_PATH="private-key-path"
```

- In order to set these settings manually, edit `demo_config.h` in `demos/mqtt/mqtt_demo_mutual_auth/` and `demos/http/http_demo_mutual_auth/` to `#define` the following:
- In order to set these configurations manually, edit `demo_config.h` in `demos/mqtt/mqtt_demo_mutual_auth/` and `demos/http/http_demo_mutual_auth/` to `#define` the following:

- Set `AWS_IOT_ENDPOINT` to your custom endpoint. This is found on the *Settings* page of the AWS IoT Console and has a format of `ABCDEFG1234567.iot.us-east-2.amazonaws.com`.

Expand Down
32 changes: 31 additions & 1 deletion demos/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,41 @@ function(check_aws_credentials demo_name)
endif()
endfunction()

if(DOWNLOAD_CERTS)
# Download the Amazon Root CA certificate.
message( "Downloading the Amazon Root CA certificate..." )
file(MAKE_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/certificates)
execute_process(
COMMAND curl --url https://www.amazontrust.com/repository/AmazonRootCA1.pem
-o ${CMAKE_CURRENT_LIST_DIR}/certificates/AmazonRootCA1.crt
)
endif()

# Copy the certificates and client keys to the build directory.
file(COPY "${CMAKE_CURRENT_LIST_DIR}/certificates"
DESTINATION "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}")
if(BUILD_TESTS)
file(COPY "${CMAKE_CURRENT_LIST_DIR}/certificates"
DESTINATION "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/tests")
endif()

# Set prefix to PWD if any path flags are relative
if(DEFINED ENV{PWD})
if(NOT ROOT_CA_CERT_PATH MATCHES "/$")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if these variables are not passed in the CMake command in which case they would (probably) have empty values?
Would these generate invalid variables for the credential variables in that case?

Copy link

@aggarw13 aggarw13 Aug 14, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the mqtt_demo_basic_tls locally with not providing the ROOT_CA_CERT_PATH with the following command:
cmake .. -DBUILD_TESTS="ON" -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_FLAGS='-Wextra -Wall -O0 -ggdb'

The build logs show that the absolute path logic is causing the ROOT_CA_CERT_PATH CMake variable to be defined even though it wasn't passed by me, and thus, it is building the demo target with an incorrect value (of /home/ubuntu/Repos/aws-iot-device-sdk-embedded-C/build/ as can be seen in the logs)

╭─ubuntu@ip-172-31-24-209 ~/Repos/aws-iot-device-sdk-embedded-C/build  ‹development*› 
╰─➤  ./bin/mqtt_demo_basic_tls         
[INFO] [DEMO] [mqtt_demo_basic_tls.c:400] Establishing a TLS session to test.mosquitto.org:8883.
[DEBUG] [Sockets] [sockets_posix.c:170] Performing DNS lookup: Host=test.mosquitto.org.
[DEBUG] [Sockets] [sockets_posix.c:210] Attempting to connect to server: Host=test.mosquitto.org, IP address=5.196.95.208.
[DEBUG] [Sockets] [sockets_posix.c:225] Connected to IP address: 5.196.95.208.
[DEBUG] [Sockets] [sockets_posix.c:256] Established TCP connection: Server=test.mosquitto.org.

[DEBUG] [Transport_OpenSSL_Sockets] [openssl_posix.c:144] Attempting to open Root CA certificate: Path=/home/ubuntu/Repos/aws-iot-device-sdk-embedded-C/build/.
[ERROR] [Transport_OpenSSL_Sockets] [openssl_posix.c:195] PEM_read_X509 failed to parse root CA.
[ERROR] [Transport_OpenSSL_Sockets] [openssl_posix.c:474] Setting up credentials failed.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix is simple by just updating each of the nested if conditions to check if their respective CMake variables are defined

set(ROOT_CA_CERT_PATH "$ENV{PWD}/${ROOT_CA_CERT_PATH}")
endif()
if(NOT CLIENT_CERT_PATH MATCHES "/$")
set(CLIENT_CERT_PATH "$ENV{PWD}/${CLIENT_CERT_PATH}")
endif()
if(NOT CLIENT_PRIVATE_KEY_PATH MATCHES "/$")
set(CLIENT_PRIVATE_KEY_PATH "$ENV{PWD}/${CLIENT_PRIVATE_KEY_PATH}")
endif()
endif()

# Include each subdirectory that has a CMakeLists.txt file in it
file(GLOB demo_dirs "${DEMOS_DIR}/*/*")
foreach(demo_dir IN LISTS demo_dirs)
if(IS_DIRECTORY "${demo_dir}" AND EXISTS "${demo_dir}/CMakeLists.txt")
get_filename_component( DEMO_EXE_NAME ${demo_dir} NAME )
add_subdirectory(${demo_dir})
endif()
endforeach()
0 ..._demo_mutual_auth/certificates/.gitignore → demos/certificates/.gitignore
View file
Open in desktop
File renamed without changes.
29 changes: 12 additions & 17 deletions demos/http/http_demo_basic_tls/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,15 @@ target_include_directories(
${LOGGING_INCLUDE_DIRS}
)

# Download the Amazon Root CA certificate.
message( "Downloading the Amazon Root CA certificate..." )
file(MAKE_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/certificates)
execute_process(
COMMAND curl --url https://www.amazontrust.com/repository/AmazonRootCA1.pem
-o ${CMAKE_CURRENT_LIST_DIR}/certificates/AmazonRootCA1.crt
)

# Copy the certificates and client key to the binary directory.
add_custom_command(
TARGET
${DEMO_NAME}
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${CMAKE_CURRENT_LIST_DIR}/certificates"
"$<TARGET_FILE_DIR:${DEMO_NAME}>/certificates"
)
if(ROOT_CA_CERT_PATH)
target_compile_definitions(
${DEMO_NAME} PRIVATE
ROOT_CA_CERT_PATH="${ROOT_CA_CERT_PATH}"
)
endif()
if(SERVER_HOST)
target_compile_definitions(
${DEMO_NAME} PRIVATE
SERVER_HOST="${SERVER_HOST}"
)
endif()
10 changes: 7 additions & 3 deletions demos/http/http_demo_basic_tls/demo_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,21 +54,25 @@
*
* @note This demo uses httpbin.org: A simple HTTP Request & Response Service.
*/
#define SERVER_HOST "httpbin.org"
#ifndef SERVER_HOST
#define SERVER_HOST "httpbin.org"
#endif

/**
* @brief HTTP server port number.
*
* In general, port 443 is for TLS HTTP connections.
*/
#define SERVER_PORT 443
#define SERVER_PORT 443

/**
* @brief Path of the file containing the server's root CA certificate for TLS authentication.
*
* @note This certificate should be PEM-encoded.
*/
#define ROOT_CA_CERT_PATH "certificates/AmazonRootCA1.crt"
#ifndef ROOT_CA_CERT_PATH
#define ROOT_CA_CERT_PATH "certificates/AmazonRootCA1.crt"
#endif

/**
* @brief Paths for different HTTP methods for specified host.
Expand Down
23 changes: 6 additions & 17 deletions demos/http/http_demo_mutual_auth/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,9 @@ target_include_directories(
${LOGGING_INCLUDE_DIRS}
)

# Download the Amazon Root CA certificate
message( "Downloading the Amazon Root CA certificate..." )
file(MAKE_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/certificates)
execute_process(
COMMAND curl --url https://www.amazontrust.com/repository/AmazonRootCA1.pem
-o ${CMAKE_CURRENT_LIST_DIR}/certificates/AmazonRootCA1.crt
)

# Copy the certificates and client key to the binary directory.
add_custom_command(
TARGET
${DEMO_NAME}
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${CMAKE_CURRENT_LIST_DIR}/certificates"
"$<TARGET_FILE_DIR:${DEMO_NAME}>/certificates"
)
if(ROOT_CA_CERT_PATH)
target_compile_definitions(
${DEMO_NAME} PRIVATE
ROOT_CA_CERT_PATH="${ROOT_CA_CERT_PATH}"
)
endif()
6 changes: 4 additions & 2 deletions demos/http/http_demo_mutual_auth/demo_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
* @note Port 443 requires use of the ALPN TLS extension with the ALPN protocol
* name being x-amzn-http-ca. When using port 8443, ALPN is not required.
*/
#define AWS_IOT_PORT 443
#define AWS_IOT_PORT 443

/**
* @brief Path of the file containing Amazon's root CA certificate for TLS
Expand All @@ -78,7 +78,9 @@
*
* @note This certificate should be PEM-encoded.
*/
#define ROOT_CA_CERT_PATH "certificates/AmazonRootCA1.crt"
#ifndef ROOT_CA_CERT_PATH
#define ROOT_CA_CERT_PATH "certificates/AmazonRootCA1.crt"
#endif

/**
* @brief ALPN protocol name to be sent as part of the ClientHello message.
Expand Down
7 changes: 7 additions & 0 deletions demos/http/http_demo_plaintext/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,10 @@ target_include_directories(
PUBLIC
${LOGGING_INCLUDE_DIRS}
)

if(SERVER_HOST)
target_compile_definitions(
${DEMO_NAME} PRIVATE
SERVER_HOST="${SERVER_HOST}"
)
endif()
4 changes: 3 additions & 1 deletion demos/http/http_demo_plaintext/demo_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@
*
* @note This demo uses httpbin.org: A simple HTTP Request & Response Service.
*/
#define SERVER_HOST "httpbin.org"
#ifndef SERVER_HOST
#define SERVER_HOST "httpbin.org"
#endif

/**
* @brief HTTP server port number.
Expand Down
35 changes: 18 additions & 17 deletions demos/mqtt/mqtt_demo_basic_tls/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,21 @@ target_include_directories(
${LOGGING_INCLUDE_DIRS}
)

# Download the Mosquitto Root CA certificate.
message( "Downloading the Mosquitto Root CA certificate..." )
file(MAKE_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/certificates)
execute_process(
COMMAND curl --url https://test.mosquitto.org/ssl/mosquitto.org.crt
-o ${CMAKE_CURRENT_LIST_DIR}/certificates/mosquitto.org.crt
)

# Copy the server certificate file to the binary directory.
add_custom_command(
TARGET
${DEMO_NAME}
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${CMAKE_CURRENT_LIST_DIR}/certificates/mosquitto.org.crt"
"$<TARGET_FILE_DIR:${DEMO_NAME}>/certificates/mosquitto.org.crt"
)
if(ROOT_CA_CERT_PATH)
target_compile_definitions(
${DEMO_NAME} PRIVATE
ROOT_CA_CERT_PATH="${ROOT_CA_CERT_PATH}"
)
endif()
if(BROKER_ENDPOINT)
target_compile_definitions(
${DEMO_NAME} PRIVATE
BROKER_ENDPOINT="${BROKER_ENDPOINT}"
)
endif()
if(CLIENT_IDENTIFIER)
target_compile_definitions(
${DEMO_NAME} PRIVATE
CLIENT_IDENTIFIER="${CLIENT_IDENTIFIER}"
)
endif()
4 changes: 3 additions & 1 deletion demos/mqtt/mqtt_demo_basic_tls/demo_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@
*
* No two clients may use the same client identifier simultaneously.
*/
#define CLIENT_IDENTIFIER "testclient"
#ifndef CLIENT_IDENTIFIER
#define CLIENT_IDENTIFIER "testclient"
#endif

#endif /* ifndef DEMO_CONFIG_H */
13 changes: 13 additions & 0 deletions demos/mqtt/mqtt_demo_lightweight/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,16 @@ target_include_directories(
${CMAKE_CURRENT_LIST_DIR}
${LOGGING_INCLUDE_DIRS}
)

if(BROKER_ENDPOINT)
target_compile_definitions(
${DEMO_NAME} PRIVATE
BROKER_ENDPOINT="${BROKER_ENDPOINT}"
)
endif()
if(CLIENT_IDENTIFIER)
target_compile_definitions(
${DEMO_NAME} PRIVATE
CLIENT_IDENTIFIER="${CLIENT_IDENTIFIER}"
)
endif()
4 changes: 3 additions & 1 deletion demos/mqtt/mqtt_demo_lightweight/demo_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@
*
* No two clients may use the same client identifier simultaneously.
*/
#define CLIENT_IDENTIFIER "testclient"
#ifndef CLIENT_IDENTIFIER
#define CLIENT_IDENTIFIER "testclient"
#endif

#endif /* ifndef DEMO_CONFIG_H */
35 changes: 18 additions & 17 deletions demos/mqtt/mqtt_demo_mutual_auth/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,20 +35,21 @@ target_include_directories(
${LOGGING_INCLUDE_DIRS}
)

# Download the Amazon Root CA certificate.
message( "Downloading the Amazon Root CA certificate..." )
file(MAKE_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/certificates)
execute_process(
COMMAND curl --url https://www.amazontrust.com/repository/AmazonRootCA1.pem
-o ${CMAKE_CURRENT_LIST_DIR}/certificates/AmazonRootCA1.crt
)

# Copy the certificates and client key to the binary directory.
add_custom_command(
TARGET
${DEMO_NAME}
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${CMAKE_CURRENT_LIST_DIR}/certificates"
"$<TARGET_FILE_DIR:${DEMO_NAME}>/certificates"
)
if(ROOT_CA_CERT_PATH)
target_compile_definitions(
${DEMO_NAME} PRIVATE
ROOT_CA_CERT_PATH="${ROOT_CA_CERT_PATH}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if the file path is passed as a cmake flag, will it still work properly from any directory if the given flag is a relative path and not an absolute path? If not, is it possible to parse this flag using get_filename_component and convert it into an absolute path first?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

)
endif()
if(BROKER_ENDPOINT)
target_compile_definitions(
${DEMO_NAME} PRIVATE
BROKER_ENDPOINT="${BROKER_ENDPOINT}"
)
endif()
if(CLIENT_IDENTIFIER)
target_compile_definitions(
${DEMO_NAME} PRIVATE
CLIENT_IDENTIFIER="${CLIENT_IDENTIFIER}"
)
endif()
6 changes: 4 additions & 2 deletions demos/mqtt/mqtt_demo_mutual_auth/demo_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
* @note Port 443 requires use of the ALPN TLS extension with the ALPN protocol
* name. When using port 8883, ALPN is not required.
*/
#define AWS_MQTT_PORT ( 8883 )
#define AWS_MQTT_PORT ( 8883 )

/**
* @brief Path of the file containing the server's root CA certificate.
Expand All @@ -84,7 +84,9 @@
* @note This path is relative from the demo binary created. Update
* ROOT_CA_CERT_PATH to the absolute path if this demo is executed from elsewhere.
*/
#define ROOT_CA_CERT_PATH "certificates/AmazonRootCA1.crt"
#ifndef ROOT_CA_CERT_PATH
#define ROOT_CA_CERT_PATH "certificates/AmazonRootCA1.crt"
#endif

/**
* @brief Path of the file containing the client certificate.
Expand Down
12 changes: 12 additions & 0 deletions demos/mqtt/mqtt_demo_plaintext/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,15 @@ target_include_directories(
${LOGGING_INCLUDE_DIRS}
)

if(BROKER_ENDPOINT)
target_compile_definitions(
${DEMO_NAME} PRIVATE
BROKER_ENDPOINT="${BROKER_ENDPOINT}"
)
endif()
if(CLIENT_IDENTIFIER)
target_compile_definitions(
${DEMO_NAME} PRIVATE
CLIENT_IDENTIFIER="${CLIENT_IDENTIFIER}"
)
endif()
6 changes: 4 additions & 2 deletions demos/mqtt/mqtt_demo_plaintext/demo_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,15 @@
*
* In general, port 1883 is for unsecured MQTT connections.
*/
#define BROKER_PORT ( 1883 )
#define BROKER_PORT ( 1883 )

/**
* @brief MQTT client identifier.
*
* No two clients may use the same client identifier simultaneously.
*/
#define CLIENT_IDENTIFIER "testclient"
#ifndef CLIENT_IDENTIFIER
#define CLIENT_IDENTIFIER "testclient"
#endif

#endif /* ifndef DEMO_CONFIG_H */
8 changes: 0 additions & 8 deletions libraries/standard/mqtt/integration-test/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,14 +58,6 @@ create_test(${stest_name}
"${test_include_directories}"
)

# Download the Mosquitto Root CA certificate.
message( "Downloading the Mosquitto Root CA certificate..." )
file(MAKE_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/certificates)
execute_process(
COMMAND curl --url https://test.mosquitto.org/ssl/mosquitto.org.crt
-o ${CMAKE_CURRENT_LIST_DIR}/certificates/mosquitto.org.crt
)

# Copy the certificates and client key to the binary directory.
add_custom_command(
TARGET
Expand Down