Merge tag 'v3.0.1'

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,6 @@
+*Issue #, if available:*
+
+*Description of changes:*
+
+
+By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

.travis.yml

@@ -0,0 +1,66 @@
+language: c
+
+# Get Coverity certificate.
+before_install:
+  - echo -n | openssl s_client -connect scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
+
+# Coverity configuration.
+addons:
+  coverity_scan:
+    project:
+      name: "aws-iot-device-sdk-embedded-C"
+      description: "SDK for connecting to AWS IoT from a device using embedded C. "
+    notification_email: nobody@amazon.com
+    build_command_prepend: "cd tests/integration"
+    build_command: "make app"
+    branch_pattern: master
+
+install:
+  # Remove placeholders.
+  - rm external_libs/CppUTest/*
+  - rm -rf external_libs/mbedTLS
+
+  # Get mbedtls.
+  - git clone https://github.com/ARMmbed/mbedtls.git external_libs/mbedTLS
+
+  # Get CppUTest.
+  - wget -qO- https://github.com/cpputest/cpputest/archive/v3.6.tar.gz | tar xvz -C external_libs/CppUTest --strip-components=1
+
+script:
+  # Verify that the samples build.
+  - cd samples/linux/jobs_sample
+  - make
+  - cd ../shadow_sample
+  - make
+  - cd ../shadow_sample_console_echo
+  - make
+  - cd ../subscribe_publish_library_sample
+  - make
+  - cd ../subscribe_publish_sample
+  - make
+
+  # Set the AWS IoT endpoint.
+  - cd ../../../tests/integration
+  - sed -i 's/^.*#define AWS_IOT_MQTT_HOST.*$/#define AWS_IOT_MQTT_HOST "'"$INTEGRATION_TEST_ENDPOINT"'"/' include/aws_iot_config.h
+
+  # Build the integration tests.
+  - make app
+
+  # Build the unit tests.
+  - cd ../../
+  - make build-cpputest
+  - make all_no_tests
+
+  # Execute unit tests.
+  - ./IotSdkC_tests
+
+  # Import credentials.
+  - echo -e $INTEGRATION_TEST_CLIENT_CERT > certs/cert.pem
+  - echo -e $INTEGRATION_TEST_ROOT_CA > certs/rootCA.crt
+  - echo -e $INTEGRATION_TEST_PRIVATE_KEY > certs/privkey.pem
+
+  # Execute integration tests.
+  - cd tests/integration
+  - ./integration_tests_mbedtls
+  - ./integration_tests_mbedtls_mt
+

CHANGELOG.md

@@ -1,4 +1,41 @@
 # Change Log
+
+## [3.0.1](https://github.com/aws/aws-iot-device-sdk-embedded-C/releases/tag/v3.0.1) (May 10, 2018)
+
+Bugfixes:
+
+  - [#167], [#168] Fixed issues reported by Coverity Scan.
+  - [#177] Fixes a memory corruption bug and handling of timeouts.
+
+Other:
+
+  - Add .travis.yml for Travis CI.
+  - Removed C++ sample.
+  - Removed includes of `inttypes.h`, which doesn't exist on some systems.
+  - [#175] Added comments on static allocation of MQTT topics.
+
+## [3.0.0](https://github.com/aws/aws-iot-device-sdk-embedded-C/releases/tag/v3.0.0) (Apr 17, 2018)
+
+Bugfixes:
+
+  - [#152] Fixes potential buffer overflows in `parseStringValue` by requiring a size parameter in `jsonStruct_t`.
+  - [#155] Fixes other memory corruption bugs; also improves stability.
+
+The two bug fixes above are not backwards compatible with v2.3.0.  Please see [README.md](README.md#migrating-from-2x-to-3x) for details on migrating to v3.0.0.
+
+## [2.3.0](https://github.com/aws/aws-iot-device-sdk-embedded-C/releases/tag/v2.3.0) (Mar 21, 2018)
+
+New Features:
+
+  - Add [AWS IoT Jobs](https://docs.aws.amazon.com/iot/latest/developerguide/iot-jobs.html) support.
+  - Use AWS IoT Core support for MQTT over port 443. MQTT connection now defaults to port 443.
+
+Pull requests:
+
+  - [#124](https://github.com/aws/aws-iot-device-sdk-embedded-C/pull/124) - Thing Shadow: Fix potential shadow buffer overflow
+  - [#135](https://github.com/aws/aws-iot-device-sdk-embedded-C/pull/135) - mbedtls_wrap: Fix unintialized variable usage
+  - Fix bugs in long-running integration tests.
+
 ## [2.2.1](https://github.com/aws/aws-iot-device-sdk-embedded-C/releases/tag/v2.2.1) (Dec 26, 2017)
 
 Bugfixes:

CODE_OF_CONDUCT.md

@@ -0,0 +1,4 @@
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
+opensource-codeofconduct@amazon.com with any additional questions or comments.

CONTRIBUTING.md

@@ -0,0 +1,61 @@
+# Contributing Guidelines
+
+Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 
+documentation, we greatly value feedback and contributions from our community.
+
+Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 
+information to effectively respond to your bug report or contribution.
+
+
+## Reporting Bugs/Feature Requests
+
+We welcome you to use the GitHub issue tracker to report bugs or suggest features.
+
+When filing an issue, please check [existing open](https://github.com/aws/aws-iot-device-sdk-embedded-C/issues), or [recently closed](https://github.com/aws/aws-iot-device-sdk-embedded-C/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 
+reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
+
+* A reproducible test case or series of steps
+* The version of our code being used
+* Any modifications you've made relevant to the bug
+* Anything unusual about your environment or deployment
+
+
+## Contributing via Pull Requests
+Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
+
+1. You are working against the latest source on the *master* branch.
+2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
+3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
+
+To send us a pull request, please:
+
+1. Fork the repository.
+2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
+3. Ensure local tests pass.
+4. Commit to your fork using clear commit messages.
+5. Send us a pull request, answering any default questions in the pull request interface.
+6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
+
+GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 
+[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
+
+
+## Finding contributions to work on
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws/aws-iot-device-sdk-embedded-C/labels/help%20wanted) issues is a great place to start. 
+
+
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
+opensource-codeofconduct@amazon.com with any additional questions or comments.
+
+
+## Security issue notifications
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
+
+
+## Licensing
+
+See the [LICENSE](https://github.com/aws/aws-iot-device-sdk-embedded-C/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+
+We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

Makefile

@@ -55,11 +55,11 @@ TLS_LIB_DIR = $(TEMP_MBEDTLS_SRC_DIR)
 TLS_INCLUDE_DIR = -I $(TEMP_MBEDTLS_SRC_DIR)
 
 # Logging level control
-LOG_FLAGS += -DENABLE_IOT_DEBUG
+#LOG_FLAGS += -DENABLE_IOT_DEBUG
 #LOG_FLAGS += -DENABLE_IOT_TRACE
-LOG_FLAGS += -DENABLE_IOT_INFO
-LOG_FLAGS += -DENABLE_IOT_WARN
-LOG_FLAGS += -DENABLE_IOT_ERROR
+#LOG_FLAGS += -DENABLE_IOT_INFO
+#LOG_FLAGS += -DENABLE_IOT_WARN
+#LOG_FLAGS += -DENABLE_IOT_ERROR
 COMPILER_FLAGS += $(LOG_FLAGS)
 
 EXTERNAL_LIBS += -L$(CPPUTEST_BUILD_LIB)
@@ -128,4 +128,4 @@ clean:
 	$(RM) -rf build_output
 	$(RM) -rf gcov
 	$(RM) -rf objs
-	$(RM) -rf testLibs
+	$(RM) -rf testLibs

README.md

@@ -1,3 +1,11 @@
+
+[![Build Status](https://travis-ci.org/aws/aws-iot-device-sdk-embedded-C.svg?branch=master)](https://travis-ci.org/aws/aws-iot-device-sdk-embedded-C)
+
+<a href="https://scan.coverity.com/projects/aws-iot-device-sdk-embedded-c">
+  <img alt="Coverity Scan Build Status"
+       src="https://scan.coverity.com/projects/15543/badge.svg"/>
+</a>
+
 ## ***** NOTICE *****
 This repository is moving to a new branching system. The master branch will now contain bug fixes/features that have been minimally tested to ensure nothing major is broken. The release branch will contain new releases for the SDK that have been tested thoroughly on all supported platforms. Please ensure that you are tracking the release branch for all production work.
 
@@ -16,6 +24,9 @@ The Device SDK provides functionality to create and maintain a mutually authenti
 ### Thing Shadow
 The Device SDK implements the specific protocol for Thing Shadows to retrieve, update and delete Thing Shadows adhering to the protocol that is implemented to ensure correct versioning and support for client tokens. It abstracts the necessary MQTT topic subscriptions by automatically subscribing to and unsubscribing from the reserved topics as needed for each API call. Inbound state change requests are automatically signalled via a configurable callback.
 
+### Jobs
+The Device SDK implements features to facilitate use of the AWS Jobs service. The Jobs service can be used for device management tasks such as updating program files, rotating device certificates, or running other maintenance tasks such are restoring device settings or restarting devices.
+
 ## Design Goals of this SDK
 The embedded C SDK was specifically designed for resource constrained devices (running on micro-controllers and RTOS).
 
@@ -82,6 +93,156 @@ The 2.x branch makes several changes to the SDK. This section provides informati
 
 You can find more information on how to use the new APIs in the Readme file for samples that can be found [here](https://github.com/aws/aws-iot-device-sdk-embedded-c/blob/master/samples/README.md)
 
+## Migrating from 2.x to 3.x
+AWS IoT Device SDK for Embedded C v3.0.0 fixes two bugs (see #152 and #155) that create a potential buffer overflows. This version is not backward compatible with previous versions, so users will need to recompile their applications with the new version.
+
+Users of AWS IoT Device Shadows or Json utility functions such as `extractClientToken`, `emptyJsonWithClientToken`, `isJsonValidAndParse` and `isReceivedJsonValid` are encouraged to upgrade to version v3.0.0. For users who cannot upgrade, review all parts of your solution where user input can be sent to the device, and ensure sufficient authorization of these operations is enforced.
+
+Details of the required changes to public functions and data structures are shown below:
+
+### Changes in the `jsonStruct` data structure:
+The member `dataLength` has been added to struct `jsonStruct`, which is declared in [include/aws_iot_shadow_json_data.h](include/aws_iot_shadow_json_data.h#L60).
+
+```c
+struct jsonStruct {
+    const char *         pKey;
+    void *               pData;
+    size_t               dataLength;
+    JsonPrimitiveType    type;
+    JsonStructCallback_t cb;
+};
+```
+
+The size of the buffer `pData` must now be specified by `dataLength`. **Failure to do so may result in undefined behavior**. Below are examples of the code changes required to use the new jsonStruct.
+
+With a primitive data type, such as `int32_t`:
+
+```c
+…
+jsonStruct_t exampleJsonStruct;
+int32_t value = 0L;
+
+/* Set the members of exampleJsonStruct. */
+exampleJsonStruct.pKey = “exampleKey”;
+exampleJsonStruct.pData = &value;
+exampleJsonStruct.type = SHADOW_JSON_INT32;
+exampleJsonStruct.cb = exampleCallback;
+
+/* Register a delta callback using example JsonStruct. */
+aws_iot_shadow_register_delta(&mqttClient, &exampleJsonStruct); 
+…
+```
+
+Version 3.0.0 will require the following code:
+
+```c
+…
+jsonStruct_t exampleJsonStruct;
+int32_t value = 0L;
+
+/* Set the members of exampleJsonStruct. */
+exampleJsonStruct.pKey = “exampleKey”;
+exampleJsonStruct.pData = &value;
+exampleJsonStruct.dataLength = sizeof(int32_t); /* sizeof(value) also OK.*/
+exampleJsonStruct.type = SHADOW_JSON_INT32;
+exampleJsonStruct.cb = exampleCallback;
+
+/* Register a delta callback using example JsonStruct. */
+aws_iot_shadow_register_delta(&mqttClient, &exampleJsonStruct);
+…
+```
+
+With a string, versions up to v2.3.0 would require the following code:
+
+```c
+…
+jsonStruct_t exampleJsonStruct;
+char stringBuffer[SIZE_OF_BUFFER];
+/* Set the members of exampleJsonStruct. */
+exampleJsonStruct.pKey = “exampleKey”;
+exampleJsonStruct.pData = stringBuffer;
+exampleJsonStruct.type = SHADOW_JSON_STRING;
+exampleJsonStruct.cb = exampleCallback;
+/* Register a delta callback using example JsonStruct. */
+aws_iot_shadow_register_delta(&mqttClient, &exampleJsonStruct);
+…
+```
+
+Version 3.0.0 will require the following code:
+
+```c
+…
+jsonStruct_t exampleJsonStruct;
+char stringBuffer[SIZE_OF_BUFFER];
+/* Set the members of exampleJsonStruct. */
+exampleJsonStruct.pKey = “exampleKey”;
+exampleJsonStruct.pData = stringBuffer;
+exampleJsonStruct.dataLength = SIZE_OF_BUFFER;
+exampleJsonStruct.type = SHADOW_JSON_STRING;
+exampleJsonStruct.cb = exampleCallback;
+/* Register a delta callback using example JsonStruct. */
+aws_iot_shadow_register_delta(&mqttClient, &exampleJsonStruct);
+…
+```
+
+### Changes in parseStringValue:
+The function `parseStringValue`, declared in [include/aws_iot_json_utils.h](include/aws_iot_json_utils.h#L179) and implemented in [src/aws_iot_json_utils.c](src/aws_iot_json_utils.c#L184), now requires the inclusion of a buffer length. Its new function signature is:
+
+```c
+IoT_Error_t parseStringValue(char *buf, size_t bufLen, const char *jsonString, jsmntok_t *token);
+```
+
+Below is an example of code changes required to use the new parseStringValue.
+
+With up to version v2.3.0:
+
+```c
+…
+char* jsonString = “…”;
+jsmntok_t jsmnTokens[NUMBER_OF_JSMN_TOKENS];
+char stringBuffer[SIZE_OF_BUFFER];
+parseStringValue(stringBuffer, jsonString, jsmnTokens);
+…
+```
+
+Version 3.0.0 will require the following code:
+
+```c
+…
+char* jsonString = “…”;
+jsmntok_t jsmnTokens[NUMBER_OF_JSMN_TOKENS];
+char stringBuffer[SIZE_OF_BUFFER];
+parseStringValue(stringBuffer, SIZE_OF_BUFFER, jsonString, jsmnTokens);
+…
+```
+
+### Changes to functions intended for internal usage:
+Version 3.0.0 changes the signature of four functions intended for internal usage. The new signatures explicitly carry the information for the size of the buffer or JSON document passed as a parameter to the functions. Users of the SDK may need to change their code and recompile to ingest the changes. We report the old and new signatures below.
+
+#### Old signatures:
+
+```c
+bool extractClientToken(const char *pJsonDocument, char *pExtractedClientToken); 
+
+static void emptyJsonWithClientToken(char *pBuffer);
+
+bool isJsonValidAndParse(const char *pJsonDocument, void *pJsonHandler, int32_t *pTokenCount); 
+
+bool isReceivedJsonValid(const char *pJsonDocument);
+```
+
+#### New signatures:
+
+```c
+bool extractClientToken(const char *pJsonDocument, size_t jsonSize, char *pExtractedClientToken, size_t clientTokenSize); 
+
+static void emptyJsonWithClientToken(char *pBuffer, size_t bufferSize);
+
+bool isJsonValidAndParse(const char *pJsonDocument, size_t jsonSize, void *pJsonHandler, int32_t *pTokenCount);
+
+bool isReceivedJsonValid(const char *pJsonDocument, size_t jsonSize);
+```
+
 ## Resources
 [API Documentation](http://aws-iot-device-sdk-embedded-c-docs.s3-website-us-east-1.amazonaws.com/index.html)