Add CBOR support #709

mlasch · 2023-06-09T16:17:43Z

Add CBOR support to LwM2M server and client introduced with the LwM2M 1.1 standard. This PR is based on work from @sbertin-telular with changes to work with the current build system. Thanks for the contribution.

CBOR support can be enabled by compiling the client with LWM2M_SUPPORT_SENML_CBOR flag enabled. The server will support SenML-CBOR when compiled with LwM2M 1.1 support.

LukasWoodtli · 2023-06-29T11:13:18Z

tests/cbor_tests.c

+"Unknown"))))))))
+#endif
+
+#if 0


Why is this disabled. Should probably not be committed. At least it should be configurable by a proprocessor define.

I just imported the code and my IDE conveniently hides the section. I will remove it.

LukasWoodtli · 2023-06-29T11:33:50Z

data/senml_cbor.c

@@ -139,6 +139,8 @@ static int prv_parseItem(const uint8_t *buffer, size_t bufferLen, senml_record_t
            bvSeen = true;
            memcpy(baseValue, &data, sizeof(lwm2m_data_t));
            /* Convert explicit 0 to implicit 0 */
+            _Pragma("GCC diagnostic push");
+            _Pragma("GCC diagnostic ignored \"-Wfloat-equal\"");


Comparing floats for equality is not a very good idea.
Could use fpclassify (since C99) or something like fabsf(a - b) < FLT_EPSILON

IMHO we can keep it this way for now.

I really don't like the compiler specific _Pragma. Not everyone uses gcc.

I think there are just 3 equality checks in there outside of the examples and tests. I'm more concerned with the base library than those.

Check for NaN which could be replaced by isnan(), I was just targeting C90 when I wrote it.

Check whether or not anything would be lost converting double to float (and back). If not, then it can be encoded single precision. Another method would probably need to break it down into the exponent and mantissa and check those. That may need to make assumptions about the underlying hardware representation, although that assumption is probably already being made.

Explicitly check for 0. Something like fabsf(a - b) < FLT_EPSILON would involve extra floating point computations and classify anything less than FLT_EPSILON as 0, which would be incorrect. Again, checks on the mantissa and exponent could be done, but be careful that -0 and +0 are treated the same. That would also likely be more computation than a simple compare.

I think we could use fpclassify since we rely on C11.

I think we could use fpclassify since we rely on C11.

Will do this in a separate PR. There are already some Pragma in the code to mute floating point comparison compiler warnings. They can be removed as well.

sbertin-telular · 2023-06-29T13:59:35Z

data/cbor.c

@@ -14,12 +14,14 @@
 *    Scott Bertin, AMETEK, Inc. - Please refer to git log
 *
 *******************************************************************************/
-
+#define _DEFAULT_SOURCE
+#include <endian.h>


endian.h is glibc specific and may not exist with other compilers or libraries.

sbertin-telular · 2023-06-29T15:43:22Z

data/senml_cbor.c

-static int prv_serializeTlv(const lwm2m_data_t *tlvP, const uint8_t *baseUriStr, size_t baseUriLen,
-                            uri_depth_t baseLevel, const uint8_t *parentUriStr, size_t parentUriLen, uri_depth_t level,
-                            bool *baseNameOutput, uint8_t *buffer, size_t bufferLen) {
+static int prv_serializeCbor(const lwm2m_data_t *tlvP, const uint8_t *baseUriStr, size_t baseUriLen,


Why the name change? It is serializing TLV (lwm2m_data_t) to CBOR.

I wasn't aware that the TLV term is also used for the internal data representation lwm2m_data_t. I will change it back.

This commit contains the contribution from Scott Bertin to support SenML-CBOR with only changes resolving conflicts. Signed-off-by: Scott Bertin <sbertin@telular.com> Signed-off-by: Pascal Brogle <pascal.brogle@husqvarnagroup.com> Signed-off-by: Marc Lasch <marc.lasch@husqvarnagroup.com>

core/internals.h

rettichschnidi · 2023-11-06T11:54:57Z

core/internals.h

+#ifdef LWM2M_SUPPORT_SENML_CBOR
+// defined in cbor.c
+int cbor_get_type_and_value(const uint8_t *buffer, size_t bufferLen, cbor_type_t *type, uint64_t *value);
+int cbor_get_singular(const uint8_t *buffer, size_t bufferLen, lwm2m_data_t *dataP);
+int cbor_put_type_and_value(uint8_t *buffer, size_t bufferLen, cbor_type_t type, uint64_t val);
+int cbor_put_singular(uint8_t *buffer, size_t bufferLen, const lwm2m_data_t *dataP);
+#if !defined(LWM2M_VERSION_1_0) && !defined(LWM2M_VERSION_1_1)
+int cbor_parse(const lwm2m_uri_t *uriP, const uint8_t *buffer, size_t bufferLen, lwm2m_data_t **dataP);
+int cbor_serialize(const lwm2m_uri_t *uriP, int size, const lwm2m_data_t *dataP, uint8_t **bufferP);
+#endif
+
+// defined in senml_cbor.c
+int senml_cbor_parse(const lwm2m_uri_t *uriP, const uint8_t *buffer, size_t bufferLen, lwm2m_data_t **dataP);
+int senml_cbor_serialize(const lwm2m_uri_t *uriP, int size, const lwm2m_data_t *tlvP, uint8_t **bufferP);
+#endif
+
+// defined in senml_common.c
+#if defined(LWM2M_SUPPORT_JSON) || defined(LWM2M_SUPPORT_SENML_JSON) || defined(LWM2M_SUPPORT_SENML_CBOR)
+int senml_convert_records(const lwm2m_uri_t *uriP, senml_record_t *recordArray, int numRecords,
+                          senml_convertValue convertValue, lwm2m_data_t **dataP);
+lwm2m_data_t *senml_extendData(lwm2m_data_t *parentP, lwm2m_data_type_t type, uint16_t id);
+int senml_dataStrip(int size, lwm2m_data_t *dataP, lwm2m_data_t **resultP);
+lwm2m_data_t *senml_findDataItem(lwm2m_data_t *listP, size_t count, uint16_t id);
+uri_depth_t senml_decreaseLevel(uri_depth_t level);
+int senml_findAndCheckData(const lwm2m_uri_t *uriP, uri_depth_t baseLevel, size_t size, const lwm2m_data_t *tlvP,
+                           lwm2m_data_t **targetP, uri_depth_t *targetLevelP);
+#endif


It is not really done at all in Wakaama, but having (at least) the return values documented would be reeeeeeeally nice. How is success encoded? >0? How an error? <0?

(Feel free to ignore/"resolve conversation" this input. Will approve anyway.)

tests/cbor_tests.c

tests/senml_cbor_tests.c

sbertin-telular · 2023-11-06T13:44:48Z

I'm not involved in any projects using LWM2M anymore, so don't wait for my review. My only objection remains the use of the gcc specific _Pragma.

This change only contains cosmetic changes to comply with the current code format style for the SenML-CBOR contribution enforced by clang-format and cmake-format.

With -Wstrict-prototypes enforced, all functions need a prototype.

because zephyr does not support float16

Silence the existing, benign looking code, but ensure we get a warning for new code with such issues.

With -Wstrict-aliasing enforced, simple type casting between different lvalue is not allowed.

Convert endianness to also support big-endian systems.

This will silence -Wsign-compare warnings.

Add capability to log data types to improve logging in CBOR code.

rettichschnidi

Not tested/used the code, but looks sane enough to me.

mlasch requested a review from rettichschnidi June 9, 2023 16:17

LukasWoodtli reviewed Jun 29, 2023

View reviewed changes

sbertin-telular reviewed Jun 29, 2023

View reviewed changes

mlasch mentioned this pull request Jun 29, 2023

Supported compilers, architectures, language versions #494

Open

sbertin-telular reviewed Jun 29, 2023

View reviewed changes

LukasWoodtli force-pushed the gardena/feature-senml-cbor branch 4 times, most recently from 080be99 to 278dbd7 Compare October 5, 2023 13:29

mlasch force-pushed the gardena/feature-senml-cbor branch from 6538ba9 to b43ed07 Compare November 4, 2023 18:27

mlasch force-pushed the gardena/feature-senml-cbor branch from b43ed07 to 94d896a Compare November 6, 2023 10:34

mlasch requested review from LukasWoodtli and sbertin-telular November 6, 2023 10:39

rettichschnidi requested changes Nov 6, 2023

View reviewed changes

core/internals.h Outdated Show resolved Hide resolved

core/internals.h Show resolved Hide resolved

mlasch force-pushed the gardena/feature-senml-cbor branch from 94d896a to 16d7a7c Compare November 6, 2023 11:03

rettichschnidi requested changes Nov 6, 2023

View reviewed changes

mlasch and others added 8 commits November 6, 2023 16:29

Modifiy the SenML-CBOR contribution to comply with project formatting

9dd0336

This change only contains cosmetic changes to comply with the current code format style for the SenML-CBOR contribution enforced by clang-format and cmake-format.

refactor: Enforce prototypes on functions

daa70be

With -Wstrict-prototypes enforced, all functions need a prototype.

feat: add support to disable cbor float16 encoding

db77613

because zephyr does not support float16

Silence -Wfloat-equal warnings

9397787

Silence the existing, benign looking code, but ensure we get a warning for new code with such issues.

Enforce strict aliasing with type punning in SenML-CBOR

caa4ee4

With -Wstrict-aliasing enforced, simple type casting between different lvalue is not allowed.

Refactor CBOR value parsing

d7b543e

Convert endianness to also support big-endian systems.

Fix signed integer comparison warnings

9424780

This will silence -Wsign-compare warnings.

Improve logging in CBOR code

389bf5f

Add capability to log data types to improve logging in CBOR code.

LukasWoodtli approved these changes Nov 6, 2023

View reviewed changes

mlasch force-pushed the gardena/feature-senml-cbor branch from 16d7a7c to 389bf5f Compare November 6, 2023 15:42

rettichschnidi approved these changes Nov 8, 2023

View reviewed changes

sbertin-telular approved these changes Nov 8, 2023

View reviewed changes

mlasch merged commit 676c520 into eclipse-wakaama:master Nov 9, 2023
31 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CBOR support #709

Add CBOR support #709

mlasch commented Jun 9, 2023

LukasWoodtli Jun 29, 2023

mlasch Jul 2, 2023

LukasWoodtli Jun 29, 2023

LukasWoodtli Jun 29, 2023

sbertin-telular Jun 29, 2023

sbertin-telular Jun 29, 2023

LukasWoodtli Jun 29, 2023

mlasch Nov 9, 2023

sbertin-telular Jun 29, 2023

sbertin-telular Jun 29, 2023

mlasch Nov 4, 2023

rettichschnidi Nov 6, 2023 •

edited

Loading

sbertin-telular commented Nov 6, 2023

rettichschnidi left a comment

Add CBOR support #709

Add CBOR support #709

Conversation

mlasch commented Jun 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rettichschnidi Nov 6, 2023 • edited Loading

Choose a reason for hiding this comment

sbertin-telular commented Nov 6, 2023

rettichschnidi left a comment

Choose a reason for hiding this comment

rettichschnidi Nov 6, 2023 •

edited

Loading