Fragmentation support. (#23)

* Refs. #3800 First approach.

* Refs #3800. Added callback call.

* Refs #3801. Behaviour extended to all ser/deser functions.

* Finished callback logic. Added fragmentation example.

* Refs #3802. Added fragmentation test.

* Refs 3800. Changed name from the finished buffer to full buffer.

* Refs #3803. Added docs.

* Removed unused code.

CMakeLists.txt

@@ -50,9 +50,11 @@ endif()
 ###############################################################################
 option(EPROSIMA_BUILD "Activate internal building" OFF)
 option(EPROSIMA_BUILD_TESTS "Activate the building of tests" OFF)
+option(EPROSIMA_BUILD_EXAMPLES "Compile examples" OFF)
 
 if(EPROSIMA_BUILD)
     set(EPROSIMA_BUILD_TESTS ON)
+    set(EPROSIMA_BUILD_EXAMPLES ON)
 endif()
 
 ###############################################################################
@@ -135,10 +137,18 @@ get_target_property(TARGET_TYPE ${PROJECT_NAME} TYPE)
 if((MSVC OR MSVC_IDE) AND (TARGET_TYPE STREQUAL "SHARED_LIBRARY"))
     target_compile_definitions(${PROJECT_NAME}
         PUBLIC
-            -D${PROJECT_NAME}_SHARED
+            ${PROJECT_NAME}_SHARED
         )
 endif()
 
+###############################################################################
+# Examples
+###############################################################################
+if(EPROSIMA_BUILD_EXAMPLES)
+    add_subdirectory(examples/basic)
+    add_subdirectory(examples/fragmentation)
+endif()
+
 ###############################################################################
 # Testing
 ###############################################################################

README.md

@@ -148,37 +148,15 @@ bool ucdr_buffer_error(const ucdrBuffer* ub);
 Returns the status error of the `ucdrBuffer`.
 - `ub`: the `ucdrBuffer` struct
 
-
 ### Serialization/deserialization functions
 Adding to this, there is a big set of functions for deserialize and deserialize different kind of types:
 - Basics: `bool`, `char`, `int8_t`, `uint8_t`,`int16_t`, `uint16_t`,`int32_t`, `uint32_t`,`int64_t`, `uint64_t`,`float`, `double`.
 - Arrays: Any fixed size of basics types.
 - Sequence: Similar to arrays, but the information about the size is serialized along with the data.
+- String: Wrapper of char sequence, but easily to use.
 
-### Endianness
-*MicroCDR* supports little and big endianness.
-The configuration can be done by cmake with the cmake `__BIG_ENDIAN__` variable.
-A `0` value implies that the serialization will performed into a little endian machine, and `1` into a big endian machine.
-
-The default endianness serialization can be choosen by setting the `endianness` parameter of a `ucdrBuffer`  to `UCDR_BIG_ENDIANNESS` or `UCDR_LITTLE_ENDIANNESS`.
-Also, there are a functions that allow to force an endianness in their serialization/deserialization.
-These functions contains the name `endiannness` in their signature.
-
-### Error
-All serialization/deserialization functions return a boolean indicating the result of their operations.
-When a serialization/deserialization could not be possible (the type can not be serialized, or the capacity of the destination buffer is not enough),
-an status error is setted into the `ucdrBuffer`.
-If a `ucdrBuffer` has an error state, the next serialization/deserialization operations will not works and will return `false` in their execution.
-A buffer marked with an error can be used, but any serialization/deserialization operation over it will not produce any effect.
-
-If is kwown that an operation can fails over a `ucdrBuffer`, and its necessary to continue with the serialization/deserialization if it happens,
-the `ucdrBuffer` state can be saved using the `ucdr_copy_buffer` function.
-After the application of the wrong serialization/deserialization, only the `ucdrBuffer` that performed the operation will have a dirty state.
-
-## Serialization/deserialization list
 The available modes of serialization/deserializations in *MicroCDR* are shown in the following table.
 
-
 | Type                 | Endianness |
 | -------------------- | ---------- |
 | bool                 |            |
@@ -248,3 +226,29 @@ The available modes of serialization/deserializations in *MicroCDR* are shown in
 | double sequence      |            |
 | double sequence      | endianness |
 
+## Additional features
+### Endianness
+*MicroCDR* supports little and big endianness.
+The configuration can be done by cmake with the cmake `__BIG_ENDIAN__` variable.
+A `0` value implies that the serialization will performed into a little endian machine, and `1` into a big endian machine.
+
+The default endianness serialization can be choosen by setting the `endianness` parameter of a `ucdrBuffer`  to `UCDR_BIG_ENDIANNESS` or `UCDR_LITTLE_ENDIANNESS`.
+Also, there are a functions that allow to force an endianness in their serialization/deserialization.
+These functions contains the name `endiannness` in their signature.
+
+### Error
+All serialization/deserialization functions return a boolean indicating the result of their operations.
+When a serialization/deserialization could not be possible (the type can not be serialized, or the capacity of the destination buffer is not enough),
+an status error is setted into the `ucdrBuffer`.
+If a `ucdrBuffer` has an error state, the next serialization/deserialization operations will not works and will return `false` in their execution.
+A buffer marked with an error can be used, but any serialization/deserialization operation over it will not produce any effect.
+
+If is kwown that an operation can fails over a `ucdrBuffer`, and its necessary to continue with the serialization/deserialization if it happens,
+the `ucdrBuffer` state can be saved using the `ucdr_copy_buffer` function.
+After the application of the wrong serialization/deserialization, only the `ucdrBuffer` that performed the operation will have a dirty state.
+
+### Full buffer callback
+MicroCDR provides a callback that the user can set in order to control the behavior when the `ucdrBuffer` can not serialize/deserialize anymore because the buffer is full.
+This allows to create a better management error and/or modify the buffer location of the `ucdrBuffer`.
+The last possibility gives the user the capacity to use several small buffers for a big serialization (see the *fragmentation* example).
+

examples/basic/CMakeLists.txt

@@ -0,0 +1,23 @@
+project(basic_example)
+
+add_executable(${PROJECT_NAME} basic.c)
+if(MSVC OR MSVC_IDE)
+    target_compile_options(${PROJECT_NAME} PRIVATE /wd4996)
+endif()
+
+set_target_properties(${PROJECT_NAME} PROPERTIES
+    C_STANDARD 99
+    C_STANDARD_REQUIRED YES
+    )
+
+target_link_libraries(${PROJECT_NAME} microcdr)
+
+install(TARGETS ${PROJECT_NAME}
+    RUNTIME DESTINATION examples/ucdr/${PROJECT_NAME}/${BIN_INSTALL_DIR}
+    )
+
+install(DIRECTORY ${PROJECT_SOURCE_DIR}/
+    DESTINATION  examples/ucdr/${PROJECT_NAME}
+    FILES_MATCHING PATTERN "*.h"
+                   PATTERN "*.c"
+    )

examples/fragmentation/CMakeLists.txt

@@ -0,0 +1,23 @@
+project(fragmentation_example)
+
+add_executable(${PROJECT_NAME} fragmentation.c)
+if(MSVC OR MSVC_IDE)
+    target_compile_options(${PROJECT_NAME} PRIVATE /wd4996)
+endif()
+
+set_target_properties(${PROJECT_NAME} PROPERTIES
+    C_STANDARD 99
+    C_STANDARD_REQUIRED YES
+    )
+
+target_link_libraries(${PROJECT_NAME} microcdr)
+
+install(TARGETS ${PROJECT_NAME}
+    RUNTIME DESTINATION examples/ucdr/${PROJECT_NAME}/${BIN_INSTALL_DIR}
+    )
+
+install(DIRECTORY ${PROJECT_SOURCE_DIR}/
+    DESTINATION  examples/ucdr/${PROJECT_NAME}
+    FILES_MATCHING PATTERN "*.h"
+                   PATTERN "*.c"
+    )

examples/fragmentation/fragmentation.c

@@ -0,0 +1,83 @@
+// Copyright 2017 Proyectos y Sistemas de Mantenimiento SL (eProsima).
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <ucdr/microcdr.h>
+#include <stdio.h>
+#include <string.h>
+#include <inttypes.h>
+
+#define BUFFER_LENGTH 12
+#define SLOTS 12
+#define STRING_MAX 128
+
+bool on_full_buffer(ucdrBuffer* ub, void* args)
+{
+    uint8_t* buffer =  (uint8_t*) args;
+
+    // This value correspong with the ub->error, and will be returned by this function to indicates
+    // if the serialization must continue or must stop because of an error.
+    bool error = true;
+
+    // Leave the odd slots empty.
+    uint32_t next_slot = 2 + (uint32_t)(ub->init - buffer) / BUFFER_LENGTH;
+    if(next_slot < SLOTS)
+    {
+        // Modify the internal buffer
+        ub->init = buffer + BUFFER_LENGTH * next_slot;
+        ub->iterator = ub->init;
+        ub->final = ub->init + BUFFER_LENGTH;
+
+        printf("  Extend buffer to slot %u\n", next_slot);
+
+        // As we want to continue the serialization without errors, we return false.
+        error = false;
+    }
+
+    return error;
+}
+
+int main()
+{
+    // Data buffer
+    uint8_t buffer[SLOTS * BUFFER_LENGTH];
+
+    // Structs for handle the buffer.
+    ucdrBuffer writer;
+    ucdrBuffer reader;
+
+    // Initialize the MicroBuffers for working with an user-managed buffer.
+    ucdr_init_buffer(&writer, buffer, BUFFER_LENGTH);
+    ucdr_init_buffer(&reader, buffer, BUFFER_LENGTH);
+
+    // Add a full buffer behavior to the writer and the reader
+    ucdr_set_on_full_buffer_callback(&writer, on_full_buffer, buffer);
+    ucdr_set_on_full_buffer_callback(&reader, on_full_buffer, buffer);
+
+    // Serialize data
+    printf("Serializing...\n");
+    char input[STRING_MAX] = "Hello MicroCDR! this message is fragmented";
+    ucdr_serialize_string(&writer, input);
+    printf("\n");
+
+    // Deserialize data
+    printf("Deserializing...\n");
+    char output[STRING_MAX];
+    ucdr_deserialize_string(&reader, output, STRING_MAX);
+    printf("\n");
+
+    printf("Input: %s\n", input);
+    printf("Output: %s\n", output);
+
+    return 0;
+}

include/ucdr/common.h

@@ -32,6 +32,9 @@ typedef enum ucdrEndianness {
 
 } ucdrEndianness;
 
+struct ucdrBuffer;
+typedef bool (*OnFullBuffer)(struct ucdrBuffer* buffer, void* args);
+
 typedef struct ucdrBuffer
 {
     uint8_t *init;
@@ -43,30 +46,34 @@ typedef struct ucdrBuffer
 
     bool error;
 
+    OnFullBuffer on_full_buffer;
+    void* args;
+
 } ucdrBuffer;
 
 UCDRDLLAPI extern const ucdrEndianness UCDR_MACHINE_ENDIANNESS;
 
 // ------------------------------------------------
 //              Main library functions
 // ------------------------------------------------
-UCDRDLLAPI void ucdr_init_buffer               (ucdrBuffer* ub, uint8_t* data, const uint32_t size);
-UCDRDLLAPI void ucdr_init_buffer_offset        (ucdrBuffer* ub, uint8_t* data, const uint32_t size, uint32_t offset);
-UCDRDLLAPI void ucdr_init_buffer_offset_endian (ucdrBuffer* ub, uint8_t* data, const uint32_t size, uint32_t offset, ucdrEndianness endianness);
-UCDRDLLAPI void ucdr_copy_buffer               (ucdrBuffer* ub_dest, const ucdrBuffer* ub_source);
+UCDRDLLAPI void ucdr_init_buffer                     (ucdrBuffer* ub, uint8_t* data, const uint32_t size);
+UCDRDLLAPI void ucdr_init_buffer_offset              (ucdrBuffer* ub, uint8_t* data, const uint32_t size, uint32_t offset);
+UCDRDLLAPI void ucdr_init_buffer_offset_endian       (ucdrBuffer* ub, uint8_t* data, const uint32_t size, uint32_t offset, ucdrEndianness endianness);
+UCDRDLLAPI void ucdr_copy_buffer                     (ucdrBuffer* ub_dest, const ucdrBuffer* ub_source);
+UCDRDLLAPI void ucdr_set_on_full_buffer_callback     (ucdrBuffer* ub, OnFullBuffer on_full_buffer, void* args);
 
 UCDRDLLAPI void ucdr_reset_buffer        (ucdrBuffer* ub);
 UCDRDLLAPI void ucdr_reset_buffer_offset (ucdrBuffer* ub, const uint32_t offset);
 
-UCDRDLLAPI void     ucdr_align_to              (ucdrBuffer* ub, const uint32_t alignment);
-UCDRDLLAPI uint32_t ucdr_alignment             (uint32_t buffer_position, const uint32_t data_size);
-UCDRDLLAPI uint32_t ucdr_buffer_alignment(const ucdrBuffer* ub, const uint32_t data_size);
+UCDRDLLAPI void     ucdr_align_to         (ucdrBuffer* ub, const uint32_t alignment);
+UCDRDLLAPI uint32_t ucdr_alignment        (uint32_t buffer_position, const uint32_t data_size);
+UCDRDLLAPI uint32_t ucdr_buffer_alignment (const ucdrBuffer* ub, const uint32_t data_size);
 
-UCDRDLLAPI size_t     ucdr_buffer_size      (const ucdrBuffer* ub);
-UCDRDLLAPI size_t     ucdr_buffer_length    (const ucdrBuffer* ub);
-UCDRDLLAPI size_t     ucdr_buffer_remaining (const ucdrBuffer* ub);
-UCDRDLLAPI ucdrEndianness ucdr_buffer_endianness(const ucdrBuffer* ub);
-UCDRDLLAPI bool       ucdr_buffer_has_error (const ucdrBuffer* ub);
+UCDRDLLAPI size_t         ucdr_buffer_size       (const ucdrBuffer* ub);
+UCDRDLLAPI size_t         ucdr_buffer_length     (const ucdrBuffer* ub);
+UCDRDLLAPI size_t         ucdr_buffer_remaining  (const ucdrBuffer* ub);
+UCDRDLLAPI ucdrEndianness ucdr_buffer_endianness (const ucdrBuffer* ub);
+UCDRDLLAPI bool           ucdr_buffer_has_error  (const ucdrBuffer* ub);
 
 #ifdef __cplusplus
 }

include/ucdr/microcdr.h

@@ -21,9 +21,9 @@ extern "C" {
 
 #include <ucdr/common.h>
 #include <ucdr/types/basic.h>
-#include <ucdr/types/string.h>
 #include <ucdr/types/array.h>
 #include <ucdr/types/sequence.h>
+#include <ucdr/types/string.h>
 
 #ifdef __cplusplus
 }

src/c/common.c

@@ -22,23 +22,51 @@
     const ucdrEndianness UCDR_MACHINE_ENDIANNESS = UCDR_LITTLE_ENDIANNESS;
 #endif
 
+static uint32_t ucdr_next_remaining_size(ucdrBuffer* ub, const uint32_t bytes, const uint32_t data_size);
+
 // -------------------------------------------------------------------
 //                   INTERNAL UTIL IMPLEMENTATIONS
 // -------------------------------------------------------------------
-bool ucdr_check_buffer(ucdrBuffer* ub, const uint32_t bytes)
+
+bool ucdr_check_buffer_available_for(ucdrBuffer* ub, const uint32_t bytes)
+{
+    return !ub->error && (ub->iterator + bytes <= ub->final);
+}
+
+bool ucdr_check_final_buffer_behavior(ucdrBuffer* ub, const uint32_t data_size)
 {
-    if(!ub->error)
+    if(!ub->error && ub->iterator + data_size > ub->final)
     {
-        bool fit = ub->iterator + bytes <= ub->final;
-        if(!fit)
-        {
-            ub->error = true;
-        }
+        ub->error = (NULL != ub->on_full_buffer) ? ub->on_full_buffer(ub, ub ->args) : true;
     }
 
     return !ub->error;
 }
 
+uint32_t ucdr_next_remaining_size(ucdrBuffer* ub, const uint32_t bytes, const uint32_t data_size)
+{
+    uint32_t remaining = (uint32_t)ucdr_buffer_remaining(ub);
+    return (ub->iterator + bytes <= ub->final)
+           ? bytes
+           : remaining - (remaining % data_size);
+}
+
+uint32_t ucdr_check_final_buffer_behavior_array(ucdrBuffer* ub, const uint32_t bytes, const uint32_t data_size)
+{
+    if(!ub->error && ub->iterator + data_size > ub->final && bytes > 0)
+    {
+        ub->error = (NULL != ub->on_full_buffer) ? ub->on_full_buffer(ub, ub->args) : true;
+    }
+
+    return (!ub->error) ? ucdr_next_remaining_size(ub, bytes, data_size) : 0;
+}
+
+void ucdr_set_on_full_buffer_callback(ucdrBuffer* ub, OnFullBuffer on_full_buffer, void* args)
+{
+    ub->on_full_buffer = on_full_buffer;
+    ub->args = args;
+}
+
 // -------------------------------------------------------------------
 //                       PUBLIC IMPLEMENTATION
 // -------------------------------------------------------------------
@@ -60,6 +88,8 @@ void ucdr_init_buffer_offset_endian(ucdrBuffer* ub, uint8_t* data, const uint32_
     ub->last_data_size = 0U;
     ub->endianness = endianness;
     ub->error = false;
+    ub->on_full_buffer = NULL;
+    ub->args = NULL;
 }
 
 

src/c/common_internals.h

@@ -24,7 +24,9 @@ extern "C" {
 // -------------------------------------------------------------------
 //                     INTERNAL UTIL FUNCTIONS
 // -------------------------------------------------------------------
-bool ucdr_check_buffer(ucdrBuffer* ub, const uint32_t bytes);
+bool     ucdr_check_buffer_available_for(ucdrBuffer* ub, const uint32_t bytes);
+bool     ucdr_check_final_buffer_behavior(ucdrBuffer* ub, const uint32_t bytes);
+uint32_t ucdr_check_final_buffer_behavior_array(ucdrBuffer* ub, const uint32_t bytes, const uint32_t data_size);
 
 #ifdef __cplusplus
 }