Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add json serialization/deserialization support for c++ generator #99

Merged
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Djinni generator parses an interface definition file and generates:
- Python implementation of types
- C++/CLI implementation of types
- C++ code to convert between C++ and Java over JNI
- C++ code to serialize/deserialize types to/from JSON
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI you may want to update the feature list on the gitpage as well: https://github.com/cross-language-cpp/cross-language-cpp.github.io/blob/main/docs/overview.md#main-features

This will show up here on the documentation website. :)

- Objective-C++ code to convert between C++ and Objective-C
- Python and C code to convert between C++ and Python over CFFI
- C++/CLI code to convert between C++ and C#
Expand Down
1 change: 1 addition & 0 deletions docs/cli-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ djinni \
| `--cpp-nn-check-expression <header>` | The expression to use for building non-nullable pointers |
| `--cpp-use-wide-strings <true/false>` | Use wide strings in C++ code (default: `false`) |
| `--cpp-omit-default-record-constructor <true/false>` | Omit the default constructor for records in C++ code (default: `false`) |
| `--cpp-json-serialization <nlohmann_json>` | If specified, generate serializers to/from JSON and C++ types using [nlohmann/json](https://github.com/nlohmann/json).|

### JNI

Expand Down
125 changes: 96 additions & 29 deletions docs/generated-code-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,17 @@
### Includes & Build target
The following headers / code will be generated for each defined type:

| Type | C++ header | C++ source | Java | JNI header | JNI source |
|------------|------------------------|----------------------------|---------------------|-----------------------|-----------------------|
| Enum/Flags | my\_enum.hpp | | MyEnum.java | NativeMyEnum.hpp | NativeMyEnum.cpp |
| Record | my\_record[\_base].hpp | my\_record[\_base].cpp (+) | MyRecord[Base].java | NativeMyRecord.hpp | NativeMyRecord.cpp |
| Interface | my\_interface.hpp | my\_interface.cpp (+) | MyInterface.java | NativeMyInterface.hpp | NativeMyInterface.cpp |
| Type | C++ header | C++ source | Java | JNI header | JNI source |
|------------|----------------------------|------------------------------|---------------------|-----------------------|-----------------------|
| Enum/Flags | my\_enum.hpp | | MyEnum.java | NativeMyEnum.hpp | NativeMyEnum.cpp |
| | my\_enum+json.hpp :two: | | | | |
| Record | my\_record[\_base].hpp | my\_record[\_base].cpp :one: | MyRecord[Base].java | NativeMyRecord.hpp | NativeMyRecord.cpp |
| | my\_record[\_base]+json.hpp| | | | |
| | :two:| | | | |
| Interface | my\_interface.hpp | my\_interface.cpp :one: | MyInterface.java | NativeMyInterface.hpp | NativeMyInterface.cpp |

(+) Generated only for types that contain constants.
- :one: Generated only for types that contain constants.
- :two: Generated only if cpp json serialization is enabled.

Additionally `djinni_jni_main.cpp` is generated to provide a default implementation for `JNI_OnLoad` and `JNI_OnUnload`, if `--jni-generate-main=true`.

Expand Down Expand Up @@ -55,16 +59,19 @@ If want to provide your own implementation of `JNI_Onload` and `JNI_OnUnload`, t
### Includes & Build Target
Generated files for Objective-C / C++ are as follows (assuming prefix is `DB`):

| Type | C++ header | C++ source | Objective-C files | Objective-C++ files |
|------------|------------------------|------------------------------|---------------------------|-----------------------------|
| Enum/Flags | my\_enum.hpp | | DBMyEnum.h | |
| Record | my\_record[\_base].hpp | my\_record[\_base].cpp :one: | DBMyRecord[Base].h | DBMyRecord[Base]+Private.h |
| | | | DBMyRecord[Base].mm :two: | DBMyRecord[Base]+Private.mm |
| Interface | my\_interface.hpp | my\_interface.cpp :one: | DBMyInterface.h | DBMyInterface+Private.h |
| | | | | DBMyInterface+Private.mm |
| Type | C++ header | C++ source | Objective-C files | Objective-C++ files |
|------------|----------------------------|------------------------------|---------------------------|-----------------------------|
| Enum/Flags | my\_enum.hpp | | DBMyEnum.h | |
| | my\_enum+json.hpp :three: | | | |
| Record | my\_record[\_base].hpp | my\_record[\_base].cpp :one: | DBMyRecord[Base].h | DBMyRecord[Base]+Private.h |
| | my\_record[\_base]+json.hpp| | DBMyRecord[Base].mm :two: | DBMyRecord[Base]+Private.mm |
| | :three:| | | |
| Interface | my\_interface.hpp | my\_interface.cpp :one: | DBMyInterface.h | DBMyInterface+Private.h |
| | | | | DBMyInterface+Private.mm |

- :one: Generated only for types that contain constants.
- :two: Generated only for types with derived operations and/or constants. These have `.mm` extensions to allow non-trivial constants.
- :three: Generated only if cpp json serialization is enabled.

Add all generated files to your build target, and link against the [djinni-support-lib](https://github.com/cross-language-cpp/djinni-support-lib).

Expand All @@ -90,19 +97,20 @@ When bridging to Python, Djinni generates 4 types of output:

Generated files for Python / C++ are as follows:

| Type | C++ header | C++ source | Python files | CFFI | C Wrapper |
|------------|------------------------|------------------------------|---------------------|---------------------|----------------------|
| Enum/Flags | my\_enum.hpp | | my_enum.py | | dh__my_enum.cpp |
| | | | | | dh__my_enum.h |
| | | | | | dh__my_enum.hpp |
| Record | my\_record[\_base].hpp | my\_record[\_base].cpp :one: | my_record[_base].py | | dh__my_record.cpp |
| | | | | | dh__my_record.h |
| | | | | | dh__my_record.hpp |
| Interface | my\_interface.hpp | my\_interface.cpp :one: | my_interface.py | pycffi_lib_build.py | cw__my_interface.cpp |
| | | | | | cw__my_interface.h |
| | | | | | cw__my_interface.hpp |
| Type | C++ header | C++ source | Python files | CFFI | C Wrapper |
|------------|----------------------------|------------------------------|---------------------|---------------------|----------------------|
| Enum/Flags | my\_enum.hpp | | my_enum.py | | dh__my_enum.cpp |
| | my\_enum+json.hpp :two: | | | | dh__my_enum.h |
| | | | | | dh__my_enum.hpp |
| Record | my\_record[\_base].hpp | my\_record[\_base].cpp :one: | my_record[_base].py | | dh__my_record.cpp |
| | my\_record[\_base]+json.hpp| | | | dh__my_record.h |
| | :two:| | | | dh__my_record.hpp |
| Interface | my\_interface.hpp | my\_interface.cpp :one: | my_interface.py | pycffi_lib_build.py | cw__my_interface.cpp |
| | | | | | cw__my_interface.h |
| | | | | | cw__my_interface.hpp |

- :one: Generated only for types that contain constants.
- :two: Generated only if cpp json serialization is enabled.

Additional C Wrapper files are generated for data structures; their names are encoded as:

Expand Down Expand Up @@ -143,13 +151,72 @@ just needs to be added to your C# project as reference, and you can call your Dj

The following code will be generated for each defined type:

| Type | C++ header | C++ source | C++/CLI header/sources |
|------------|------------------------|----------------------------|-------------------------------------|
| Enum/Flags | my\_enum.hpp | | MyEnum.hpp, MyEnum.cpp |
| Record | my\_record.hpp | my\_record.cpp | MyRecord.hpp, MyRecord.cpp |
| Interface | my\_interface.hpp | my\_interface.cpp (+) | MyInterface.hpp, MyInterface.cpp |
| Type | C++ header | C++ source | C++/CLI header/sources |
|------------|--------------------------|----------------------------|-------------------------------------|
| Enum/Flags | my\_enum.hpp | | MyEnum.hpp, MyEnum.cpp |
| | my\_enum+json.hpp :two: | | |
| Record | my\_record.hpp | my\_record.cpp | MyRecord.hpp, MyRecord.cpp |
| | my\_record+json.hpp :two:| | |
| Interface | my\_interface.hpp | my\_interface.cpp :one: | MyInterface.hpp, MyInterface.cpp |

- :one: Generated only for types that contain constants.
- :two: Generated only if cpp json serialization is enabled.

Add all generated files to your build target, and link against the [djinni-support-lib](https://github.com/cross-language-cpp/djinni-support-lib).

C++/CLI sources have to be compiled with MSVC and the [`/clr` (Common Language Runtime Compilation)](https://docs.microsoft.com/en-us/cpp/build/reference/clr-common-language-runtime-compilation?view=msvc-160) option.

## C++ JSON Serialization support

Serialization from C++ types to/from JSON is supported. This feature is currently only enabled for `nlohmann/json`, and if enabled creates `to_json`/`from_json` methods for all djinni records and enums.

```cpp
#include "my_record.hpp"
#include "my_record+json.hpp"
#include <iostream>
#include <nlohmann/json.hpp>

void foo(const my_record& record) {
// convert record to json object
nlohmann::json j = record;
// dump serialized string
std::cerr << j.dump(4);
// create new instance of record from json object
my_record cloned = j.get<my_record>();
}
```

### Json support for the date data type ###

Since there are many ways of converting a date and from json, a simple implementation is provided by default which stores the date as the number of milliseconds elapsed since 00:00:00 UTC on January 1, 1970.

This default can be deactivated by adding a -DDJINNI_CUSTOM_JSON_DATE compilation flag to your compiler; in this case, you can implement your own date json serialiser which better matches your requirements.

One such solution using Howard Hinnant's date library could be implemented as follows:
a4z marked this conversation as resolved.
Show resolved Hide resolved

```cpp
#include <nlohmann/json.hpp>
#include <date/date.h>

namespace nlohmann {
template <>
struct adl_serializer<std::chrono::system_clock::time_point>
{
static void to_json(json &j, const std::chrono::system_clock::time_point& tp) {
j = date::format("%F %T %Z", tp);
}

static void from_json(const json &j, std::chrono::system_clock::time_point& value) {
if (j.is_null()) {
auto dur = std::chrono::milliseconds(0);
value = std::chrono::time_point<std::chrono::system_clock>(dur);
} else {
std::istringstream json_time{j.get<std::string>()};
std::chrono::system_clock::time_point parsed_time{};
// Time saved in UTC, so no need to extract time zone
json_time >> date::parse("%F %T", value);
}
}
};
}
```
a4z marked this conversation as resolved.
Show resolved Hide resolved
33 changes: 33 additions & 0 deletions src/it/resources/all_datatypes_json.djinni
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
enum_data = enum {
FirstEnumValue;
SecondEnumValue;
}

# flag comment
my_flags = flags {
# flag option comment
flag1;
flag2;
flag3;
no_flags = none;
all_flags = all;
}

all_datatypes_json = record {
booleanData: bool;
integer8Data: i8;
integer16Data: i16;
integer32Data: i32;
integer64Data: i64;
float32Data: f32;
float64Data: f64;
stringData: string;
binaryData: binary;
dateData: date;
listData: list<bool>;
setData: set<bool>;
mapData: map<i8, bool>;
optionalData: optional<bool>;
enum_data: enum_data;
myFlags: my_flags;
}
16 changes: 16 additions & 0 deletions src/it/resources/all_json_specialized_datatypes.djinni
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
my_enum = enum {
FirstEnumValue;
SecondEnumValue;
}

my_flags = flags {
flag1;
flag2;
no_flags = none;
all_flags = all;
}

my_record = record {
myEnum: my_enum;
myFlags: my_flags;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
// AUTOGENERATED FILE - DO NOT MODIFY!
// This file was generated by Djinni from all_datatypes_json.djinni

#pragma once

#include "enum_data+json.hpp"
#include "json+extension.hpp"
#include "my_flags+json.hpp"
#include <chrono>
#include <cstdint>
#include <nlohmann/json.hpp>
#include <optional>
#include <string>
#include <unordered_map>
#include <unordered_set>
#include <vector>

namespace nlohmann {

template <>
struct adl_serializer<::AllDatatypesJson> {
static ::AllDatatypesJson from_json(const json & j) {
auto result = ::AllDatatypesJson();
if (j.contains("booleanData")) {
j.at("booleanData").get_to(result.booleanData);
}
if (j.contains("integer8Data")) {
j.at("integer8Data").get_to(result.integer8Data);
}
if (j.contains("integer16Data")) {
j.at("integer16Data").get_to(result.integer16Data);
}
if (j.contains("integer32Data")) {
j.at("integer32Data").get_to(result.integer32Data);
}
if (j.contains("integer64Data")) {
j.at("integer64Data").get_to(result.integer64Data);
}
if (j.contains("float32Data")) {
j.at("float32Data").get_to(result.float32Data);
}
if (j.contains("float64Data")) {
j.at("float64Data").get_to(result.float64Data);
}
if (j.contains("stringData")) {
j.at("stringData").get_to(result.stringData);
}
if (j.contains("binaryData")) {
j.at("binaryData").get_to(result.binaryData);
}
if (j.contains("dateData")) {
j.at("dateData").get_to(result.dateData);
}
if (j.contains("listData")) {
j.at("listData").get_to(result.listData);
}
if (j.contains("setData")) {
j.at("setData").get_to(result.setData);
}
if (j.contains("mapData")) {
j.at("mapData").get_to(result.mapData);
}
if (j.contains("optionalData")) {
j.at("optionalData").get_to(result.optionalData);
}
if (j.contains("enum_data")) {
j.at("enum_data").get_to(result.enum_data);
}
if (j.contains("myFlags")) {
j.at("myFlags").get_to(result.myFlags);
}
return result;
}
static void to_json(json & j, const ::AllDatatypesJson & item) {
j = json {
{"booleanData", item.booleanData},
{"integer8Data", item.integer8Data},
{"integer16Data", item.integer16Data},
{"integer32Data", item.integer32Data},
{"integer64Data", item.integer64Data},
{"float32Data", item.float32Data},
{"float64Data", item.float64Data},
{"stringData", item.stringData},
{"binaryData", item.binaryData},
{"dateData", item.dateData},
{"listData", item.listData},
{"setData", item.setData},
{"mapData", item.mapData},
{"optionalData", item.optionalData},
{"enum_data", item.enum_data},
{"myFlags", item.myFlags}
};
}
};

} // namespace nlohmann
Loading