Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

conversion from/to user-defined types #435

Merged
merged 101 commits into from
Jan 27, 2017
Merged
Show file tree
Hide file tree
Changes from 99 commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
b443edf
add first version support for user-defined types
Oct 16, 2016
fe628b5
anonymous namespace renamed to detail
theodelrieu Oct 17, 2016
d54d6bb
add alias templates to reduce boilerplate
theodelrieu Oct 18, 2016
877d96c
rename __static_const to _static_const (reserved identifier)
Oct 20, 2016
12b4555
use uncvref_t<T> instead of remove_cv_t<remove_reference_t<T>>>
Oct 20, 2016
03b391c
remove has_destructor and has_json_traits, use decltype instead
Oct 20, 2016
4cdc61e
move most SFINAE trickery in to/from_json_fn
Oct 21, 2016
7dc268e
add first version for alternate implementation
theodelrieu Nov 7, 2016
33abccf
add template arguments for JSONSerializer
Nov 8, 2016
837b81d
renamed unit-constructor3.cpp to unit-udt.cpp
Nov 8, 2016
2bc685f
to_json and from_json takes both two arguments now
theodelrieu Nov 9, 2016
178441c
add basic test for custom serializer
Nov 15, 2016
23bd2bc
add is_compatible_* traits
theodelrieu Nov 16, 2016
8881944
remove constraints on JSONSerializer instances, constrain correctly n…
theodelrieu Nov 16, 2016
0d91113
add negation, disjunction, and conjunction from C++17
theodelrieu Nov 20, 2016
e2dbe7a
correctly constrain basic_json udt constructor
Nov 21, 2016
9b40197
add a macro for has_xxx types
Nov 17, 2016
ee19aca
add is_compatible_basic_json_type trait
Nov 21, 2016
47bc402
only use conjunction & co when needed to avoid MSVC crashing
Nov 22, 2016
907484f
format biggest lines
theodelrieu Nov 23, 2016
74bb11d
remove some useless checks, format a bit, added some comments
Nov 24, 2016
e5999c6
add a few tests
theodelrieu Nov 26, 2016
60e6f82
add support for non-default-constructible udt
Nov 29, 2016
c0c72b5
rewrite unit-udt: basic usage
Nov 30, 2016
1eafac7
remove explicit keyword on udt-constructor
theodelrieu Dec 4, 2016
f5cb089
add an enum constructor (quickfix)
theodelrieu Dec 7, 2016
8e43d47
add more tests to unit-udt.cpp
theodelrieu Dec 13, 2016
3d405c6
add support for enum classes
theodelrieu Dec 13, 2016
7e750ec
fix msvc, by doubling parenthesis on catch assertions
theodelrieu Dec 14, 2016
1c21c87
use u8 prefix in unit-udt.cpp
theodelrieu Dec 14, 2016
d5ee583
add more tests
theodelrieu Dec 14, 2016
aa2679a
fix tests, avoid instantiating JSONSerializer when it will not be used
Dec 15, 2016
be1d3de
:lipstick: moved changes to re2c file and ran `make pretty`
nlohmann Dec 18, 2016
034d5ed
:lipstick: some cleanup
nlohmann Dec 18, 2016
d359684
move enum class value_t outside of basic_json
Jan 7, 2017
c833b22
move type_name outside of basic_json, make it a friend
Jan 7, 2017
6b89785
replace constructor by from/to_json: boolean_t
Jan 7, 2017
bbe4064
replace constructor by from/to_json: string_t
Jan 7, 2017
d257149
replace constructor by from/to_json: number_float_t
Jan 7, 2017
a32de3b
replace constructor by from/to_json: number_unsigned_t
Jan 8, 2017
f008983
replace constructor by from/to_json: number_integer_t
Jan 8, 2017
6d427ac
replace constructor by from/to_json: unscoped enum types
Jan 8, 2017
c847e0e
replace constructor by from/to_json: array_t
Jan 8, 2017
7e6a6f9
replace constructor by from/to_json: array_t
Jan 8, 2017
4e8089b
remove old get/get_impl overloads (doc removal is of course temporary)
Jan 8, 2017
317883b
cleanup, details in body
Jan 8, 2017
be6b417
tweak SFINAE checks for internal types (see commit body)
Jan 8, 2017
b2543e0
removed is_compatible_float_type trait
Jan 8, 2017
b4cea68
remove is_compatible_array_type_impl trait
Jan 8, 2017
5839795
remove useless helpers
Jan 8, 2017
29f9fe6
remove one has_to/from_json template parameter
Jan 8, 2017
1f25ec5
add some constexpr + noexcept
Jan 8, 2017
3494014
new unit-udt.cpp tests
Jan 8, 2017
cb3d455
do not const_cast when calling get_ptr
Jan 9, 2017
e678c07
check for is_number_unsigned before is_number_integer
Jan 9, 2017
d0d8070
add static_asserts
theodelrieu Jan 12, 2017
e247e01
use static_casts to silence MSVC warnings
theodelrieu Jan 12, 2017
a9d5ae4
put back a specialization for containers with a reserve method
Jan 14, 2017
1554baa
attempting to fix coverage. testing that reserve is called
Jan 14, 2017
b801287
add noexcept checks, and some missing noexcepts
Jan 14, 2017
63e4249
use a priority_tag instead of int and longs with sfinae-dispatch
Jan 14, 2017
f2c71fa
minor message/comments tweak
Jan 14, 2017
f1482d1
more tests in unit-udt
Jan 14, 2017
07bc82d
put const to the left
Jan 15, 2017
68081cd
remove useless file
Jan 15, 2017
794dae8
apply changes to json.hpp.re2c
Jan 15, 2017
e60e458
move static_const to the detail namespace
Jan 15, 2017
1d87097
remove some boilerplate
Jan 15, 2017
af94e71
add basic doxygen documentations
Jan 15, 2017
b56117b
add noexcept tests
Jan 16, 2017
fbac056
add readme
Jan 16, 2017
3e15b55
run make pretty
Jan 17, 2017
447c6a6
run make re2c
Jan 17, 2017
1e20887
use JSON_THROW
Jan 20, 2017
d566bb8
use switchs where appropriate
Jan 20, 2017
a6b0282
move conjunction/disjunction to the top of detail
Jan 20, 2017
889b269
rename template argument: BasicJson -> Json
Jan 20, 2017
708eb96
disable reserve tests when exceptions are disabled
Jan 21, 2017
7f35901
rename template argument Json -> BasicJsonType
Jan 21, 2017
40ba5a8
remove inline keyword from anonymous namespace (not needed)
Jan 21, 2017
f997758
remove useless forward declaration of primitive_iterator_t
Jan 21, 2017
7d771c7
put back type_name in basic_json
Jan 21, 2017
37fd20b
put uncvref_t, enable_if_t, value_t and operator<(value_t) in detail
Jan 21, 2017
ba0b35f
use typename U = uncvref_t where appropriate
Jan 21, 2017
9c6ef74
add static_asserts/remove the need for CopyConstructible in get
Jan 21, 2017
9f8b270
fix some warnings
Jan 22, 2017
9f103d1
add implicit conversions test (operator T())
Jan 22, 2017
3857e55
:lipstick: ran "make pretty"
nlohmann Jan 24, 2017
030cf67
:construction: fixed a warning
nlohmann Jan 24, 2017
250e5bf
:lipstick: overworked documentation and indentation
nlohmann Jan 24, 2017
daf8dcd
:memo: overworked README
nlohmann Jan 24, 2017
781fd09
:memo: cleanup
nlohmann Jan 24, 2017
50a3f3b
:memo: added nicer example
nlohmann Jan 24, 2017
c154f31
:construction: rephrased assertions
nlohmann Jan 24, 2017
4139bb6
:memo: adjusted test count
nlohmann Jan 24, 2017
ec03c9c
:memo: overworked docs #435
nlohmann Jan 25, 2017
94d9b7b
:memo: overworked documentation (re-added constructor examples)
nlohmann Jan 25, 2017
4d3053c
:lipstick: some cleanup
nlohmann Jan 25, 2017
77bb7af
:memo: added more documentation
nlohmann Jan 26, 2017
1305e03
:memo: fixed documentation
nlohmann Jan 26, 2017
cd9701b
:lipstick: cleanup
nlohmann Jan 26, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
232 changes: 228 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
- [Conversion from STL containers](#conversion-from-stl-containers)
- [JSON Pointer and JSON Patch](#json-pointer-and-json-patch)
- [Implicit conversions](#implicit-conversions)
- [Conversions to/from arbitrary types](#arbitrary-types-conversions)
- [Binary formats (CBOR and MessagePack)](#binary-formats-cbor-and-messagepack)
- [Supported compilers](#supported-compilers)
- [License](#license)
Expand Down Expand Up @@ -442,6 +443,225 @@ int vi = jn.get<int>();
// etc.
```

### Arbitrary types conversions

Every type can be serialized in JSON, not just STL-containers and scalar types. Usually, you would do something along those lines:

```cpp
namespace ns {
// a simple struct to model a person
struct person {
std::string name;
std::string address;
int age;
};
}

ns::person p = {"Ned Flanders", "744 Evergreen Terrace", 60};

// convert to JSON: copy each value into the JSON object
json j;
j["name"] = p.name;
j["address"] = p.address;
j["age"] = p.age;

// ...

// convert from JSON: copy each value from the JSON object
ns::person p {
j["name"].get<std::string>(),
j["address"].get<std::string>(),
j["age"].get<int>()
};
```

It works, but that's quite a lot of boilerplate... Fortunately, there's a better way:

```cpp
// create a person
ns::person p {"Ned Flanders", "744 Evergreen Terrace", 60};

// conversion: person -> json
json j = p;

std::cout << j << std::endl;
// {"address":"744 Evergreen Terrace","age":60,"name":"Ned Flanders"}

// conversion: json -> person
ns::person p2 = j;

// that's it
assert(p == p2);
```

#### Basic usage

To make this work with one of your types, you only need to provide two functions:

```cpp
using nlohmann::json;

namespace ns {
void to_json(json& j, const person& p) {
j = json{{"name", p.name}, {"address", p.address}, {"age", p.age}};
}

void from_json(const json& j, person& p) {
p.name = j["name"].get<std::string>();
p.address = j["address"].get<std::string>();
p.age = j["age"].get<int>();
}
} // namespace ns
```

That's all! When calling the `json` constructor with your type, your custom `to_json` method will be automatically called.
Likewise, when calling `get<your_type>()`, the `from_json` method will be called.

Some important things:

* Those methods **MUST** be in your type's namespace (which can be the global namespace), or the library will not be able to locate them (in this example, they are in namespace `ns`, where `person` is defined).
* When using `get<your_type>()`, `your_type` **MUST** be [DefaultConstructible](http://en.cppreference.com/w/cpp/concept/DefaultConstructible). (There is a way to bypass those requirements described later.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rephrase the sentence in parenthesis : (There is a way to bypass this requirement described later).


#### How do I convert third-party types?

This requires a bit more advanced technique. But first, let's see how this conversion mechanism works:

The library uses **JSON Serializers** to convert types to json.
The default serializer for `nlohmann::json` is `nlohmann::adl_serializer` (ADL means [Argument-Dependent Lookup](http://en.cppreference.com/w/cpp/language/adl)).

It is implemented like this (simplified):

```cpp
template <typename T>
struct adl_serializer {
static void to_json(json& j, const T& value) {
// calls the "to_json" method in T's namespace
}

static void from_json(const json& j, T& value) {
// same thing, but with the "from_json" method
}
};
```

This serializer works fine when you have control over the type's namespace. However, what about `boost::optional`, or `std::filesystem::path` (C++17)? Hijacking the `boost` namespace is pretty bad, and it's illegal to add something other than template specializations to `std`...

To solve this, you need to add a specialization of `adl_serializer` to the `nlohmann` namespace, here's an example:

```cpp
// partial specialization (full specialization works too)
namespace nlohmann {
template <typename T>
struct adl_serializer<boost::optional<T>> {
static void to_json(json& j, const boost::optional<T>& opt) {
if (opt == boost::none) {
j = nullptr;
} else {
j = *opt; // this will call adl_serializer<T>::to_json which will
// find the free function to_json in T's namespace!
}
}

static void from_json(const json& j, boost::optional<T>& opt) {
if (!j.is_null()) {
opt = j.get<T>(); // same as above, but with
// adl_serializer<T>::from_json
}
}
};
}
```

#### How can I use `get()` for non-default constructible/non-copyable types?

There is a way, if your type is [MoveConstructible](http://en.cppreference.com/w/cpp/concept/MoveConstructible). You will need to specialize the `adl_serializer` as well, but with a special `from_json` overload:

```cpp
struct move_only_type {
move_only_type() = delete;
move_only_type(int ii): i(ii) {}
move_only_type(const move_only_type&) = delete;
move_only_type(move_only_type&&) = default;

private:
int i;
};

namespace nlohmann {
template <>
struct adl_serializer<move_only_type> {
// note: the return type is no longer 'void', and the method only takes
// one argument
static move_only_type from_json(const json& j) {
return {j.get<int>()};
}

// Here's the catch! You must provide a to_json method! Otherwise you
// will not be able to convert move_only_type to json, since you fully
// specialized adl_serializer on that type
static void to_json(json& j, move_only_type t) {
j = t.i;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

t.i is private here

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right... Did I change this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, but just removing private: is fine for this example

}
};
}
```

#### Can I write my own serializer? (Advanced use)

Yes. You might want to take a look at [`unit-udt.cpp`](https://github.com/nlohmann/json/blob/develop/test/src/unit-udt.cpp) in the test suite, to see a few examples.

If you write your own serializer, you'll need to do a few things:

* use a different `basic_json` alias than `nlohmann::json` (the last template parameter of `basic_json` is the `JSONSerializer`)
* use your `basic_json` alias (or a template parameter) in all your `to_json`/`from_json` methods
* use `nlohmann::to_json` and `nlohmann::from_json` when you need ADL

Here is an example, without simplifications, that only accepts types with a size <= 32, and uses ADL.

```cpp
// You should use void as a second template argument
// if you don't need compile-time checks on T
template<typename T, typename SFINAE = typename std::enable_if<sizeof(T) <= 32>::type>
struct less_than_32_serializer {
template <typename BasicJsonType>
static void to_json(BasicJsonType& j, T value) {
// we want to use ADL, and call the correct to_json overload
using nlohmann::to_json; // this method is called by adl_serializer,
// this is where the magic happens
to_json(j, value);
}

template <typename BasicJsonType>
static void from_json(const BasicJsonType& j, T& value) {
// same thing here
using nlohmann::from_json;
from_json(j, value);
}
};
```

Be **very** careful when reimplementing your serializer, you can stack overflow if you don't pay attention:

```cpp
template <typename T, void>
struct bad_serializer
{
template <typename BasicJsonType>
static void to_json(BasicJsonType& j, const T& value) {
// this calls BasicJsonType::json_serializer<T>::to_json(j, value);
// if BasicJsonType::json_serializer == bad_serializer ... oops!
j = value;
}

template <typename BasicJsonType>
static void to_json(const BasicJsonType& j, T& value) {
// this calls BasicJsonType::json_serializer<T>::from_json(j, value);
// if BasicJsonType::json_serializer == bad_serializer ... oops!
value = j.template get<T>(); // oops!
}
};
```

### Binary formats (CBOR and MessagePack)

Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [CBOR](http://cbor.io) (Concise Binary Object Representation) and [MessagePack](http://msgpack.org) to efficiently encode JSON values to byte vectors and to decode such vectors.
Expand Down Expand Up @@ -546,7 +766,7 @@ I deeply appreciate the help of the following people.
- [Eric Cornelius](https://github.com/EricMCornelius) pointed out a bug in the handling with NaN and infinity values. He also improved the performance of the string escaping.
- [易思龙](https://github.com/likebeta) implemented a conversion from anonymous enums.
- [kepkin](https://github.com/kepkin) patiently pushed forward the support for Microsoft Visual studio.
- [gregmarr](https://github.com/gregmarr) simplified the implementation of reverse iterators and helped with numerous hints and improvements.
- [gregmarr](https://github.com/gregmarr) simplified the implementation of reverse iterators and helped with numerous hints and improvements. In particular, he pushed forward the implementation of user-defined types.
- [Caio Luppi](https://github.com/caiovlp) fixed a bug in the Unicode handling.
- [dariomt](https://github.com/dariomt) fixed some typos in the examples.
- [Daniel Frey](https://github.com/d-frey) cleaned up some pointers and implemented exception-safe memory allocation.
Expand Down Expand Up @@ -574,7 +794,7 @@ I deeply appreciate the help of the following people.
- [duncanwerner](https://github.com/duncanwerner) found a really embarrassing performance regression in the 2.0.0 release.
- [Damien](https://github.com/dtoma) fixed one of the last conversion warnings.
- [Thomas Braun](https://github.com/t-b) fixed a warning in a test case.
- [Théo DELRIEU](https://github.com/theodelrieu) patiently and constructively oversaw the long way toward [iterator-range parsing](https://github.com/nlohmann/json/issues/290).
- [Théo DELRIEU](https://github.com/theodelrieu) patiently and constructively oversaw the long way toward [iterator-range parsing](https://github.com/nlohmann/json/issues/290). He also implemented the magic behind the serialization/deserialization of user-defined types.
- [Stefan](https://github.com/5tefan) fixed a minor issue in the documentation.
- [Vasil Dimov](https://github.com/vasild) fixed the documentation regarding conversions from `std::multiset`.
- [ChristophJud](https://github.com/ChristophJud) overworked the CMake files to ease project inclusion.
Expand All @@ -588,6 +808,9 @@ I deeply appreciate the help of the following people.
- [Bosswestfalen](https://github.com/Bosswestfalen) merged two iterator classes into a smaller one.
- [Daniel599](https://github.com/Daniel599) helped to get Travis execute the tests with Clang's sanitizers.
- [Jonathan Lee](https://github.com/vjon) fixed an example in the README file.
- [gnzlbg](https://github.com/gnzlbg) supported the implementation of user-defined types.
- [Alexej Harm](https://github.com/qis) helped to get the user-defined types working with Visual Studio.
- [Jared Grubb](https://github.com/jaredgrubb) supported the implementation of user-defined types.

Thanks a lot for helping out! Please [let me know](mailto:mail@nlohmann.me) if I forgot someone.

Expand All @@ -611,10 +834,11 @@ Thanks a lot for helping out! Please [let me know](mailto:mail@nlohmann.me) if I
To compile and run the tests, you need to execute

```sh
$ make check
$ make json_unit -Ctest
$ ./test/json_unit "*""

===============================================================================
All tests passed (11202040 assertions in 44 test cases)
All tests passed (11202052 assertions in 47 test cases)
```

Alternatively, you can use [CMake](https://cmake.org) and run
Expand Down
2 changes: 1 addition & 1 deletion doc/Doxyfile
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ RECURSIVE = NO
EXCLUDE =
EXCLUDE_SYMLINKS = NO
EXCLUDE_PATTERNS =
EXCLUDE_SYMBOLS = nlohmann::anonymous_namespace
EXCLUDE_SYMBOLS = nlohmann::detail
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we bump the version number here too?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I shall change the version number once I created a release branch.

EXAMPLE_PATH = examples
EXAMPLE_PATTERNS =
EXAMPLE_RECURSIVE = NO
Expand Down
58 changes: 0 additions & 58 deletions doc/examples/basic_json__CompatibleArrayType.cpp

This file was deleted.

1 change: 0 additions & 1 deletion doc/examples/basic_json__CompatibleArrayType.link

This file was deleted.

9 changes: 0 additions & 9 deletions doc/examples/basic_json__CompatibleArrayType.output

This file was deleted.

27 changes: 0 additions & 27 deletions doc/examples/basic_json__CompatibleIntegerNumberType.cpp

This file was deleted.

This file was deleted.

5 changes: 0 additions & 5 deletions doc/examples/basic_json__CompatibleIntegerNumberType.output

This file was deleted.