Repository navigation
v0.5.0
An event keeps the JSON document its decoder built, so a typed read or write
parses the payload at most once and never serialises it to text in between.
That changes data_t, so it ships as a third API generation, ce::v3, which is
the inline namespace. ce::v2 keeps what v0.4.0 published, so v0.4.0 code has a
way to keep compiling.
Three generations
ce::v3is the inline namespace, soce::eventnow names the v3 class.ce::v2holds the v0.4.0 surface.
The headers that mentioneventordata_tare copied underinclude/cloudevents/v2/:core.hpp,format/json_format.hpp,format/typed_payload.hppand the four binding headers.
A v2 declaration does not change; v2 takes defect fixes only.- The attribute types are shared by v2 and v3:
id,source,timestamp,message, the literals, the typed extensions and the rest ofattributes.hppare declared once, ince::v2, and are one type throughce::,ce::v2::andce::v3::. - What v0.3.0 already shared stays shared:
errc,error,result, the three codecs, base64 and the describe seam are one type through all three generations. - v3 codecs provide
equal,copy,extract,for_each_mutable_elementandidentity.
ce::v3::json::json_codecadds five members to the v1 concept:C::equal(const value&, const value&) -> bool, JSON value equality with object members in any order;C::copy(const value&) -> value, a deep copy;C::extract(value& object, std::string_view key) -> value, which moves a member out of an object and returns null for a keyfindwould not return;C::for_each_mutable_element(value& array, F visit), which hands each element tovisitas avalue&; andC::identity, a non-emptystatic constexpr std::string_viewnaming the codec.
The three in-tree codecs gain all five, and keep their v1 declarations; their identities areio.cloudevents.cpp.codec.nlohmann,io.cloudevents.cpp.codec.boost_jsonandio.cloudevents.cpp.codec.rapidjson.
A third-party codec must add them to servece::v3, with an identity that is a reverse-DNS name under a domain its author controls and that no other codec uses.
It keeps servingce::v1andce::v2unchanged, since their concept is the v1 one. ce::json_documentholds a parsed JSON document incore.hpp, which still names no codec type.
json_document::make<Codec>(dom)builds one,get<Codec>()returns the DOM to a codec declaring the identity of the one that built it andnullptrto any other, anddump()serialises it.
Documents compare as JSON values, across codecs too.
Copies share one immutable DOM, so a document can be copied, compared and read from several threads at once.
A document is never empty: it has no default constructor, and moving one copies it.data_thas a fifth alternative,json_document.
Code that visitsdata_texhaustively stops compiling againstce::v3until it handles it; that is the migration.
An event holdingjson_textnever equals one holdingjson_document, even for the same JSON value, so equality stays transitive.- A structured decode keeps the payload as a
json_documentbuilt by the decoding codec, where v2 gavejson_text.
decodeanddecode_batchmove thedatamember out of the document they parsed, so the payload is neither serialised nor copied;from_valuecopies it.
A binary-mode body, a non-JSON string payload anddata_base64decode as before. ce::json::decode_options{.retain_document_up_to = 16 * 1024}bounds what one event can pin: input longer than the limit keeps its payload asjson_text, and 0 always does.
The default is 16 KiB, nameddecode_options::default_retention_limit: a retained document measured up to 3.3 times the bytes of its text, so larger payloads stay text unless the caller raises the limit.
decodeanddecode_batchtake it. A batch keeps documents when its text is at most the limit times its number of events, and otherwise every event in it keeps text.- A payload kept as text above the limit is the input's own text of the
datamember, without the whitespace around it, rather than the codec's serialisation of it.
Copying the slice avoids the serialisation, which measured 25 to 49 percent more instructions and 58 to 210 percent more allocated bytes indecode_large.
The decoder falls back to the codec's serialisation when a top-level member name carries an escape, whendataappears twice, or when the input holds something strict JSON does not allow; a batch decides per element.
The text keeps the sender's spelling, so the same JSON formatted differently no longer compares equal asjson_textafter decode. - Encoding a
json_documentbuilt by the encoding codec copies its DOM, with no serialisation and no parse.
A document from another codec converts through the building codec's text. data_as<T, Codec>reads a document built byCodecdirectly, throughget<Codec>(), with no serialisation and no parse.
A document from another codec still converts through text.set_data<T, Codec>stores ajson_documentbuilt byCodec, where v2 storedjson_text.
An event written byset_datatherefore equals the same payload held as a document, never as text.- Typed entry points parse and write once.
ce::decode_as<T, Codec>(text, options)returnsce::decoded<T>{.event, .payload}: the eventdecodereturns, and the payload read from the parseddatamember.
ce::decode_batch_asdoes the same for a batch and fails the whole batch when one payload does not read asT;ce::from_value_asreads a document the caller parsed.
ce::encode_as<T, Codec>(event, payload)writes the payload's DOM underdatawithout changing the event, writingapplication/jsonwhendatacontenttypeis absent and refusing a non-JSON one withtype_mismatch.
They fail asdata_asdoes when the payload is absent (missing_required_attribute) or is not JSON (type_mismatch).
The optional module exports them, andjson::decode_options. - The optional module exports
ce::v3only.
CR-0003 and ADR-0010 record why: the v0.5.0 event model adds an alternative to data_t, and SPEC section 3 rule 4 sends a breaking change to a new generation.
Staying on v0.4.0 code
ce::X is now the v3 entity.
To keep the v0.4.0 declarations, spell ce::v2::X and include the v2/ copy of each header that has one:
#include <cloudevents/v2/binding/http.hpp>
using namespace ce::v2::literals;
auto order = ce::v2::event::builder{.id = "A1"_id, .source = "/orders"_source,
.type = "com.example.placed"_type}.build();
auto request = ce::v2::http::to_message<ce::v2::codec::nlohmann_codec>(
*order, ce::v2::content_mode::binary_mode);Code that names only shared entities, such as an attribute type or a codec, needs no change.
Moving to ce::v3
data_thas a fifth alternative. Add ajson_documentcase wherever you visitdata_t;document.dump()gives its compact text.
Code that readstd::get<ce::json_text>(event.data()).rawafter a structured decode now finds ajson_document: read it withdocument.get<Codec>()for the DOM, ordocument.dump()for text.
Code that compares a decoded event with one built fromjson_textnow compares unequal; build the expected event with ajson_document, or compare the payloads by value.- A codec you wrote needs five more members to serve
ce::v3:equal,copy,extract,for_each_mutable_elementand astatic constexpr std::string_view identity, as listed above.
static_assert(ce::json::json_codec<MyCodec>)fails until all five are present.
Choose an identity under a domain you control; two codecs sharing one would hand each other's DOM to the wrong type.
Without the five members the codec still servesce::v1andce::v2unchanged. decode_optionsbounds what a decoded event pins, 16 KiB by default.
decode,decode_batch,decode_asanddecode_batch_astakece::json::decode_optionsas a second argument; omitting it keeps a document for input up todecode_options::default_retention_limit(16 KiB) and text above it.
Pass{.retain_document_up_to = 0}to keep the v2 behaviour of always decoding to text. The text is now the sender's own spelling of the payload, where v2 gave the codec's serialisation.
Raise the limit when large payloads are read as typed values and memory allows it.set_data<T, Codec>stores a document. Code that readstd::get<ce::json_text>(event.data())afterset_datanow finds ajson_document; read the payload withdata_as, or the text withdump().- Use the typed entry points. Replace
decodefollowed bydata_aswithdecode_as,decode_batchfollowed by a loop ofdata_aswithdecode_batch_as, andset_dataon a copy followed byencodewithencode_as, to save a parse or a serialisation.
Tooling
- Every pull request is measured against main and against budgets.
Theperfworkflow gates on instructions under Callgrind, heap allocations and the bytes a decoded event retains, per operation and codec, and comments the table on the pull request.
Wall time and binary size are reported.
Each merge to main is recorded on thebench-databranch.
docs/PERFORMANCE.md explains the table and how to accept a deliberate cost. - The bench's Boost.JSON codec builds its values with parentheses, so it works with Boost before 1.84.
Faster, in every generation
- A decode reads a document in one pass over its members, instead of one
findper reserved attribute and a second walk for the extensions.
Measured withbench/codec_benchbefore the perf job existed: 7 to 20 percent less CPU time for an event-sized document on all four bench codecs, 7 to 14 percent for a batch of 100. - The bindings stop copying bytes the SDK already holds.
A structured decode parses the message body in place, a binary-mode decode checks header uniqueness without building a second header container, anddecode_batchreserves its vector.
nlohmann_codec::findandextractlook a key up without building astd::string, andrapidjson_codec::dumpreturns its buffer without copying it.
These are body-only changes that keep every declaration and every result, so the frozen generations take them as fixes (D-CODEC-3).
Measured by the perf job
The perf job (x86_64, g++-14, Valgrind 3.22, malloc counted) records main on the bench-data branch.
The figures compare its first record, b8757ca, where a decode still kept its payload as the codec's serialised text, with the release candidate, de63b9e.
A retained figure is the heap a decoded event holds.
- Retained memory. A 52,889-byte event retains 52,799 bytes on every codec, which is its payload's own text: down from 76,865 (nlohmann), 79,907 (Glaze) and 65,601 (Boost.JSON), and level on RapidJSON.
A 400-byte event under the 16 KiB limit keeps its document, which costs more than the text did: 652 to 1,507 bytes on nlohmann, 633 to 859 on RapidJSON, 633 to 1,045 on Boost.JSON and 1,044 to 1,539 on Glaze. - Typed read.
data_asof a payload written by the same codec runs 63 to 83 percent fewer instructions (nlohmann 16,243 to 2,711; RapidJSON 6,215 to 2,272) with 2 allocations instead of 7 to 22.
The earlier probe read a payload nlohmann wrote for every codec, and this one reads a payload the reading codec wrote (D-PERF-1). - Typed write.
set_dataruns 23 to 52 percent fewer instructions (nlohmann 11,084 to 6,348; RapidJSON 4,964 to 2,382).
encode_asof the full event costs 30 to 31 percent fewer instructions thanencodeon nlohmann and Glaze and 16 to 21 percent fewer on RapidJSON and Boost.JSON, measured in the same run. - Decode.
decode_largeruns 5 to 19 percent fewer instructions and allocates 23 to 55 percent fewer bytes;decode_full3 to 7 percent fewer instructions;decode_batch_1000 to 12 percent fewer instructions and 34 to 57 percent fewer allocated bytes. - Bindings, measured when they entered the job, before the copies were removed and after: a binary-mode decode makes 10 allocations instead of 21 on HTTP and Kafka and 22 on NATS, and allocates 675 bytes instead of 2,866 on HTTP; HTTP binary decode runs 10.8 percent fewer instructions, Kafka 12.7 and NATS 11.0; an HTTP structured decode runs 9 to 18 percent fewer.
- Binary size. The stripped minimal consumer grew 3.1 to 5.5 percent (nlohmann 297,264 to 313,648 bytes).
Fixed
- The macro contract matches the headers.
CE_DESCRIBEis the only public macro;CE_FIELD,CE_HAS_*andCE_DETAIL_*are reserved names the headers leave defined, which you must not define or use, exceptCE_FIELDinside aCE_DESCRIBElist.
The documentation had promised the helpers were undefined, which they cannot be:CE_DESCRIBEexpands into them where you write it.
A suite now reads the preprocessor's own list after including every public header and fails on any otherCE_macro, and on a documented one that is gone. - Regenerating the interop goldens no longer reorders a Go golden's extensions, which Go's randomised map iteration had shuffled on every run.
Measured on the release candidate
- Line coverage 95.9% (3473 of 3621), function coverage 92.1% (1565 of 1699), branch coverage 59.8% (3982 of 6656), against a floor of 90% lines, from the
coverage floorjob. - CI: GCC 13 at C++20 and C++23, Clang 16 with libstdc++ 13, Clang 17 with libc++ 17, AppleClang at C++20 and C++23, MSVC 19, GCC 16 with C++26 static reflection, plus the forced polyfill, exceptions disabled, no default codec, every codec on Linux and macOS, ASan with UBSan, TSan, install-and-consume, and interop with the Go and Java SDKs.
- Twenty-one fuzz targets ran on pull requests at one minute each.
Known limitations
- libc++ 17 and 18 implement
std::formatwithout defining__cpp_lib_format; the SDK carves them out by version (D-CI-1). - Clang 16 cannot compile libstdc++ 13 or 14 in C++23 mode, so the C++23 Clang job uses libc++ (D-CI-1).
- The module interface is usable, with constraints on what an importing translation unit may also include (D-MODULE-1), and it exports
ce::v3only. - Given the same non-JSON payload, the Java SDK writes
data_base64where Go and this SDK write a JSON string. Both are legal (D-INTEROP-1). - HTTP binary mode percent-encodes header values as the binding requires; the Go and Java SDKs do not, so talking to them needs
ce::http::literal_values(D-HTTP-1). - A decoded document under the retention limit takes 1.4 to 3.3 times the memory of its text, measured above; lower the limit where many events are held at once.