feat(JSON): add skip_null_fields option to omit null-valued record fields

[ggreif]

Motivation

External HTTP APIs frequently reject request bodies that carry optional fields as explicit null — they validate field types strictly and consider "field": null a type mismatch. Twitter's POST /2/tweets is a concrete example: it rejects 16 such entries in a single request when the generated client forwards a partly-filled payload.

Motoko records serialised through to_candid |> JSON.toText always emit every field, including ?T = null values, which hits exactly that rejection case for any OpenAPI-style client that mirrors a nullable-optional spec field as an ?T on the Motoko side. Today there is no option to suppress them in serde.

What this PR does

Adds a new skip_null_fields : Bool option to CandidType.Options:

let options = { Candid.defaultOptions with skip_null_fields = true };
JSON.toText(blob, keys, ?options)

When true, the JSON encoder omits entries in #Record/#Map whose value resolves to #Null. #Option(#Null) — which is what ?T = null becomes after the candid round-trip — is the common case; deeply nested #Option(#Option(#Null)) also collapses correctly.

Defaults to false, so all existing runtime behaviour is preserved.

Also exposes fromCandidWith(candid, skip_null_fields) for callers that already hold a decoded Candid value and want the same behaviour without going through toText.

⚠️ Breaking change (semver-major bump)

Adding a required field to the public Options record is a type-level breaking change under Motoko's structural record typing. Code that constructs Options as a full record literal will no longer type-check:

// breaks after this change:
let opts : Options = {
  renameKeys = [];
  use_icrc_3_value_type = false;
  types = null;
};

The idiomatic { Candid.defaultOptions with … } pattern is unaffected — it inherits the new field from defaultOptions = false. All of serde's own tests and (spot-checked) the main consumers I know of use that pattern and continue to compile and pass.

Recommended version bump: 3.5.0 → 4.0.0. Happy to land the change under whichever bump the maintainers prefer.

Before / after

type Post = {
  text : Text;
  for_super_followers_only : ?Bool;   // null
  poll : ?Text;                       // null
  reply_settings : ?Text;             // null
};

let post = { text = "hello"; for_super_followers_only = null; poll = null; reply_settings = null };
let blob = to_candid(post);

// Default (existing):
JSON.toText(blob, keys, null)
// => {"text": "hello", "for_super_followers_only": null, "poll": null, "reply_settings": null}

// New:
let opts = { Candid.defaultOptions with skip_null_fields = true };
JSON.toText(blob, keys, ?opts)
// => {"text": "hello"}

Tests

tests/JSON.Test.mo gains a skip_null_fields omits null optional fields case that verifies:

• Default behaviour still emits null entries for optional fields.
• With the flag, null entries are omitted and the output matches {"text":"hello"} exactly.
• Non-null optionals continue to round-trip (?false → false, ?"everyone" → "everyone").
• No stray null tokens leak through.

Full test suite: npx ic-mops test — 10/10 files pass (9 previously-green + the new one).

🤖 Generated with Claude Code

[ggreif]

Extended the patch to CBOR and UrlEncoded as well — both had the same unconditional record/map walk that emits null-valued optional fields (CBOR as majorType7(#_null), UrlEncoded as literal key=null). Same treatment under the skip_null_fields flag.

• src/CBOR/lib.mo: skip record entries whose value encodes to #majorType7(#_null).
• src/UrlEncoded/ToText.mo: new fromCandidWith(candid, skip_null_fields) analogous to JSON's, threads into toKeyValuePairs, elides key=null at the #Null leaf.

Tests added in tests/CBOR.Test.mo and tests/UrlEncoded.Test.mo. Full suite: 10/10 files pass.

Commit: a63d60b.