From 863c5526dedd1ff1062616e9792519b7c985f90d Mon Sep 17 00:00:00 2001 From: Kenny Pflug Date: Sun, 2 Aug 2026 11:17:10 +0200 Subject: [PATCH 1/7] docs(metadata): plan allocation-free canonical formatting MetadataValue.TryFormatCanonical honors its span contract only for Double and Single; the remaining ten kinds detour through ToCanonicalString and allocate a throwaway string. The same missing primitives cost an allocation per candidate on the TryGetXxx read path. Key decisions recorded in the plan: - Port the formatters from dotnet/runtime v6.0.36 rather than writing them from scratch. That line predates the intrinsics rewrites, so it is scalar and compiles on netstandard2.0, and XsdDuration is the normative source of the TimeSpan encoding the validators compare against. - Cover UTF-8 as well as UTF-16 via the existing TCodeUnit renderer, closing the serializer integration deferred by #61. - Ship exactly one implementation for both assets. Whether net10.0 should call the framework span formatters is deferred to a follow-up issue; this plan only produces the benchmark evidence. - Do not route String, Char, or Uri through the UTF-8 API in the serializers: Utf8JsonWriter does not agree bytewise with pre-transcoded replacement bytes under UnsafeRelaxedJsonEscaping. - Normalize DateTimeKind.Local to UTC in the formatter, matching FromDateTime, which keeps the 28-character bound. - Preserve the allocation-free ToCanonicalString paths for Null, Boolean, String, and Uri. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AZTXiVaNguCvmjrTjwWLfQ --- ...0-try-format-canonical-zero-allocations.md | 236 ++++++++++++++++++ 1 file changed, 236 insertions(+) create mode 100644 ai-plans/0070-try-format-canonical-zero-allocations.md diff --git a/ai-plans/0070-try-format-canonical-zero-allocations.md b/ai-plans/0070-try-format-canonical-zero-allocations.md new file mode 100644 index 0000000..4a5b1e6 --- /dev/null +++ b/ai-plans/0070-try-format-canonical-zero-allocations.md @@ -0,0 +1,236 @@ +# Allocation-Free Canonical Formatting for MetadataValue + +## Rationale + +`MetadataValue.TryFormatCanonical` promises a span-based formatting API, but only `Double` and `Single` honor it. Every other kind falls through to `ToCanonicalString()` and copies the resulting string into the destination. For `Null`, `Boolean`, `String`, and `Uri` that is already free — they return a literal or the stored reference — but for the remaining ten kinds the caller pays a string allocation for exactly the values it tried to format without one. `JsonCloudEventsExtensions.WriteStringAttribute` is the visible victim: it stack-allocates a buffer, calls `TryFormatCanonical`, and still allocates a throwaway string for every `Guid`, `DateTime`, or `Int64` extension attribute it writes. The same missing primitives cost allocations on the read path, where every `TryGetXxx` overload that accepts canonical text validates by formatting the parsed value back to a `string` and comparing it. + +The library's serializers ultimately write UTF-8. `CanonicalFloatingPointFormatter` already renders both encodings from one implementation by generalizing its renderer over the output code unit, and #61 deferred the serializer integration that makes the saving observable — `MetadataExtensions.WriteNumberValue` still allocates a string per `Double` and lets `System.Text.Json` transcode it back to ASCII. Because the remaining canonical encodings are ASCII too, extending that same generic renderer to them yields the UTF-8 path at almost no additional cost, and skipping it now would mean a second pass over these files later. + +Introduce canonical span formatters for the remaining primitive kinds in both encodings, add the matching UTF-16 and UTF-8 APIs to `MetadataValue`, derive `ToCanonicalString` and the round-trip validators from those formatters so a single implementation defines each encoding, and adopt them in the JSON and CloudEvents writers. + +## Acceptance Criteria + +- [ ] `MetadataValue` exposes `TryFormatCanonical` and a new `TryFormatCanonicalUtf8`, and neither allocates for any primitive kind — including `String`, `Char`, and `Uri`, whose text is copied or transcoded into the caller's destination. +- [ ] One implementation serves both package assets and both encodings: no target-specific formatter, no routing through the other encoding, and no intermediate output buffer — each method writes directly into the caller's destination in the requested encoding. The shared expected-output corpus passes unchanged against both assets. +- [ ] A public `CanonicalTextFormatter` with XML documentation exposes the new canonical formatting primitives in both encodings, plus per-type maximum-length constants that bound both encodings, on both package assets. +- [ ] `MetadataValue` exposes a documented public constant bounding the canonical length of every bounded primitive kind in either encoding, and `JsonCloudEventsExtensions` sizes its stack buffer with that constant instead of a private magic number. +- [ ] For every value of every primitive kind, `TryFormatCanonical` produces exactly the text that `ToCanonicalString` produced before this change, and `charsWritten` equals that text's length. `TryFormatCanonicalUtf8` produces the replacement-fallback UTF-8 encoding of that same text — lossy only for malformed UTF-16, which has no valid UTF-8 encoding — with `bytesWritten` equal to `charsWritten` for every kind whose canonical text is ASCII. +- [ ] The existing failure contracts are unchanged and hold in both encodings: a destination that is too small returns `false`, writes zero to the count, and leaves the destination unmodified; `Array` and `Object` still throw `InvalidOperationException`; and a corrupt `DateTime`, `DateOnly`, or `TimeOnly` payload still throws `InvalidOperationException`. +- [ ] `CanonicalTextFormatter.TryFormat(DateTime, …)` normalizes `DateTimeKind.Local` to UTC, matching `MetadataValue.FromDateTime`, so every `DateTime` it accepts fits `MaximumDateTimeLength`. `Utc` and `Unspecified` values are unaffected, and the behavior is covered by a test against the formatter itself. +- [ ] The `TryGetXxx` round-trip validators compare against span-formatted text and no longer allocate a string per candidate value. +- [ ] `ToCanonicalString` still allocates nothing for the four kinds that are allocation-free today — `Null`, `Boolean`, `String`, and `Uri` — and its output is unchanged for every kind. +- [ ] The JSON metadata writer emits string-shaped and `Double`/`Single` number-shaped metadata values without materializing canonical text, and the bytes it produces for every metadata value are unchanged under both the default and the `UnsafeRelaxedJsonEscaping` encoder, including for text containing non-ASCII characters and unpaired surrogates. +- [ ] Automated tests cover, per kind and per encoding, the canonical text, the written count, the exactly-sufficient and one-short destination, the boundary values of each kind, the invalid-payload throws, and the absence of allocations. The metadata test project passes against both the `net10.0` and the `netstandard2.0` library asset. +- [ ] A microbenchmark compares the new `Guid` and `DateTime` formatters with the framework `TryFormat` APIs on `net10.0`, and its result is recorded in the pull request as evidence for the deferred follow-up decision. No target-specific implementation is introduced in this issue regardless of the result. +- [ ] The CloudEvents and HTTP write benchmark fixtures carry metadata of the kinds this issue affects, and before/after allocation figures for both are recorded in the pull request. +- [ ] `THIRD-PARTY-NOTICES.md` and the folder README record the newly adapted upstream files and the adaptations made to them, following the existing provenance pattern, and every file containing adapted runtime code retains the .NET Foundation MIT header. +- [ ] The `netstandard2.0` decimal path documents its runtime assumption and enforces it at type initialization, so a runtime that violates it fails loudly instead of producing wrong text. +- [ ] The package release notes mention the new formatting APIs and the removed allocations. +- [ ] Both target frameworks build in Release with warnings as errors, package validation succeeds, the Native AOT sample publishes successfully, and test coverage remains above 95%. + +## Technical Details + +### Why the framework APIs are not enough + +`netstandard2.0` has no span-formatting API for any of the affected types, and the `Polyfill` package's `TryFormat` extensions are not a substitute: they call `ToString` and copy the result, which is what this issue removes. Meeting the criterion on both assets requires the library to own these formatters regardless of what the `net10.0` asset does, and UTF-8 widens the gap further — `IUtf8SpanFormattable` does not exist on `netstandard2.0` at all. + +This issue therefore ships exactly one implementation, serving both assets and both encodings, with no `#if NET10_0_OR_GREATER` split onto framework APIs anywhere. A single path removes the risk of the two assets diverging in wire format — the failure #58 fixed for floating-point text — keeps one test surface, and costs one implementation per type instead of two per encoding. + +Calling into the framework span formatters on `net10.0` remains an open question, not a rejected one: `Guid.TryFormat` in particular has a dedicated vectorized `D` path upstream. That question is deliberately deferred to a follow-up issue, in the same way #61 deferred its serializer integration. This plan produces the evidence for that decision — the microbenchmark below — and stops there. Do not introduce a target-specific branch here, even if the benchmark favors one. + +The benchmark also tells the follow-up where to look. `Guid` is the plausible candidate. The date kinds likely are not: their canonical form is a custom format string, so `DateTime.TryFormat` routes into the general custom-format interpreter rather than the fixed-shape `TryFormatO` fast path this plan ports. + +### Porting from the BCL + +Do not write these formatters from scratch. `Numbers/` already establishes the pattern — adapted `dotnet/runtime` code under the .NET Foundation MIT header, with provenance in `THIRD-PARTY-NOTICES.md` and the folder README — and every encoding this issue needs has a portable upstream implementation. Porting also retires a correctness risk: these encodings are pinned by the `TryGetXxx` validators, so a hand-derived rule that is subtly wrong breaks round-tripping of data already on the wire. + +Stay on the tag the repository already cites, `v6.0.36` (commit `f1dd57165bfd91875761329ac3a8b17f6606ad18`). That is not only for consistency: `release/6.0` predates the `System.Runtime.Intrinsics` rewrites of `Guid.TryFormat` and the number formatters, so its implementations are scalar and compile on `netstandard2.0`, where intrinsics do not exist. Newer lines would have to be de-vectorized by hand. + +| Kind | Upstream source | Notes | +| --- | --- | --- | +| `Int64`, `UInt64` | `Number.Formatting.cs`: `TryUInt64ToDecStr`, `UInt32ToDecChars`, `Int64DivMod1E9`; `FormattingHelpers.CountDigits` | Scalar integer math over a `fixed` destination. Direct port. | +| `Decimal` | `Number.Formatting.cs`: `DecimalToNumber`; `Decimal.DecCalc.cs`: `DecDivMod1E9` | `DecDivMod1E9` is twelve lines of integer arithmetic. `DecimalToNumber` emits digits and scale into a `NumberBuffer` — the type this repository already ported — so decimal reuses existing infrastructure and only needs a renderer that honors the scale. | +| `DateTime`, `DateTimeOffset`, `DateOnly`, `TimeOnly` | `DateTimeFormat.cs`: `TryFormatO`, `WriteTwoDecimalDigits`, `WriteFourDecimalDigits`, `WriteDigits` | Port `TryFormatO`, **not** `FormatCustomized`. See below. | +| `TimeSpan` | `System.Private.Xml`, `XsdDuration.cs`: the `TimeSpan` constructor and `ToString(DurationType)` | This is the normative source of `XmlConvert.ToString(TimeSpan)`, which the validators compare against, so porting it settles the encoding by construction rather than by inference. | +| `Guid` | `Guid.cs`: the `D` branch of `TryFormat`, `HexsToChars`, `HexConverter.ToCharLower` | Confirms the field-reinterpretation approach: upstream formats the numeric fields (`_a >> 24`, …), which is endianness-independent. | + +Two adaptations are unavoidable and one target is a trap: + +- **`FormatCustomized` is the wrong target for the date kinds.** It is a general custom-format interpreter driven by `DateTimeFormatInfo` and `Calendar`, carrying Hebrew and Japanese calendar special cases and `StringBuilderCache`; porting it would be far more work than the fixed template it would produce. `TryFormatO` is roughly seventy-five lines, span-based, culture-free, and already writes `yyyy-MM-ddTHH:mm:ss` followed by a fraction and an offset or `Z`. Adapt it in three places: it writes exactly seven fractional digits where the canonical form omits the fraction entirely when the sub-second ticks are zero and otherwise trims trailing zeros; it reads the components through the runtime-internal `GetDate`/`GetTimePrecise`, so either port those two helpers as well or use the public `Year`/`Month`/`Day`/… properties and accept that each recomputes from the tick count; and its `DateTimeKind.Local` branch, which queries `TimeZoneInfo.Local` and appends the six-character offset, is dropped for `DateTime` because the kind contract normalizes `Local` beforehand. Only the `DateTimeOffset` overload keeps an offset path. +- **Everything upstream renders to `char`.** The ported renderers must write through this repository's `TCodeUnit` conversion helper instead, so both encodings come from one body. This is the same adaptation `CanonicalFloatingPointFormatter` already documents. +- **`decimal.GetBits` is a `netstandard2.0` trap, and the way around it is narrower than it looks.** The `Span` overload is .NET Core 3.0 and later; on `netstandard2.0` only `int[] GetBits(decimal)` exists, and it allocates a four-element array per call — which would defeat the entire issue. Upstream sidesteps it by reading the value through `Unsafe.As`, but that is a weaker guarantee than it appears. `decimal.GetBits` documents the *logical* representation — low, middle, high, flags — not the private field order, and upstream `DecCalc` carries an explicit `#if BIGENDIAN` layout, so endianness is part of the assumption rather than incidental to it. + + Split the value extraction by target and keep the formatter itself single: + + - `net10.0` calls `decimal.GetBits(decimal, Span)`. It is contractual, allocation-free, and endianness-independent, so this target carries no layout assumption at all. + - `netstandard2.0` uses the reinterpret, with the assumption stated in the XML documentation: a little-endian runtime with the historical `flags`, `hi`, `lo`, `mid` field order. + + This does not reopen the single-implementation decision. What differs is how the four integers are obtained; the digit generation and the renderer are one body, and both extractions are pinned to the same contractual quadruple by `GetBits` semantics. + + Verify the `netstandard2.0` assumption at type initialization rather than trusting it: compare the reinterpreted fields against `decimal.GetBits` for a probe value once, and throw `PlatformNotSupportedException` on mismatch. The allocating overload is acceptable there because it runs once per process, and the check converts silent wrong output on an unforeseen runtime into an immediate, diagnosable failure. Note that a mutant in that static initializer will be reported as survived regardless of coverage, per the blind spot in `tests/AGENTS.md`. + + A unit test comparing the reinterpret against `decimal.GetBits` is still worth having, but be precise about what it proves: the test suite runs both package assets on the same .NET 10 host, so it validates the compilation target, never the `netstandard2.0` asset's behavior on .NET Framework or Mono. Only the runtime guard covers that case. + +### Formatting rules to verify + +The ports above define the encodings; these are the cases to assert explicitly, because they are where an adaptation slip stays invisible until data fails to round-trip: + +- **Decimal.** The scale is significant — `19.50m` renders as `19.50`, not `19.5`. A negative zero (`decimal.Negate(0m)`, sign bit set, all digits zero) renders **without** a sign, matching the framework. +- **TimeSpan.** `PT0S` for zero, a leading `-` before the `P`, the days component omitted when zero, the `T` present only when a time component follows, and fractional seconds trimmed of trailing zeros (`P2DT3H4M5.06S`, `PT0.0000001S`). Upstream derives the magnitude with `unchecked((ulong)-ticks)` precisely so that `TimeSpan.MinValue`, whose magnitude is not representable as a positive `long`, does not overflow; keep that. +- **DateTime.** `Utc` values end in `Z` and `Unspecified` values carry no designator, matching `yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK`. `Local` values are normalized as described below. `DateTimeOffset` uses the same shape with `+hh:mm`/`-hh:mm` and never `Z`. +- **Guid.** Lowercase `D` format. + +### The DateTime kind contract + +`TryFormat(DateTime, …)` normalizes `DateTimeKind.Local` to UTC and renders the result with the `Z` designator, matching `MetadataValue.FromDateTime`. `Utc` and `Unspecified` values are rendered as they are. Document this on the method: for `Local` input the output depends on the machine's local time zone, which is exactly why the library normalizes at construction rather than at the boundary. + +Without this rule the public formatter would contradict its own bound. `K` renders a `Local` value with a `±hh:mm` offset — 33 characters, five past `MaximumDateTimeLength`, and the upstream `TryFormatO` reserves those six characters for precisely that case. Normalizing first keeps the bound at 28 and keeps the standalone formatter consistent with the only kinds a `MetadataValue` can hold. + +Two consequences for the implementation: + +- **Normalize conditionally.** Call `ToUniversalTime()` only when `Kind == DateTimeKind.Local`. `DateTime.ToUniversalTime` treats an `Unspecified` value as local and shifts it, so an unconditional call would silently move every `Unspecified` value by the host's offset and destroy the no-designator form that this library deliberately preserves. `FromDateTime` already guards the same way. +- **Normalization cannot throw.** `ToUniversalTime` saturates at `DateTime.MinValue` and `DateTime.MaxValue` instead of overflowing, so the formatter gains no failure mode and the conversion needs no guard. + +Because `FromDateTime` normalizes on the way in and `TryReadStoredDateTime` rejects a stored `Local` payload as corrupt, no `MetadataValue` can drive this branch. Cover it with a direct test against `CanonicalTextFormatter` — the case `tests/AGENTS.md` reserves solitary tests for — asserting the normalized text, the written count, and that the output still fits `MaximumDateTimeLength`. + +### Public API + +```csharp +namespace Light.PortableResults.Text; + +public static class CanonicalTextFormatter +{ + public const int MaximumInt64Length = 20; + public const int MaximumUInt64Length = 20; + public const int MaximumDecimalLength = 31; + public const int MaximumCharLength = 3; + + public const int MaximumDayNumber = 3_652_058; // DateOnly.MaxValue.DayNumber + public const long MaximumTimeOfDayTicks = 863_999_999_999; // TimeSpan.TicksPerDay - 1 + public const int MaximumDateTimeLength = 28; + public const int MaximumDateTimeOffsetLength = 33; + public const int MaximumDateLength = 10; + public const int MaximumTimeLength = 16; + public const int MaximumTimeSpanLength = 27; + public const int MaximumGuidLength = 36; + + public static bool TryFormat(char value, Span destination, out int charsWritten); + public static bool TryFormat(long value, Span destination, out int charsWritten); + public static bool TryFormat(ulong value, Span destination, out int charsWritten); + public static bool TryFormat(decimal value, Span destination, out int charsWritten); + public static bool TryFormat(DateTime value, Span destination, out int charsWritten); + public static bool TryFormat(DateTimeOffset value, Span destination, out int charsWritten); + public static bool TryFormat(TimeSpan value, Span destination, out int charsWritten); + public static bool TryFormat(Guid value, Span destination, out int charsWritten); + public static bool TryFormatDate(int dayNumber, Span destination, out int charsWritten); + public static bool TryFormatTime(long ticks, Span destination, out int charsWritten); + + // One TryFormatUtf8 counterpart per overload above, taking Span and reporting bytesWritten. + + public static bool TryFormatUtf8( + ReadOnlySpan text, + Span destination, + out int bytesWritten + ); +} +``` + +The signatures are exact. + +The `char` overload is required for correctness, not convenience, and must not be dropped as redundant with the `long` one. `char` converts implicitly to `long`, so without an exact overload `TryFormat('x', destination, out _)` compiles and writes `120` — silently, in both encodings. `char` is the only affected type: `int`, `short`, `byte`, and `uint` also bind to the `long` overload, but their canonical text *is* their integer text, so those bindings are correct. Renaming the integer methods instead does not fix it; the call then binds to `TryFormat(decimal)` and still writes `120`. Only an exact match resolves ahead of the implicit conversions. + +**`false` means one thing: the destination was too small.** Invalid input throws. `TryFormatDate` and `TryFormatTime` accept a `dayNumber` in `[0, MaximumDayNumber]` and `ticks` in `[0, MaximumTimeOfDayTicks]` respectively, and throw `ArgumentOutOfRangeException` outside those ranges. Overloading `false` to mean both "invalid" and "does not fit" would leave callers unable to distinguish them and would force `MetadataValue` to re-derive the ranges to know which exception to raise. This also matches the sibling class: `CanonicalFloatingPointFormatter.TryFormat` throws `ArgumentException` for a non-finite value and reserves `false` for capacity. + +`MetadataValue` keeps its own guard rather than relying on that throw, because its contract is `InvalidOperationException` with the existing message, not `ArgumentOutOfRangeException`. The published bounds are what makes that guard safe: it checks against the same constants the formatter enforces, so no magic number is duplicated and the two cannot drift. The formatter's own `ArgumentOutOfRangeException` is consequently unreachable through `MetadataValue` and needs a direct test. + +`MaximumCharLength` is 3 rather than 1 because, by the same both-encodings convention as the other constants, it must bound the UTF-8 output: a BMP character occupies up to three UTF-8 bytes, and an unpaired surrogate encodes to the three-byte replacement character. The UTF-16 overload never writes more than one. + +`MaximumDateTimeLength` is 28 rather than 33 because `TryFormat(DateTime, …)` normalizes `Local` before rendering, so no input reaches the offset form; see the kind contract above. `TryFormatDate` and `TryFormatTime` take the stored payload rather than `DateOnly` and `TimeOnly`, because `MetadataKind.DateOnly` and `MetadataKind.TimeOnly` are formatted on both assets while the BCL types exist only on `net10.0`. The trailing `TryFormatUtf8(ReadOnlySpan, …)` is the transcoding entry point that the `String`, `Char`, and `Uri` kinds need; it has no UTF-16 counterpart because copying chars is `TryCopyTo`. + +`CanonicalFloatingPointFormatter` stays where it is and keeps its own constants and API; moving it would break callers for no gain. + +`MetadataValue` gains one constant and one method: + +```csharp +public const int MaximumPrimitiveCanonicalLength = 36; + +public bool TryFormatCanonicalUtf8(Span destination, out int bytesWritten); +``` + +The constant's value is the `Guid` bound, the largest of the bounded kinds. Document that it bounds both encodings and that it excludes `String` and `Uri`, whose canonical text is the caller's own unbounded text — those are the only kinds for which a destination of this size can still return `false`, which is why `WriteStringAttribute` keeps its materializing fallback. `Char` is bounded in both encodings and needs no exclusion: every UTF-16 code unit encodes to at most three UTF-8 bytes. + +### The shared renderer + +Follow the `TCodeUnit` pattern that `CanonicalFloatingPointFormatter.TryRender` established: a private generic core per type, `where TCodeUnit : unmanaged`, writing ASCII through the existing `typeof(TCodeUnit) == typeof(byte)` conversion helper that the JIT folds away. Every encoding below is pure ASCII, so one renderer produces both outputs and the two public overloads are thin forwarders. Compute the required length before the first write so the all-or-nothing contract holds in both encodings. + +Lift the conversion helper to a shared internal home rather than duplicating it, and keep it out of the public surface. The `unmanaged` constraint permits instantiations other than `char` and `byte`; as in #61, do not add a guard for the impossible case, because an unreachable `throw` is a coverage hole. + +### Transcoding the text-bearing kinds + +`String`, `Char`, and `Uri` are the only kinds whose canonical text is not ASCII, so they are the only ones the UTF-8 path cannot render through the shared renderer. Transcode them with replacement fallback: invalid UTF-16 — an unpaired surrogate, which `FromChar` and `FromString` both accept — becomes U+FFFD rather than a failure, because returning `false` would conflate invalid text with an undersized destination. + +That is a lossy encoding, and it must be described as one. Malformed UTF-16 has no corresponding valid UTF-8, so the UTF-8 output for these three kinds is the *replacement-fallback* encoding of the canonical text, not the encoding of that text. `bytesWritten` therefore does not equal `charsWritten` for them, unlike every other kind, and the round trip through UTF-8 is not identity. Scope both assertions accordingly. + +**These three kinds must not be routed through the UTF-8 API by the serializers.** `Utf8JsonWriter` does not agree bytewise with pre-transcoded replacement bytes: when it transcodes malformed UTF-16 itself it emits the *escaped* `�`, whereas valid UTF-8 replacement bytes handed to it are subject only to the encoder's normal escaping rules. Measured against the pinned System.Text.Json 10.0.10, the two routes agree under the default encoder — which escapes all non-ASCII — and diverge under `UnsafeRelaxedJsonEscaping`: + +```text +input "a\uD800b", relaxed encoder + via ReadOnlySpan 22-61-5C-75-46-46-46-44-62-22 "a�b" + via transcoded UTF-8 22-61-EF-BF-BD-62-22 "ab" +``` + +The results are equivalent after parsing but not byte-identical, which violates the unchanged-output criterion. Note that an already-valid U+FFFD in the input converges under both encoders; only malformed input diverges. See the adoption rules below for what each kind uses instead. + +On `net10.0` the transcoding is `Utf8.FromUtf16` with `replaceInvalidSequences: true`. On `netstandard2.0` neither that API nor the `Span`-based `Encoding.GetBytes` overload exists; use the pointer overload under `fixed`, whose default replacement fallback produces the same U+FFFD bytes. Both paths need the byte count before the first write to preserve the all-or-nothing contract, which costs a second pass over the text — acceptable, and unavoidable without partial-write semantics. + +### Single source of truth + +Invert the current relationship: the span methods become primary and `ToCanonicalString` formats into a stack buffer and calls `ToString` on the written slice. + +The inversion must not regress `ToCanonicalString`, which is allocation-free today for exactly four kinds. Keep every one of them on a fast path that returns an existing reference rather than building a string: + +- `Null` and `Boolean` return string literals (`"null"`, `"true"`, `"false"`). +- `String` and `Uri` return the stored instance. + +Routing the first two through a stack buffer and `ToString()` would allocate on every call, on paths that are free now — `HttpHeaderValueFormatter` formats every Boolean header value this way, and `ToString()` and the validation message formatter both go through `ToCanonicalString`. + +Single-source the literals through `private const string` fields so the two APIs cannot drift: `ToCanonicalString` returns the constant, and the span methods write it through a small ASCII writer built on the shared `TCodeUnit` conversion helper, which serves both encodings from the same constant. Do not introduce a separate `u8` literal for the UTF-8 path; a second literal is a second source of truth. + +Apply the same inversion to the private `FormatXxx` helpers behind the `TryGetXxx` validators: format into a stack buffer and compare with `MemoryExtensions.SequenceEqual` instead of allocating a string per comparison. `TryGetInt64` compares against `value.ToString(CultureInfo.InvariantCulture)` inline and needs the same treatment. + +### Serializer adoption + +The rule is that the writer performs any UTF-16 transcoding itself; the serializers only hand it UTF-8 for kinds whose canonical text is ASCII, where the two encodings cannot disagree. + +- `MetadataExtensions.WriteNumberValue` writes the `Double` and `Single` canonical text as UTF-8 through `WriteRawValue(ReadOnlySpan, skipInputValidation: true)`, which closes the integration deferred by #61. These are ASCII, so this is safe. +- `MetadataExtensions.WriteMetadataValue` writes the ASCII-bounded string-shaped kinds — the date kinds, `TimeSpan`, `Guid`, `UInt64` — from a stack buffer, in either encoding. +- `String` and `Uri` keep their current path and are passed to the writer as the stored string. They already own one, so nothing is allocated, and the writer's own transcoding is what produces today's bytes. +- `Char` is written through the UTF-16 span overload, not the UTF-8 one. It is a single code unit that may be an unpaired surrogate, so the writer must be the one to transcode it. +- `JsonCloudEventsExtensions.WriteStringAttribute` keeps its shape and switches to the constant. It stays on the UTF-16 path, which is what it already uses; CloudEvents extension attributes additionally reject unpaired surrogates upstream of it, so the divergence above cannot arise there. +- `HttpHeaderValueFormatter` is unchanged: it returns `StringValues` and needs strings regardless. + +### Benchmarks + +The existing write benchmarks cannot demonstrate anything as they stand: both fixtures build their metadata exclusively from `MetadataValue.FromString`, and the `String` kind returns the stored reference before and after this change. Measured unmodified, the before/after delta is identically zero. Extend the fixtures first, then capture the figures. + +- **CloudEvents.** Add `Guid`, `DateTime`, `Double`, and an `Int64` outside the inclusive 32-bit signed range, annotated `SerializeInCloudEventsExtensionAttributes`. The range qualifier is load-bearing: an in-range `Int64` takes the `Integer` attribute encoding and is written with `WriteNumberValue`, never touching canonical text, so only the out-of-range value routes to `WriteStringAttribute` — the method this issue fixes. +- **HTTP.** Add the same kinds annotated `SerializeInHttpResponseBody`. Do not try to show the improvement through `SerializeInHttpHeader`: `HttpHeaderValueFormatter` returns `StringValues` and needs a materialized string regardless, so the header path keeps its allocation by design and would flatline. + +### Testing + +Test the canonical text through `MetadataValue` (sociable), and reach `CanonicalTextFormatter` directly only for inputs no `MetadataValue` can carry. Derive expectations from the framework `ToString` calls the current implementation uses, so the tests state that the encoding is unchanged rather than restating the new implementation; assert the UTF-8 output against the ASCII bytes of those same expectations, following #61's precedent. Cover per kind: minimum and maximum values, zero and negative zero where representable, the scale-bearing decimal cases, `TimeSpan.MinValue`/`MaxValue`/`Zero` and the component-omission combinations, sub-second-tick presence and absence for the date kinds, and both `DateTimeKind` values `FromDateTime` can store. Reach `CanonicalTextFormatter` directly for the `Local` kind, which no `MetadataValue` can carry, and assert there that an `Unspecified` value is *not* shifted — the regression an unconditional `ToUniversalTime` would introduce. For the text-bearing kinds add non-ASCII text, a surrogate pair, and an unpaired surrogate in both encodings, asserting the replacement-fallback result rather than a round trip. Pair that with a writer-level test asserting the emitted JSON bytes are unchanged for those inputs under **both** the default and the `UnsafeRelaxedJsonEscaping` encoder — the default encoder escapes all non-ASCII and hides the divergence that motivated the adoption rules, so a single-encoder test proves nothing here. + +Pin the `char` overload resolution with a test that calls `CanonicalTextFormatter.TryFormat` with a `char` literal and asserts the character, not its code point — the failure this guards against is a binding change, so the test must pass a `char` typed argument rather than a variable already narrowed elsewhere. + +Assert the absence of allocations with `GC.GetAllocatedBytesForCurrentThread` around a warmed-up loop over one value of each affected kind in each encoding, following the precedent in the floating-point formatter tests. Cover `ToCanonicalString` for `Null`, `Boolean`, `String`, and `Uri` in the same way: those four are allocation-free today, so the assertion guards a regression the inversion could otherwise introduce silently, since the returned text would still be correct. + +Extend the same assertions to the `TryGetXxx` validator paths, which the formatting-API tests do not reach. Those paths run only when the value is a `String` kind being parsed — `TryGetGuid` on a `Guid`-kind value returns early without formatting anything — so the test must build `String`-kind values holding canonical text and call `TryGetGuid`, `TryGetDateTime`, `TryGetInt64`, and the rest against them. That is where the per-candidate string allocation lives today. + +Every new method carries an `out` parameter, which puts it in Stryker's Safe Mode blind spot documented in `tests/AGENTS.md`: the mutants fail to compile and the enclosing methods receive no mutation coverage at all. Mutation score therefore carries no information about this change. Argue adequacy by hand in the pull request, naming the behavior each formatter promises and the test that constrains it. + +### Scope + +Deferred to follow-up issues: + +- Whether the `net10.0` asset should call the framework span formatters for any type. This plan measures it and records the numbers; it does not act on them. +- `HttpHeaderValueFormatter` stays as it is: it returns `StringValues` and needs strings regardless. From 1aaab36365a6844e6da530dd2818b8ab7d706021 Mon Sep 17 00:00:00 2001 From: Kenny Pflug Date: Sun, 2 Aug 2026 11:25:43 +0200 Subject: [PATCH 2/7] docs(metadata): validate the decimal layout without allocating The netstandard2.0 layout guard called decimal.GetBits(decimal), which allocates a four-element array on first use. That contradicted the unqualified no-allocation criterion, and the warmed-up allocation test could not have caught it: warm-up triggers type initialization before measurement starts. Seed the probe from decimal(int, int, int, bool, byte) instead. The constructor defines the logical representation just as GetBits does, so the check is no weaker, and it touches nothing on the heap. Also isolate the guard in a decimal-specific helper so a layout failure cannot disable integer, date, TimeSpan, and Guid formatting, and store its outcome rather than throwing from the initializer, so callers see PlatformNotSupportedException instead of TypeInitializationException wrapping it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AZTXiVaNguCvmjrTjwWLfQ --- ...0070-try-format-canonical-zero-allocations.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/ai-plans/0070-try-format-canonical-zero-allocations.md b/ai-plans/0070-try-format-canonical-zero-allocations.md index 4a5b1e6..a5a8378 100644 --- a/ai-plans/0070-try-format-canonical-zero-allocations.md +++ b/ai-plans/0070-try-format-canonical-zero-allocations.md @@ -24,7 +24,7 @@ Introduce canonical span formatters for the remaining primitive kinds in both en - [ ] A microbenchmark compares the new `Guid` and `DateTime` formatters with the framework `TryFormat` APIs on `net10.0`, and its result is recorded in the pull request as evidence for the deferred follow-up decision. No target-specific implementation is introduced in this issue regardless of the result. - [ ] The CloudEvents and HTTP write benchmark fixtures carry metadata of the kinds this issue affects, and before/after allocation figures for both are recorded in the pull request. - [ ] `THIRD-PARTY-NOTICES.md` and the folder README record the newly adapted upstream files and the adaptations made to them, following the existing provenance pattern, and every file containing adapted runtime code retains the .NET Foundation MIT header. -- [ ] The `netstandard2.0` decimal path documents its runtime assumption and enforces it at type initialization, so a runtime that violates it fails loudly instead of producing wrong text. +- [ ] The `netstandard2.0` decimal path documents its runtime assumption and enforces it without allocating, surfacing `PlatformNotSupportedException` from the decimal formatting path itself, so a runtime that violates it fails loudly instead of producing wrong text and no other kind's formatting is affected. - [ ] The package release notes mention the new formatting APIs and the removed allocations. - [ ] Both target frameworks build in Release with warnings as errors, package validation succeeds, the Native AOT sample publishes successfully, and test coverage remains above 95%. @@ -67,7 +67,17 @@ Two adaptations are unavoidable and one target is a trap: This does not reopen the single-implementation decision. What differs is how the four integers are obtained; the digit generation and the renderer are one body, and both extractions are pinned to the same contractual quadruple by `GetBits` semantics. - Verify the `netstandard2.0` assumption at type initialization rather than trusting it: compare the reinterpreted fields against `decimal.GetBits` for a probe value once, and throw `PlatformNotSupportedException` on mismatch. The allocating overload is acceptable there because it runs once per process, and the check converts silent wrong output on an unforeseen runtime into an immediate, diagnosable failure. Note that a mutant in that static initializer will be reported as survived regardless of coverage, per the blind spot in `tests/AGENTS.md`. + Verify the `netstandard2.0` assumption once rather than trusting it, and verify it **without allocating**. Do not reach for `decimal.GetBits(decimal)` in the guard: it allocates a four-element array on first use, which breaches the unqualified no-allocation criterion, and the warmed-up allocation test cannot catch it, because the warm-up iteration triggers type initialization before measurement starts. Seed the probe from the constructor instead: + + ```csharp + var probe = new decimal(lo: 0x11111111, mid: 0x22222222, hi: 0x33333333, isNegative: true, scale: 5); + ``` + + Reinterpret that value and compare the fields against the constructor arguments and the expected flags word — `0x80050000` for the parameters above, being the sign bit and the scale in bits 16 to 23. `decimal(int, int, int, bool, byte)` defines the logical representation exactly as `GetBits` does, so this is no weaker a check, and it touches nothing on the heap: a struct construction, a reinterpret, and four integer comparisons. + + Isolate the guard so its failure is proportionate. Put it in a decimal-specific private helper rather than in `CanonicalTextFormatter`'s own initializer; a throwing initializer on the outer type would disable integer, date, `TimeSpan`, and `Guid` formatting too, none of which depend on the layout. Store the outcome in a `static readonly bool` and throw `PlatformNotSupportedException` from the decimal formatting path, rather than throwing from the initializer — an initializer that throws surfaces to callers as `TypeInitializationException` wrapping the real cause, which contradicts the exception documented on the method. The flag read folds away once the type is initialized, so it costs nothing on the formatting path. + + Note that a mutant in that initializer will be reported as survived regardless of coverage, per the blind spot in `tests/AGENTS.md`. A unit test comparing the reinterpret against `decimal.GetBits` is still worth having, but be precise about what it proves: the test suite runs both package assets on the same .NET 10 host, so it validates the compilation target, never the `netstandard2.0` asset's behavior on .NET Framework or Mono. Only the runtime guard covers that case. @@ -224,6 +234,8 @@ Pin the `char` overload resolution with a test that calls `CanonicalTextFormatte Assert the absence of allocations with `GC.GetAllocatedBytesForCurrentThread` around a warmed-up loop over one value of each affected kind in each encoding, following the precedent in the floating-point formatter tests. Cover `ToCanonicalString` for `Null`, `Boolean`, `String`, and `Uri` in the same way: those four are allocation-free today, so the assertion guards a regression the inversion could otherwise introduce silently, since the returned text would still be correct. +Be aware of what the warm-up hides: it triggers type initialization and any lazy setup before measurement begins, so a one-time allocation on first use is invisible to these tests. One-time initialization must therefore be allocation-free by construction, not merely amortized — the assertion cannot enforce it. + Extend the same assertions to the `TryGetXxx` validator paths, which the formatting-API tests do not reach. Those paths run only when the value is a `String` kind being parsed — `TryGetGuid` on a `Guid`-kind value returns early without formatting anything — so the test must build `String`-kind values holding canonical text and call `TryGetGuid`, `TryGetDateTime`, `TryGetInt64`, and the rest against them. That is where the per-candidate string allocation lives today. Every new method carries an `out` parameter, which puts it in Stryker's Safe Mode blind spot documented in `tests/AGENTS.md`: the mutants fail to compile and the enclosing methods receive no mutation coverage at all. Mutation score therefore carries no information about this change. Argue adequacy by hand in the pull request, naming the behavior each formatter promises and the test that constrains it. From f9babaa7f2aa51409e1d79e0497dccc15b382301 Mon Sep 17 00:00:00 2001 From: Kenny Pflug Date: Sun, 2 Aug 2026 11:27:58 +0200 Subject: [PATCH 3/7] docs(metadata): narrow four overreaching statements None of these change the design; they stop the plan from asserting more than is true. - The read-path rationale claimed every text-accepting TryGetXxx reformats to validate. Nine do; TryGetDecimal, TryGetChar, and TryGetUri accept text without reformatting. - "The remaining canonical encodings are ASCII" excluded nothing, but Char is included in that set and may be non-ASCII. Scope it to the non-text encodings and name the exception. - bytesWritten is not guaranteed to equal charsWritten for the text-bearing kinds, rather than never equal to it: ASCII String, Char, and Uri values still have equal counts, so the assertion has to be conditioned on content. - The one-implementation criterion read as banning the decimal extraction and transcoding shims the plan itself specifies. Require one canonical renderer, and confine target-specific code to those two shims, neither of which decides the output text. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AZTXiVaNguCvmjrTjwWLfQ --- ai-plans/0070-try-format-canonical-zero-allocations.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/ai-plans/0070-try-format-canonical-zero-allocations.md b/ai-plans/0070-try-format-canonical-zero-allocations.md index a5a8378..eadd330 100644 --- a/ai-plans/0070-try-format-canonical-zero-allocations.md +++ b/ai-plans/0070-try-format-canonical-zero-allocations.md @@ -2,16 +2,16 @@ ## Rationale -`MetadataValue.TryFormatCanonical` promises a span-based formatting API, but only `Double` and `Single` honor it. Every other kind falls through to `ToCanonicalString()` and copies the resulting string into the destination. For `Null`, `Boolean`, `String`, and `Uri` that is already free — they return a literal or the stored reference — but for the remaining ten kinds the caller pays a string allocation for exactly the values it tried to format without one. `JsonCloudEventsExtensions.WriteStringAttribute` is the visible victim: it stack-allocates a buffer, calls `TryFormatCanonical`, and still allocates a throwaway string for every `Guid`, `DateTime`, or `Int64` extension attribute it writes. The same missing primitives cost allocations on the read path, where every `TryGetXxx` overload that accepts canonical text validates by formatting the parsed value back to a `string` and comparing it. +`MetadataValue.TryFormatCanonical` promises a span-based formatting API, but only `Double` and `Single` honor it. Every other kind falls through to `ToCanonicalString()` and copies the resulting string into the destination. For `Null`, `Boolean`, `String`, and `Uri` that is already free — they return a literal or the stored reference — but for the remaining ten kinds the caller pays a string allocation for exactly the values it tried to format without one. `JsonCloudEventsExtensions.WriteStringAttribute` is the visible victim: it stack-allocates a buffer, calls `TryFormatCanonical`, and still allocates a throwaway string for every `Guid`, `DateTime`, or `Int64` extension attribute it writes. The same missing primitives cost allocations on the read path: the nine `TryGetXxx` overloads that confirm a parse by reformatting — `Int64`, `UInt64`, `Single`, the four date and time kinds, `TimeSpan`, and `Guid` — each build a throwaway `string` per candidate value just to compare it. `TryGetDecimal`, `TryGetChar`, and `TryGetUri` accept text without reformatting and are unaffected. -The library's serializers ultimately write UTF-8. `CanonicalFloatingPointFormatter` already renders both encodings from one implementation by generalizing its renderer over the output code unit, and #61 deferred the serializer integration that makes the saving observable — `MetadataExtensions.WriteNumberValue` still allocates a string per `Double` and lets `System.Text.Json` transcode it back to ASCII. Because the remaining canonical encodings are ASCII too, extending that same generic renderer to them yields the UTF-8 path at almost no additional cost, and skipping it now would mean a second pass over these files later. +The library's serializers ultimately write UTF-8. `CanonicalFloatingPointFormatter` already renders both encodings from one implementation by generalizing its renderer over the output code unit, and #61 deferred the serializer integration that makes the saving observable — `MetadataExtensions.WriteNumberValue` still allocates a string per `Double` and lets `System.Text.Json` transcode it back to ASCII. Because the remaining non-text canonical encodings are ASCII too, extending that same generic renderer to them yields the UTF-8 path at almost no additional cost, and skipping it now would mean a second pass over these files later. The text-bearing kinds — `String`, `Char`, and `Uri` — are the exception and need transcoding rather than rendering. Introduce canonical span formatters for the remaining primitive kinds in both encodings, add the matching UTF-16 and UTF-8 APIs to `MetadataValue`, derive `ToCanonicalString` and the round-trip validators from those formatters so a single implementation defines each encoding, and adopt them in the JSON and CloudEvents writers. ## Acceptance Criteria - [ ] `MetadataValue` exposes `TryFormatCanonical` and a new `TryFormatCanonicalUtf8`, and neither allocates for any primitive kind — including `String`, `Char`, and `Uri`, whose text is copied or transcoded into the caller's destination. -- [ ] One implementation serves both package assets and both encodings: no target-specific formatter, no routing through the other encoding, and no intermediate output buffer — each method writes directly into the caller's destination in the requested encoding. The shared expected-output corpus passes unchanged against both assets. +- [ ] One canonical renderer serves both package assets and both encodings. Target-specific code is confined to the two shims the Technical Details describe — `decimal` field extraction and UTF-16 to UTF-8 transcoding — neither of which decides what the output text is; there is no target-specific formatter or renderer. No method routes through the other encoding or through an intermediate output buffer: each writes directly into the caller's destination in the requested encoding. The shared expected-output corpus passes unchanged against both assets. - [ ] A public `CanonicalTextFormatter` with XML documentation exposes the new canonical formatting primitives in both encodings, plus per-type maximum-length constants that bound both encodings, on both package assets. - [ ] `MetadataValue` exposes a documented public constant bounding the canonical length of every bounded primitive kind in either encoding, and `JsonCloudEventsExtensions` sizes its stack buffer with that constant instead of a private magic number. - [ ] For every value of every primitive kind, `TryFormatCanonical` produces exactly the text that `ToCanonicalString` produced before this change, and `charsWritten` equals that text's length. `TryFormatCanonicalUtf8` produces the replacement-fallback UTF-8 encoding of that same text — lossy only for malformed UTF-16, which has no valid UTF-8 encoding — with `bytesWritten` equal to `charsWritten` for every kind whose canonical text is ASCII. @@ -179,7 +179,7 @@ Lift the conversion helper to a shared internal home rather than duplicating it, `String`, `Char`, and `Uri` are the only kinds whose canonical text is not ASCII, so they are the only ones the UTF-8 path cannot render through the shared renderer. Transcode them with replacement fallback: invalid UTF-16 — an unpaired surrogate, which `FromChar` and `FromString` both accept — becomes U+FFFD rather than a failure, because returning `false` would conflate invalid text with an undersized destination. -That is a lossy encoding, and it must be described as one. Malformed UTF-16 has no corresponding valid UTF-8, so the UTF-8 output for these three kinds is the *replacement-fallback* encoding of the canonical text, not the encoding of that text. `bytesWritten` therefore does not equal `charsWritten` for them, unlike every other kind, and the round trip through UTF-8 is not identity. Scope both assertions accordingly. +That is a lossy encoding, and it must be described as one. Malformed UTF-16 has no corresponding valid UTF-8, so the UTF-8 output for these three kinds is the *replacement-fallback* encoding of the canonical text, not the encoding of that text. `bytesWritten` is therefore not *guaranteed* to equal `charsWritten` for them, unlike every other kind, and the round trip through UTF-8 is not identity. The counts still coincide whenever the text happens to be ASCII, which most `String` and `Uri` values and every ASCII `Char` are, so the assertion must be conditioned on the content rather than asserted or denied per kind. **These three kinds must not be routed through the UTF-8 API by the serializers.** `Utf8JsonWriter` does not agree bytewise with pre-transcoded replacement bytes: when it transcodes malformed UTF-16 itself it emits the *escaped* `�`, whereas valid UTF-8 replacement bytes handed to it are subject only to the encoder's normal escaping rules. Measured against the pinned System.Text.Json 10.0.10, the two routes agree under the default encoder — which escapes all non-ASCII — and diverge under `UnsafeRelaxedJsonEscaping`: From 7c0e1b98ab6e73758c175df59ce848d747ba5e62 Mon Sep 17 00:00:00 2001 From: Kenny Pflug Date: Sun, 2 Aug 2026 11:45:27 +0200 Subject: [PATCH 4/7] docs(metadata): condense the plan without dropping decisions Tighten the allocation-free canonical formatting plan from 248 to 191 lines. All 17 acceptance criteria and every normative instruction are preserved; the reduction is redundant prose, upstream line counts, and a Scope section whose content already appeared elsewhere. Restore five justifications that the condensation had removed: - the derivations of MaximumDayNumber and MaximumTimeOfDayTicks, which netstandard2.0 cannot check against a BCL type - why the encodings must be adapted rather than rederived: the TryGetXxx validators pin them, so a wrong rule breaks data already on the wire - the evidence that the decimal layout assumption is real, namely that GetBits documents the logical representation and upstream DecCalc carries an explicit BIGENDIAN layout - Numbers/ as the established provenance pattern to follow - the rule behind the serializer adoption bullets: the writer transcodes UTF-16 itself and receives UTF-8 only for ASCII kinds Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AZTXiVaNguCvmjrTjwWLfQ --- ...0-try-format-canonical-zero-allocations.md | 238 +++++++----------- 1 file changed, 90 insertions(+), 148 deletions(-) diff --git a/ai-plans/0070-try-format-canonical-zero-allocations.md b/ai-plans/0070-try-format-canonical-zero-allocations.md index eadd330..0fbeb13 100644 --- a/ai-plans/0070-try-format-canonical-zero-allocations.md +++ b/ai-plans/0070-try-format-canonical-zero-allocations.md @@ -2,108 +2,89 @@ ## Rationale -`MetadataValue.TryFormatCanonical` promises a span-based formatting API, but only `Double` and `Single` honor it. Every other kind falls through to `ToCanonicalString()` and copies the resulting string into the destination. For `Null`, `Boolean`, `String`, and `Uri` that is already free — they return a literal or the stored reference — but for the remaining ten kinds the caller pays a string allocation for exactly the values it tried to format without one. `JsonCloudEventsExtensions.WriteStringAttribute` is the visible victim: it stack-allocates a buffer, calls `TryFormatCanonical`, and still allocates a throwaway string for every `Guid`, `DateTime`, or `Int64` extension attribute it writes. The same missing primitives cost allocations on the read path: the nine `TryGetXxx` overloads that confirm a parse by reformatting — `Int64`, `UInt64`, `Single`, the four date and time kinds, `TimeSpan`, and `Guid` — each build a throwaway `string` per candidate value just to compare it. `TryGetDecimal`, `TryGetChar`, and `TryGetUri` accept text without reformatting and are unaffected. +`MetadataValue.TryFormatCanonical` is genuinely span-based only for `Double` and `Single`; every other kind calls `ToCanonicalString()` and copies the result. `Null`, `Boolean`, `String`, and `Uri` already return a literal or stored reference, but the other ten kinds allocate. This affects `JsonCloudEventsExtensions.WriteStringAttribute`, which still creates a throwaway string for values such as `Guid`, `DateTime`, and `Int64`, and the nine `TryGetXxx` validators that reformat candidates (`Int64`, `UInt64`, `Single`, four date/time kinds, `TimeSpan`, and `Guid`). `TryGetDecimal`, `TryGetChar`, and `TryGetUri` do not reformat and are unaffected. -The library's serializers ultimately write UTF-8. `CanonicalFloatingPointFormatter` already renders both encodings from one implementation by generalizing its renderer over the output code unit, and #61 deferred the serializer integration that makes the saving observable — `MetadataExtensions.WriteNumberValue` still allocates a string per `Double` and lets `System.Text.Json` transcode it back to ASCII. Because the remaining non-text canonical encodings are ASCII too, extending that same generic renderer to them yields the UTF-8 path at almost no additional cost, and skipping it now would mean a second pass over these files later. The text-bearing kinds — `String`, `Char`, and `Uri` — are the exception and need transcoding rather than rendering. +The serializers ultimately write UTF-8. `CanonicalFloatingPointFormatter` already renders either encoding from one generic implementation, while #61 deferred its serializer integration: `MetadataExtensions.WriteNumberValue` still creates a `Double` string that System.Text.Json transcodes back to ASCII. The remaining non-text formats are also ASCII; only `String`, `Char`, and `Uri` require transcoding. -Introduce canonical span formatters for the remaining primitive kinds in both encodings, add the matching UTF-16 and UTF-8 APIs to `MetadataValue`, derive `ToCanonicalString` and the round-trip validators from those formatters so a single implementation defines each encoding, and adopt them in the JSON and CloudEvents writers. +Add allocation-free canonical formatters for both encodings, expose them through `MetadataValue`, make them the source for `ToCanonicalString` and validation, and adopt them in the JSON and CloudEvents writers. ## Acceptance Criteria -- [ ] `MetadataValue` exposes `TryFormatCanonical` and a new `TryFormatCanonicalUtf8`, and neither allocates for any primitive kind — including `String`, `Char`, and `Uri`, whose text is copied or transcoded into the caller's destination. -- [ ] One canonical renderer serves both package assets and both encodings. Target-specific code is confined to the two shims the Technical Details describe — `decimal` field extraction and UTF-16 to UTF-8 transcoding — neither of which decides what the output text is; there is no target-specific formatter or renderer. No method routes through the other encoding or through an intermediate output buffer: each writes directly into the caller's destination in the requested encoding. The shared expected-output corpus passes unchanged against both assets. -- [ ] A public `CanonicalTextFormatter` with XML documentation exposes the new canonical formatting primitives in both encodings, plus per-type maximum-length constants that bound both encodings, on both package assets. -- [ ] `MetadataValue` exposes a documented public constant bounding the canonical length of every bounded primitive kind in either encoding, and `JsonCloudEventsExtensions` sizes its stack buffer with that constant instead of a private magic number. -- [ ] For every value of every primitive kind, `TryFormatCanonical` produces exactly the text that `ToCanonicalString` produced before this change, and `charsWritten` equals that text's length. `TryFormatCanonicalUtf8` produces the replacement-fallback UTF-8 encoding of that same text — lossy only for malformed UTF-16, which has no valid UTF-8 encoding — with `bytesWritten` equal to `charsWritten` for every kind whose canonical text is ASCII. -- [ ] The existing failure contracts are unchanged and hold in both encodings: a destination that is too small returns `false`, writes zero to the count, and leaves the destination unmodified; `Array` and `Object` still throw `InvalidOperationException`; and a corrupt `DateTime`, `DateOnly`, or `TimeOnly` payload still throws `InvalidOperationException`. -- [ ] `CanonicalTextFormatter.TryFormat(DateTime, …)` normalizes `DateTimeKind.Local` to UTC, matching `MetadataValue.FromDateTime`, so every `DateTime` it accepts fits `MaximumDateTimeLength`. `Utc` and `Unspecified` values are unaffected, and the behavior is covered by a test against the formatter itself. -- [ ] The `TryGetXxx` round-trip validators compare against span-formatted text and no longer allocate a string per candidate value. -- [ ] `ToCanonicalString` still allocates nothing for the four kinds that are allocation-free today — `Null`, `Boolean`, `String`, and `Uri` — and its output is unchanged for every kind. -- [ ] The JSON metadata writer emits string-shaped and `Double`/`Single` number-shaped metadata values without materializing canonical text, and the bytes it produces for every metadata value are unchanged under both the default and the `UnsafeRelaxedJsonEscaping` encoder, including for text containing non-ASCII characters and unpaired surrogates. -- [ ] Automated tests cover, per kind and per encoding, the canonical text, the written count, the exactly-sufficient and one-short destination, the boundary values of each kind, the invalid-payload throws, and the absence of allocations. The metadata test project passes against both the `net10.0` and the `netstandard2.0` library asset. -- [ ] A microbenchmark compares the new `Guid` and `DateTime` formatters with the framework `TryFormat` APIs on `net10.0`, and its result is recorded in the pull request as evidence for the deferred follow-up decision. No target-specific implementation is introduced in this issue regardless of the result. -- [ ] The CloudEvents and HTTP write benchmark fixtures carry metadata of the kinds this issue affects, and before/after allocation figures for both are recorded in the pull request. -- [ ] `THIRD-PARTY-NOTICES.md` and the folder README record the newly adapted upstream files and the adaptations made to them, following the existing provenance pattern, and every file containing adapted runtime code retains the .NET Foundation MIT header. -- [ ] The `netstandard2.0` decimal path documents its runtime assumption and enforces it without allocating, surfacing `PlatformNotSupportedException` from the decimal formatting path itself, so a runtime that violates it fails loudly instead of producing wrong text and no other kind's formatting is affected. -- [ ] The package release notes mention the new formatting APIs and the removed allocations. -- [ ] Both target frameworks build in Release with warnings as errors, package validation succeeds, the Native AOT sample publishes successfully, and test coverage remains above 95%. +- [ ] `MetadataValue` exposes `TryFormatCanonical` and `TryFormatCanonicalUtf8`; neither allocates for any primitive kind, including copying or transcoding `String`, `Char`, and `Uri` into the caller's destination. +- [ ] One canonical renderer serves both package assets and encodings. Only decimal field extraction and UTF-16-to-UTF-8 transcoding vary by target, and neither decides the output text. There is no target-specific formatter, renderer, cross-encoding route, or intermediate output buffer; every method writes directly to the requested destination. One expected-output corpus passes unchanged against both assets. +- [ ] A public, XML-documented `CanonicalTextFormatter` exposes the new primitives in both encodings and per-type maximum lengths bounding both encodings on both assets. +- [ ] `MetadataValue` exposes a documented public bound for every bounded primitive in either encoding; `JsonCloudEventsExtensions` uses it instead of a private magic number. +- [ ] UTF-16 output exactly matches the previous `ToCanonicalString` text and count. UTF-8 output is that text's replacement-fallback encoding—lossy only for malformed UTF-16—with equal byte/character counts whenever the canonical text is ASCII. +- [ ] In both encodings, insufficient capacity returns `false`, reports zero, and leaves the destination unchanged; `Array` and `Object` still throw `InvalidOperationException`; corrupt `DateTime`, `DateOnly`, and `TimeOnly` payloads still throw `InvalidOperationException`. +- [ ] `CanonicalTextFormatter.TryFormat(DateTime, …)` normalizes `Local` to UTC like `MetadataValue.FromDateTime`, so all accepted values fit `MaximumDateTimeLength`; `Utc` and `Unspecified` are unchanged. A direct test covers this. +- [ ] `TryGetXxx` round-trip validators compare span-formatted text without allocating a candidate string. +- [ ] `ToCanonicalString` remains allocation-free for `Null`, `Boolean`, `String`, and `Uri`, and remains textually unchanged for every kind. +- [ ] The JSON metadata writer materializes no canonical string for string-shaped or `Double`/`Single` number-shaped values. Its bytes remain unchanged under the default and `UnsafeRelaxedJsonEscaping` encoders, including non-ASCII text and unpaired surrogates. +- [ ] Tests cover every kind and encoding: canonical output/count, exact and one-short capacity, boundaries, invalid-payload exceptions, and allocations. Metadata tests pass against the `net10.0` and `netstandard2.0` library assets. +- [ ] A `net10.0` microbenchmark compares the new `Guid` and `DateTime` formatters with framework `TryFormat`; results are recorded in the pull request as evidence for a deferred follow-up, with no target-specific implementation added here regardless of outcome. +- [ ] CloudEvents and HTTP write benchmarks contain affected metadata kinds; before/after allocations for both are recorded in the pull request. +- [ ] `THIRD-PARTY-NOTICES.md` and the folder README identify all adapted upstream files and adaptations; every adapted source retains the .NET Foundation MIT header. +- [ ] The `netstandard2.0` decimal path documents and allocation-freely enforces its runtime-layout assumption. A violation throws `PlatformNotSupportedException` from decimal formatting itself without disabling other formatters. +- [ ] Package release notes mention the new APIs and removed allocations. +- [ ] Both targets build in Release with warnings as errors, package validation succeeds, the Native AOT sample publishes, and coverage remains above 95%. ## Technical Details -### Why the framework APIs are not enough +### Architecture and upstream sources -`netstandard2.0` has no span-formatting API for any of the affected types, and the `Polyfill` package's `TryFormat` extensions are not a substitute: they call `ToString` and copy the result, which is what this issue removes. Meeting the criterion on both assets requires the library to own these formatters regardless of what the `net10.0` asset does, and UTF-8 widens the gap further — `IUtf8SpanFormattable` does not exist on `netstandard2.0` at all. +`netstandard2.0` has no affected span-formatting APIs; `Polyfill` extensions call `ToString` and copy, and `IUtf8SpanFormattable` is unavailable. Ship one owned implementation for both assets and encodings, with no `NET10_0_OR_GREATER` branch to framework formatting. This avoids the asset wire-format divergence fixed for floating point in #58 and keeps one implementation and test surface. -This issue therefore ships exactly one implementation, serving both assets and both encodings, with no `#if NET10_0_OR_GREATER` split onto framework APIs anywhere. A single path removes the risk of the two assets diverging in wire format — the failure #58 fixed for floating-point text — keeps one test surface, and costs one implementation per type instead of two per encoding. +Framework fast paths remain a follow-up informed by the required benchmark. `Guid.TryFormat` has a vectorized `D` path and is plausible; the date kinds are less likely because their canonical custom pattern enters the general format interpreter instead of fixed-shape `TryFormatO`. Do not branch in this issue even if a framework API wins. -Calling into the framework span formatters on `net10.0` remains an open question, not a rejected one: `Guid.TryFormat` in particular has a dedicated vectorized `D` path upstream. That question is deliberately deferred to a follow-up issue, in the same way #61 deferred its serializer integration. This plan produces the evidence for that decision — the microbenchmark below — and stops there. Do not introduce a target-specific branch here, even if the benchmark favors one. +Adapt—not rederive—the scalar implementations from the repository's existing `dotnet/runtime` baseline, tag `v6.0.36`, commit `f1dd57165bfd91875761329ac3a8b17f6606ad18`. This line predates the `System.Runtime.Intrinsics` rewrites and compiles for `netstandard2.0`; newer sources would require de-vectorization. `Numbers/` already establishes the pattern: adapted upstream code under the .NET Foundation MIT header, with provenance in `THIRD-PARTY-NOTICES.md` and a folder README. Rederiving carries the real risk—the `TryGetXxx` validators pin these encodings, so a subtly wrong hand-derived rule breaks round-tripping of data already on the wire. -The benchmark also tells the follow-up where to look. `Guid` is the plausible candidate. The date kinds likely are not: their canonical form is a custom format string, so `DateTime.TryFormat` routes into the general custom-format interpreter rather than the fixed-shape `TryFormatO` fast path this plan ports. +| Kind | Upstream source and adaptation | +| --- | --- | +| `Int64`, `UInt64` | `Number.Formatting.cs`: `TryUInt64ToDecStr`, `UInt32ToDecChars`, `Int64DivMod1E9`; `FormattingHelpers.CountDigits`. Direct scalar port over a fixed destination. | +| `Decimal` | `Number.Formatting.cs`: `DecimalToNumber`; `Decimal.DecCalc.cs`: `DecDivMod1E9`. Reuse the existing `NumberBuffer`; add a scale-preserving renderer. | +| Date/time kinds | `DateTimeFormat.cs`: `TryFormatO`, `WriteTwoDecimalDigits`, `WriteFourDecimalDigits`, `WriteDigits`. Port `TryFormatO`, not `FormatCustomized`. | +| `TimeSpan` | `System.Private.Xml/XsdDuration.cs`: `TimeSpan` constructor and `ToString(DurationType)`, the normative implementation of `XmlConvert.ToString(TimeSpan)`. | +| `Guid` | `Guid.cs`: `TryFormat`'s `D` branch, `HexsToChars`, `HexConverter.ToCharLower`. Numeric-field formatting is endianness-independent. | -### Porting from the BCL +`FormatCustomized` brings culture, calendars, Hebrew/Japanese cases, and `StringBuilderCache`; `TryFormatO` is a small culture-free fixed template. Adapt it to omit a zero fraction and otherwise trim trailing zeros; replace internal `GetDate`/`GetTimePrecise` with ported helpers or public component properties (accepting their repeated tick calculations); drop its local-offset branch for `DateTime` because local values are normalized, while retaining offsets for `DateTimeOffset`. -Do not write these formatters from scratch. `Numbers/` already establishes the pattern — adapted `dotnet/runtime` code under the .NET Foundation MIT header, with provenance in `THIRD-PARTY-NOTICES.md` and the folder README — and every encoding this issue needs has a portable upstream implementation. Porting also retires a correctness risk: these encodings are pinned by the `TryGetXxx` validators, so a hand-derived rule that is subtly wrong breaks round-tripping of data already on the wire. +All upstream renderers target `char`; route ASCII through the repository's `TCodeUnit` conversion pattern so one body writes either encoding. -Stay on the tag the repository already cites, `v6.0.36` (commit `f1dd57165bfd91875761329ac3a8b17f6606ad18`). That is not only for consistency: `release/6.0` predates the `System.Runtime.Intrinsics` rewrites of `Guid.TryFormat` and the number formatters, so its implementations are scalar and compile on `netstandard2.0`, where intrinsics do not exist. Newer lines would have to be de-vectorized by hand. +### Decimal extraction -| Kind | Upstream source | Notes | -| --- | --- | --- | -| `Int64`, `UInt64` | `Number.Formatting.cs`: `TryUInt64ToDecStr`, `UInt32ToDecChars`, `Int64DivMod1E9`; `FormattingHelpers.CountDigits` | Scalar integer math over a `fixed` destination. Direct port. | -| `Decimal` | `Number.Formatting.cs`: `DecimalToNumber`; `Decimal.DecCalc.cs`: `DecDivMod1E9` | `DecDivMod1E9` is twelve lines of integer arithmetic. `DecimalToNumber` emits digits and scale into a `NumberBuffer` — the type this repository already ported — so decimal reuses existing infrastructure and only needs a renderer that honors the scale. | -| `DateTime`, `DateTimeOffset`, `DateOnly`, `TimeOnly` | `DateTimeFormat.cs`: `TryFormatO`, `WriteTwoDecimalDigits`, `WriteFourDecimalDigits`, `WriteDigits` | Port `TryFormatO`, **not** `FormatCustomized`. See below. | -| `TimeSpan` | `System.Private.Xml`, `XsdDuration.cs`: the `TimeSpan` constructor and `ToString(DurationType)` | This is the normative source of `XmlConvert.ToString(TimeSpan)`, which the validators compare against, so porting it settles the encoding by construction rather than by inference. | -| `Guid` | `Guid.cs`: the `D` branch of `TryFormat`, `HexsToChars`, `HexConverter.ToCharLower` | Confirms the field-reinterpretation approach: upstream formats the numeric fields (`_a >> 24`, …), which is endianness-independent. | +`decimal.GetBits(decimal, Span)` is unavailable on `netstandard2.0`; its array overload allocates. Upstream sidesteps this with `Unsafe.As`, but that guarantee is weaker than it looks: `GetBits` documents the *logical* representation, not the private field order, and upstream `DecCalc` carries an explicit `#if BIGENDIAN` layout—endianness is part of the assumption rather than incidental to it. Split only extraction: -Two adaptations are unavoidable and one target is a trap: +- `net10.0` uses the contractual, allocation-free, endian-independent span overload. +- `netstandard2.0` reinterprets the value, documented as requiring a little-endian runtime with historical `flags`, `hi`, `lo`, `mid` field order. Digit generation and rendering remain shared. -- **`FormatCustomized` is the wrong target for the date kinds.** It is a general custom-format interpreter driven by `DateTimeFormatInfo` and `Calendar`, carrying Hebrew and Japanese calendar special cases and `StringBuilderCache`; porting it would be far more work than the fixed template it would produce. `TryFormatO` is roughly seventy-five lines, span-based, culture-free, and already writes `yyyy-MM-ddTHH:mm:ss` followed by a fraction and an offset or `Z`. Adapt it in three places: it writes exactly seven fractional digits where the canonical form omits the fraction entirely when the sub-second ticks are zero and otherwise trims trailing zeros; it reads the components through the runtime-internal `GetDate`/`GetTimePrecise`, so either port those two helpers as well or use the public `Year`/`Month`/`Day`/… properties and accept that each recomputes from the tick count; and its `DateTimeKind.Local` branch, which queries `TimeZoneInfo.Local` and appends the six-character offset, is dropped for `DateTime` because the kind contract normalizes `Local` beforehand. Only the `DateTimeOffset` overload keeps an offset path. -- **Everything upstream renders to `char`.** The ported renderers must write through this repository's `TCodeUnit` conversion helper instead, so both encodings come from one body. This is the same adaptation `CanonicalFloatingPointFormatter` already documents. -- **`decimal.GetBits` is a `netstandard2.0` trap, and the way around it is narrower than it looks.** The `Span` overload is .NET Core 3.0 and later; on `netstandard2.0` only `int[] GetBits(decimal)` exists, and it allocates a four-element array per call — which would defeat the entire issue. Upstream sidesteps it by reading the value through `Unsafe.As`, but that is a weaker guarantee than it appears. `decimal.GetBits` documents the *logical* representation — low, middle, high, flags — not the private field order, and upstream `DecCalc` carries an explicit `#if BIGENDIAN` layout, so endianness is part of the assumption rather than incidental to it. +Validate the legacy assumption once without calling the allocating array overload. Construct - Split the value extraction by target and keep the formatter itself single: - - - `net10.0` calls `decimal.GetBits(decimal, Span)`. It is contractual, allocation-free, and endianness-independent, so this target carries no layout assumption at all. - - `netstandard2.0` uses the reinterpret, with the assumption stated in the XML documentation: a little-endian runtime with the historical `flags`, `hi`, `lo`, `mid` field order. - - This does not reopen the single-implementation decision. What differs is how the four integers are obtained; the digit generation and the renderer are one body, and both extractions are pinned to the same contractual quadruple by `GetBits` semantics. - - Verify the `netstandard2.0` assumption once rather than trusting it, and verify it **without allocating**. Do not reach for `decimal.GetBits(decimal)` in the guard: it allocates a four-element array on first use, which breaches the unqualified no-allocation criterion, and the warmed-up allocation test cannot catch it, because the warm-up iteration triggers type initialization before measurement starts. Seed the probe from the constructor instead: - - ```csharp - var probe = new decimal(lo: 0x11111111, mid: 0x22222222, hi: 0x33333333, isNegative: true, scale: 5); - ``` - - Reinterpret that value and compare the fields against the constructor arguments and the expected flags word — `0x80050000` for the parameters above, being the sign bit and the scale in bits 16 to 23. `decimal(int, int, int, bool, byte)` defines the logical representation exactly as `GetBits` does, so this is no weaker a check, and it touches nothing on the heap: a struct construction, a reinterpret, and four integer comparisons. - - Isolate the guard so its failure is proportionate. Put it in a decimal-specific private helper rather than in `CanonicalTextFormatter`'s own initializer; a throwing initializer on the outer type would disable integer, date, `TimeSpan`, and `Guid` formatting too, none of which depend on the layout. Store the outcome in a `static readonly bool` and throw `PlatformNotSupportedException` from the decimal formatting path, rather than throwing from the initializer — an initializer that throws surfaces to callers as `TypeInitializationException` wrapping the real cause, which contradicts the exception documented on the method. The flag read folds away once the type is initialized, so it costs nothing on the formatting path. - - Note that a mutant in that initializer will be reported as survived regardless of coverage, per the blind spot in `tests/AGENTS.md`. - - A unit test comparing the reinterpret against `decimal.GetBits` is still worth having, but be precise about what it proves: the test suite runs both package assets on the same .NET 10 host, so it validates the compilation target, never the `netstandard2.0` asset's behavior on .NET Framework or Mono. Only the runtime guard covers that case. - -### Formatting rules to verify - -The ports above define the encodings; these are the cases to assert explicitly, because they are where an adaptation slip stays invisible until data fails to round-trip: - -- **Decimal.** The scale is significant — `19.50m` renders as `19.50`, not `19.5`. A negative zero (`decimal.Negate(0m)`, sign bit set, all digits zero) renders **without** a sign, matching the framework. -- **TimeSpan.** `PT0S` for zero, a leading `-` before the `P`, the days component omitted when zero, the `T` present only when a time component follows, and fractional seconds trimmed of trailing zeros (`P2DT3H4M5.06S`, `PT0.0000001S`). Upstream derives the magnitude with `unchecked((ulong)-ticks)` precisely so that `TimeSpan.MinValue`, whose magnitude is not representable as a positive `long`, does not overflow; keep that. -- **DateTime.** `Utc` values end in `Z` and `Unspecified` values carry no designator, matching `yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK`. `Local` values are normalized as described below. `DateTimeOffset` uses the same shape with `+hh:mm`/`-hh:mm` and never `Z`. -- **Guid.** Lowercase `D` format. +```csharp +var probe = new decimal( + lo: 0x11111111, + mid: 0x22222222, + hi: 0x33333333, + isNegative: true, + scale: 5 +); +``` -### The DateTime kind contract +then reinterpret and compare the fields with those arguments and flags `0x80050000`. The constructor defines the same logical representation as `GetBits`; this uses only a struct and integers. -`TryFormat(DateTime, …)` normalizes `DateTimeKind.Local` to UTC and renders the result with the `Z` designator, matching `MetadataValue.FromDateTime`. `Utc` and `Unspecified` values are rendered as they are. Document this on the method: for `Local` input the output depends on the machine's local time zone, which is exactly why the library normalizes at construction rather than at the boundary. +Keep the guard in a decimal-only private helper, store its result in `static readonly bool`, and throw `PlatformNotSupportedException` from decimal formatting—not a type initializer, which would wrap it in `TypeInitializationException` and disable unrelated formatters. The initialized flag folds away. A unit test should compare reinterpretation with `decimal.GetBits`, but both assets run on the .NET 10 host, so only the runtime guard covers .NET Framework or Mono. Per `tests/AGENTS.md`, mutants in the initializer are a known static-analysis blind spot. Warmed allocation tests also miss initialization, so first use must be allocation-free by construction. -Without this rule the public formatter would contradict its own bound. `K` renders a `Local` value with a `±hh:mm` offset — 33 characters, five past `MaximumDateTimeLength`, and the upstream `TryFormatO` reserves those six characters for precisely that case. Normalizing first keeps the bound at 28 and keeps the standalone formatter consistent with the only kinds a `MetadataValue` can hold. +### Canonical contracts -Two consequences for the implementation: +- **Decimal:** preserve scale (`19.50m` → `19.50`); omit the sign of negative zero, matching the framework. +- **TimeSpan:** zero is `PT0S`; `-` precedes `P`; omit zero days; include `T` only when a time component follows; trim fractional zeros (`P2DT3H4M5.06S`, `PT0.0000001S`). Preserve upstream's `unchecked((ulong)-ticks)` handling of `TimeSpan.MinValue`. +- **Date/time:** `DateTime` UTC ends in `Z`, Unspecified has no designator, matching `yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK`; `DateTimeOffset` uses `+hh:mm`/`-hh:mm`, never `Z`. Date/time fractions are omitted at zero and otherwise trimmed. +- **Guid:** lowercase `D` format. -- **Normalize conditionally.** Call `ToUniversalTime()` only when `Kind == DateTimeKind.Local`. `DateTime.ToUniversalTime` treats an `Unspecified` value as local and shifts it, so an unconditional call would silently move every `Unspecified` value by the host's offset and destroy the no-designator form that this library deliberately preserves. `FromDateTime` already guards the same way. -- **Normalization cannot throw.** `ToUniversalTime` saturates at `DateTime.MinValue` and `DateTime.MaxValue` instead of overflowing, so the formatter gains no failure mode and the conversion needs no guard. +For `DateTime`, conditionally call `ToUniversalTime()` only for `Kind.Local` and render `Z`, matching `FromDateTime`. Document that this input depends on the machine's local time zone. Never normalize Unspecified: `ToUniversalTime()` would treat it as local and shift it. Conversion saturates at `DateTime.MinValue`/`MaxValue`, so it adds no exception. Normalization also keeps the maximum at 28 rather than the 33 characters required by a local `±hh:mm` form. -Because `FromDateTime` normalizes on the way in and `TryReadStoredDateTime` rejects a stored `Local` payload as corrupt, no `MetadataValue` can drive this branch. Cover it with a direct test against `CanonicalTextFormatter` — the case `tests/AGENTS.md` reserves solitary tests for — asserting the normalized text, the written count, and that the output still fits `MaximumDateTimeLength`. +`FromDateTime` already normalizes Local and `TryReadStoredDateTime` rejects a stored Local payload, so only direct formatter tests can reach this branch. Assert normalized text/count/bound plus an Unspecified value that is not shifted. -### Public API +### Public API and failure behavior ```csharp namespace Light.PortableResults.Text; @@ -115,7 +96,7 @@ public static class CanonicalTextFormatter public const int MaximumDecimalLength = 31; public const int MaximumCharLength = 3; - public const int MaximumDayNumber = 3_652_058; // DateOnly.MaxValue.DayNumber + public const int MaximumDayNumber = 3_652_058; // DateOnly.MaxValue.DayNumber public const long MaximumTimeOfDayTicks = 863_999_999_999; // TimeSpan.TicksPerDay - 1 public const int MaximumDateTimeLength = 28; public const int MaximumDateTimeOffsetLength = 33; @@ -135,8 +116,7 @@ public static class CanonicalTextFormatter public static bool TryFormatDate(int dayNumber, Span destination, out int charsWritten); public static bool TryFormatTime(long ticks, Span destination, out int charsWritten); - // One TryFormatUtf8 counterpart per overload above, taking Span and reporting bytesWritten. - + // One Span/bytesWritten TryFormatUtf8 counterpart for every overload above. public static bool TryFormatUtf8( ReadOnlySpan text, Span destination, @@ -145,104 +125,66 @@ public static class CanonicalTextFormatter } ``` -The signatures are exact. - -The `char` overload is required for correctness, not convenience, and must not be dropped as redundant with the `long` one. `char` converts implicitly to `long`, so without an exact overload `TryFormat('x', destination, out _)` compiles and writes `120` — silently, in both encodings. `char` is the only affected type: `int`, `short`, `byte`, and `uint` also bind to the `long` overload, but their canonical text *is* their integer text, so those bindings are correct. Renaming the integer methods instead does not fix it; the call then binds to `TryFormat(decimal)` and still writes `120`. Only an exact match resolves ahead of the implicit conversions. - -**`false` means one thing: the destination was too small.** Invalid input throws. `TryFormatDate` and `TryFormatTime` accept a `dayNumber` in `[0, MaximumDayNumber]` and `ticks` in `[0, MaximumTimeOfDayTicks]` respectively, and throw `ArgumentOutOfRangeException` outside those ranges. Overloading `false` to mean both "invalid" and "does not fit" would leave callers unable to distinguish them and would force `MetadataValue` to re-derive the ranges to know which exception to raise. This also matches the sibling class: `CanonicalFloatingPointFormatter.TryFormat` throws `ArgumentException` for a non-finite value and reserves `false` for capacity. +These signatures are exact. The `char` overload prevents a char literal from binding to `long` (or, if integer methods were renamed, `decimal`) and rendering `'x'` as `120`. Other integral types may correctly bind to `long`. Test overload resolution with a char literal. -`MetadataValue` keeps its own guard rather than relying on that throw, because its contract is `InvalidOperationException` with the existing message, not `ArgumentOutOfRangeException`. The published bounds are what makes that guard safe: it checks against the same constants the formatter enforces, so no magic number is duplicated and the two cannot drift. The formatter's own `ArgumentOutOfRangeException` is consequently unreachable through `MetadataValue` and needs a direct test. +`false` means only insufficient capacity. `TryFormatDate` accepts `[0, MaximumDayNumber]`; `TryFormatTime` accepts `[0, MaximumTimeOfDayTicks]`; invalid values throw `ArgumentOutOfRangeException`, consistent with the floating formatter reserving `false` for capacity and throwing for non-finite input. `MetadataValue` pre-validates with the same public constants to retain its existing `InvalidOperationException` and message; test the formatter's otherwise-unreachable range exceptions directly. -`MaximumCharLength` is 3 rather than 1 because, by the same both-encodings convention as the other constants, it must bound the UTF-8 output: a BMP character occupies up to three UTF-8 bytes, and an unpaired surrogate encodes to the three-byte replacement character. The UTF-16 overload never writes more than one. +`MaximumCharLength` is 3 because the constants bound both encodings: one BMP code unit, including an unpaired surrogate replaced by U+FFFD, needs at most three UTF-8 bytes. The UTF-16 method writes one. `TryFormatDate` and `TryFormatTime` accept stored payloads because `DateOnly`/`TimeOnly` BCL types are absent from `netstandard2.0`. The text-transcoding overload needs no UTF-16 peer because chars use `TryCopyTo`. -`MaximumDateTimeLength` is 28 rather than 33 because `TryFormat(DateTime, …)` normalizes `Local` before rendering, so no input reaches the offset form; see the kind contract above. `TryFormatDate` and `TryFormatTime` take the stored payload rather than `DateOnly` and `TimeOnly`, because `MetadataKind.DateOnly` and `MetadataKind.TimeOnly` are formatted on both assets while the BCL types exist only on `net10.0`. The trailing `TryFormatUtf8(ReadOnlySpan, …)` is the transcoding entry point that the `String`, `Char`, and `Uri` kinds need; it has no UTF-16 counterpart because copying chars is `TryCopyTo`. - -`CanonicalFloatingPointFormatter` stays where it is and keeps its own constants and API; moving it would break callers for no gain. - -`MetadataValue` gains one constant and one method: +Keep `CanonicalFloatingPointFormatter` and its API/constants in place. Add to `MetadataValue`: ```csharp public const int MaximumPrimitiveCanonicalLength = 36; - public bool TryFormatCanonicalUtf8(Span destination, out int bytesWritten); ``` -The constant's value is the `Guid` bound, the largest of the bounded kinds. Document that it bounds both encodings and that it excludes `String` and `Uri`, whose canonical text is the caller's own unbounded text — those are the only kinds for which a destination of this size can still return `false`, which is why `WriteStringAttribute` keeps its materializing fallback. `Char` is bounded in both encodings and needs no exclusion: every UTF-16 code unit encodes to at most three UTF-8 bytes. - -### The shared renderer +The bound is `Guid`'s length and covers both encodings. Document that it excludes unbounded `String` and `Uri`; only they may still outgrow this buffer. `Char` remains bounded at three. CloudEvents therefore retains its materializing fallback. -Follow the `TCodeUnit` pattern that `CanonicalFloatingPointFormatter.TryRender` established: a private generic core per type, `where TCodeUnit : unmanaged`, writing ASCII through the existing `typeof(TCodeUnit) == typeof(byte)` conversion helper that the JIT folds away. Every encoding below is pure ASCII, so one renderer produces both outputs and the two public overloads are thin forwarders. Compute the required length before the first write so the all-or-nothing contract holds in both encodings. +### Rendering, transcoding, and single sourcing -Lift the conversion helper to a shared internal home rather than duplicating it, and keep it out of the public surface. The `unmanaged` constraint permits instantiations other than `char` and `byte`; as in #61, do not add a guard for the impossible case, because an unreachable `throw` is a coverage hole. +Follow `CanonicalFloatingPointFormatter.TryRender`: private generic cores constrained to `unmanaged`, a shared internal conversion helper, and JIT-foldable `typeof(TCodeUnit) == typeof(byte)` ASCII writes. Do not expose the helper, duplicate it, or guard impossible non-`char`/`byte` instantiations (an unreachable throw is a coverage hole). Calculate required length before writing to preserve all-or-nothing behavior. -### Transcoding the text-bearing kinds +`String`, `Char`, and `Uri` instead transcode with replacement fallback: unpaired surrogates accepted by `FromChar`/`FromString` become U+FFFD. This is intentionally lossy because malformed UTF-16 has no valid UTF-8 representation; `false` must still mean only insufficient capacity. Counts vary for non-ASCII/malformed text, but match for ASCII; UTF-8 round-trip identity is not promised. -`String`, `Char`, and `Uri` are the only kinds whose canonical text is not ASCII, so they are the only ones the UTF-8 path cannot render through the shared renderer. Transcode them with replacement fallback: invalid UTF-16 — an unpaired surrogate, which `FromChar` and `FromString` both accept — becomes U+FFFD rather than a failure, because returning `false` would conflate invalid text with an undersized destination. +Use `Utf8.FromUtf16(replaceInvalidSequences: true)` on `net10.0`. On `netstandard2.0`, use `Encoding`'s pointer overload under `fixed`, whose default fallback also emits U+FFFD. Both paths count bytes before writing, requiring a second pass but preventing partial output. -That is a lossy encoding, and it must be described as one. Malformed UTF-16 has no corresponding valid UTF-8, so the UTF-8 output for these three kinds is the *replacement-fallback* encoding of the canonical text, not the encoding of that text. `bytesWritten` is therefore not *guaranteed* to equal `charsWritten` for them, unlike every other kind, and the round trip through UTF-8 is not identity. The counts still coincide whenever the text happens to be ASCII, which most `String` and `Uri` values and every ASCII `Char` are, so the assertion must be conditioned on the content rather than asserted or denied per kind. - -**These three kinds must not be routed through the UTF-8 API by the serializers.** `Utf8JsonWriter` does not agree bytewise with pre-transcoded replacement bytes: when it transcodes malformed UTF-16 itself it emits the *escaped* `�`, whereas valid UTF-8 replacement bytes handed to it are subject only to the encoder's normal escaping rules. Measured against the pinned System.Text.Json 10.0.10, the two routes agree under the default encoder — which escapes all non-ASCII — and diverge under `UnsafeRelaxedJsonEscaping`: +The serializers must not use pre-transcoded UTF-8 for these three kinds. With System.Text.Json 10.0.10 and `UnsafeRelaxedJsonEscaping`, malformed input differs bytewise: ```text -input "a\uD800b", relaxed encoder - via ReadOnlySpan 22-61-5C-75-46-46-46-44-62-22 "a�b" - via transcoded UTF-8 22-61-EF-BF-BD-62-22 "ab" +input "a\uD800b" +char route: 22-61-5C-75-46-46-46-44-62-22 +UTF-8 route: 22-61-EF-BF-BD-62-22 ``` -The results are equivalent after parsing but not byte-identical, which violates the unchanged-output criterion. Note that an already-valid U+FFFD in the input converges under both encoders; only malformed input diverges. See the adoption rules below for what each kind uses instead. - -On `net10.0` the transcoding is `Utf8.FromUtf16` with `replaceInvalidSequences: true`. On `netstandard2.0` neither that API nor the `Span`-based `Encoding.GetBytes` overload exists; use the pointer overload under `fixed`, whose default replacement fallback produces the same U+FFFD bytes. Both paths need the byte count before the first write to preserve the all-or-nothing contract, which costs a second pass over the text — acceptable, and unavoidable without partial-write semantics. - -### Single source of truth - -Invert the current relationship: the span methods become primary and `ToCanonicalString` formats into a stack buffer and calls `ToString` on the written slice. +Both parse equivalently, and both routes converge for valid U+FFFD; the default encoder also hides the difference by escaping non-ASCII. Preserve bytes by letting the writer transcode text-bearing kinds. -The inversion must not regress `ToCanonicalString`, which is allocation-free today for exactly four kinds. Keep every one of them on a fast path that returns an existing reference rather than building a string: +Make span formatting primary. `ToCanonicalString` stack-formats bounded values and creates one string from the written slice, but retains allocation-free fast paths: literals for `Null`/`Boolean`, stored references for `String`/`Uri`. Share literals as `private const string`; both span encodings write the same constant through the ASCII helper—do not add separate `u8` literals. -- `Null` and `Boolean` return string literals (`"null"`, `"true"`, `"false"`). -- `String` and `Uri` return the stored instance. - -Routing the first two through a stack buffer and `ToString()` would allocate on every call, on paths that are free now — `HttpHeaderValueFormatter` formats every Boolean header value this way, and `ToString()` and the validation message formatter both go through `ToCanonicalString`. - -Single-source the literals through `private const string` fields so the two APIs cannot drift: `ToCanonicalString` returns the constant, and the span methods write it through a small ASCII writer built on the shared `TCodeUnit` conversion helper, which serves both encodings from the same constant. Do not introduce a separate `u8` literal for the UTF-8 path; a second literal is a second source of truth. - -Apply the same inversion to the private `FormatXxx` helpers behind the `TryGetXxx` validators: format into a stack buffer and compare with `MemoryExtensions.SequenceEqual` instead of allocating a string per comparison. `TryGetInt64` compares against `value.ToString(CultureInfo.InvariantCulture)` inline and needs the same treatment. +Likewise, validator `FormatXxx` helpers stack-format and compare with `MemoryExtensions.SequenceEqual`; replace `TryGetInt64`'s inline invariant `ToString` too. ### Serializer adoption -The rule is that the writer performs any UTF-16 transcoding itself; the serializers only hand it UTF-8 for kinds whose canonical text is ASCII, where the two encodings cannot disagree. - -- `MetadataExtensions.WriteNumberValue` writes the `Double` and `Single` canonical text as UTF-8 through `WriteRawValue(ReadOnlySpan, skipInputValidation: true)`, which closes the integration deferred by #61. These are ASCII, so this is safe. -- `MetadataExtensions.WriteMetadataValue` writes the ASCII-bounded string-shaped kinds — the date kinds, `TimeSpan`, `Guid`, `UInt64` — from a stack buffer, in either encoding. -- `String` and `Uri` keep their current path and are passed to the writer as the stored string. They already own one, so nothing is allocated, and the writer's own transcoding is what produces today's bytes. -- `Char` is written through the UTF-16 span overload, not the UTF-8 one. It is a single code unit that may be an unpaired surrogate, so the writer must be the one to transcode it. -- `JsonCloudEventsExtensions.WriteStringAttribute` keeps its shape and switches to the constant. It stays on the UTF-16 path, which is what it already uses; CloudEvents extension attributes additionally reject unpaired surrogates upstream of it, so the divergence above cannot arise there. -- `HttpHeaderValueFormatter` is unchanged: it returns `StringValues` and needs strings regardless. - -### Benchmarks - -The existing write benchmarks cannot demonstrate anything as they stand: both fixtures build their metadata exclusively from `MetadataValue.FromString`, and the `String` kind returns the stored reference before and after this change. Measured unmodified, the before/after delta is identically zero. Extend the fixtures first, then capture the figures. - -- **CloudEvents.** Add `Guid`, `DateTime`, `Double`, and an `Int64` outside the inclusive 32-bit signed range, annotated `SerializeInCloudEventsExtensionAttributes`. The range qualifier is load-bearing: an in-range `Int64` takes the `Integer` attribute encoding and is written with `WriteNumberValue`, never touching canonical text, so only the out-of-range value routes to `WriteStringAttribute` — the method this issue fixes. -- **HTTP.** Add the same kinds annotated `SerializeInHttpResponseBody`. Do not try to show the improvement through `SerializeInHttpHeader`: `HttpHeaderValueFormatter` returns `StringValues` and needs a materialized string regardless, so the header path keeps its allocation by design and would flatline. - -### Testing +The rule: the writer performs any UTF-16 transcoding itself; serializers hand it UTF-8 only for kinds whose canonical text is ASCII, where the two encodings cannot disagree. -Test the canonical text through `MetadataValue` (sociable), and reach `CanonicalTextFormatter` directly only for inputs no `MetadataValue` can carry. Derive expectations from the framework `ToString` calls the current implementation uses, so the tests state that the encoding is unchanged rather than restating the new implementation; assert the UTF-8 output against the ASCII bytes of those same expectations, following #61's precedent. Cover per kind: minimum and maximum values, zero and negative zero where representable, the scale-bearing decimal cases, `TimeSpan.MinValue`/`MaxValue`/`Zero` and the component-omission combinations, sub-second-tick presence and absence for the date kinds, and both `DateTimeKind` values `FromDateTime` can store. Reach `CanonicalTextFormatter` directly for the `Local` kind, which no `MetadataValue` can carry, and assert there that an `Unspecified` value is *not* shifted — the regression an unconditional `ToUniversalTime` would introduce. For the text-bearing kinds add non-ASCII text, a surrogate pair, and an unpaired surrogate in both encodings, asserting the replacement-fallback result rather than a round trip. Pair that with a writer-level test asserting the emitted JSON bytes are unchanged for those inputs under **both** the default and the `UnsafeRelaxedJsonEscaping` encoder — the default encoder escapes all non-ASCII and hides the divergence that motivated the adoption rules, so a single-encoder test proves nothing here. +- `MetadataExtensions.WriteNumberValue` stack-formats `Double`/`Single` as UTF-8 and calls `WriteRawValue(ReadOnlySpan, skipInputValidation: true)`, completing #61. +- `WriteMetadataValue` stack-writes ASCII-bounded string shapes (date kinds, `TimeSpan`, `Guid`, `UInt64`) in either encoding. +- `String`/`Uri` pass their stored string to the writer; `Char` uses UTF-16 so the writer owns surrogate handling. +- `JsonCloudEventsExtensions.WriteStringAttribute` remains UTF-16, switches its stack size to the public constant, and retains fallback. CloudEvents rejects unpaired surrogates upstream. +- `HttpHeaderValueFormatter` remains unchanged because returning `StringValues` inherently requires strings. -Pin the `char` overload resolution with a test that calls `CanonicalTextFormatter.TryFormat` with a `char` literal and asserts the character, not its code point — the failure this guards against is a binding change, so the test must pass a `char` typed argument rather than a variable already narrowed elsewhere. +### Benchmarks and tests -Assert the absence of allocations with `GC.GetAllocatedBytesForCurrentThread` around a warmed-up loop over one value of each affected kind in each encoding, following the precedent in the floating-point formatter tests. Cover `ToCanonicalString` for `Null`, `Boolean`, `String`, and `Uri` in the same way: those four are allocation-free today, so the assertion guards a regression the inversion could otherwise introduce silently, since the returned text would still be correct. +Current write fixtures contain only `FromString`, so their allocation delta is necessarily zero. Add `Guid`, `DateTime`, `Double`, and an `Int64` outside the inclusive 32-bit range: -Be aware of what the warm-up hides: it triggers type initialization and any lazy setup before measurement begins, so a one-time allocation on first use is invisible to these tests. One-time initialization must therefore be allocation-free by construction, not merely amortized — the assertion cannot enforce it. +- CloudEvents: annotate with `SerializeInCloudEventsExtensionAttributes`; the range matters because an in-range `Int64` uses numeric `Integer` encoding and bypasses `WriteStringAttribute`. +- HTTP: annotate with `SerializeInHttpResponseBody`; do not claim a header improvement, because `StringValues` still materializes text. -Extend the same assertions to the `TryGetXxx` validator paths, which the formatting-API tests do not reach. Those paths run only when the value is a `String` kind being parsed — `TryGetGuid` on a `Guid`-kind value returns early without formatting anything — so the test must build `String`-kind values holding canonical text and call `TryGetGuid`, `TryGetDateTime`, `TryGetInt64`, and the rest against them. That is where the per-candidate string allocation lives today. +Test canonical behavior sociably through `MetadataValue`, using direct `CanonicalTextFormatter` tests only for inaccessible inputs such as Local `DateTime` and invalid raw date/time ranges. Derive expected text from the same framework `ToString` calls used before this change; derive UTF-8 expectations from those strings (ASCII where applicable), as in #61. -Every new method carries an `out` parameter, which puts it in Stryker's Safe Mode blind spot documented in `tests/AGENTS.md`: the mutants fail to compile and the enclosing methods receive no mutation coverage at all. Mutation score therefore carries no information about this change. Argue adequacy by hand in the pull request, naming the behavior each formatter promises and the test that constrains it. +Cover each kind's minimum/maximum, zero/negative zero, decimal scale, `TimeSpan.MinValue`/`MaxValue`/`Zero` and component omission, fractions present/absent for date/time values, and both storable `DateTimeKind` values. For text kinds cover non-ASCII, surrogate pairs, and unpaired surrogates in both encodings. At writer level assert exact bytes with both JSON encoders; default-only coverage is insufficient because it hides the malformed-input divergence. -### Scope +Measure allocations with `GC.GetAllocatedBytesForCurrentThread` around warmed loops for every affected kind and encoding, plus the four allocation-free `ToCanonicalString` kinds. Extend this to validators using `String`-kind canonical input; typed values return early and would miss the old allocation. Warm-up hides lazy initialization, hence the allocation-free-construction requirement above. -Deferred to follow-up issues: +All new methods have `out` parameters and therefore fall into Stryker Safe Mode's documented compile-failure blind spot. Mutation score cannot assess them; the pull request must manually map each formatter contract to its constraining tests. -- Whether the `net10.0` asset should call the framework span formatters for any type. This plan measures it and records the numbers; it does not act on them. -- `HttpHeaderValueFormatter` stays as it is: it returns `StringValues` and needs strings regardless. +Framework target-specific calls and any resulting optimization remain deferred. Provenance, release notes, benchmark results, build/package/AOT validation, and coverage are required by the acceptance criteria above. From 70f2b3ff840883689516945e40298f2cf2a35175 Mon Sep 17 00:00:00 2001 From: Kenny Pflug Date: Sun, 2 Aug 2026 13:04:51 +0200 Subject: [PATCH 5/7] feat!: add allocation-free canonical formatting Add shared UTF-16 and UTF-8 canonical formatters for primitive metadata values, and route validation and JSON serialization through allocation-free span paths. Cover the new contracts with cross-asset tests, allocation assertions, benchmarks, provenance, and plan-deviation documentation. Unify floating-point formatting under the partial CanonicalTextFormatter and expose the low-level CanonicalCodeUnit helper in the focused Text namespace. BREAKING CHANGE: CanonicalFloatingPointFormatter has been removed. Use CanonicalTextFormatter for Double and Single canonical formatting. Closes #70 --- THIRD-PARTY-NOTICES.md | 12 +- ...-try-format-canonical-zero-allocations.md} | 34 +- ai-plans/0070-1-plan-deviations.md | 49 + .../CanonicalTextFormatterBenchmarks.cs | 85 ++ ...alTextFormatterFloatingPointBenchmarks.cs} | 12 +- .../CloudEventsWritingBenchmarks.cs | 30 + .../HttpWriteSerializationBenchmarks.cs | 20 + .../Writing/Json/JsonCloudEventsExtensions.cs | 6 +- .../Light.PortableResults.csproj | 6 + .../Metadata/MetadataValue.cs | 465 +++++-- src/Light.PortableResults/Numbers/README.md | 9 +- .../Writing/MetadataExtensions.cs | 31 +- .../Text/CanonicalCodeUnit.cs | 39 + .../CanonicalTextFormatter.FloatingPoint.cs} | 59 +- .../Text/CanonicalTextFormatter.cs | 1096 +++++++++++++++++ src/Light.PortableResults/Text/README.md | 41 + .../Writing/JsonCloudEventsExtensionsTests.cs | 29 +- .../Metadata/CanonicalTextFormatterTests.cs | 504 ++++++++ ...nonicalTextFormatterFloatingPointTests.cs} | 126 +- .../Writing/SharedWritingExtensionsTests.cs | 101 ++ 20 files changed, 2509 insertions(+), 245 deletions(-) rename ai-plans/{0070-try-format-canonical-zero-allocations.md => 0070-0-try-format-canonical-zero-allocations.md} (94%) create mode 100644 ai-plans/0070-1-plan-deviations.md create mode 100644 benchmarks/Benchmarks/CanonicalTextFormatterBenchmarks.cs rename benchmarks/Benchmarks/{CanonicalFloatingPointFormatterBenchmarks.cs => CanonicalTextFormatterFloatingPointBenchmarks.cs} (92%) create mode 100644 src/Light.PortableResults/Text/CanonicalCodeUnit.cs rename src/Light.PortableResults/{Numbers/CanonicalFloatingPointFormatter.cs => Text/CanonicalTextFormatter.FloatingPoint.cs} (86%) create mode 100644 src/Light.PortableResults/Text/CanonicalTextFormatter.cs create mode 100644 src/Light.PortableResults/Text/README.md create mode 100644 tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs rename tests/Light.PortableResults.Tests/Numbers/{CanonicalFloatingPointFormatterTests.cs => CanonicalTextFormatterFloatingPointTests.cs} (80%) diff --git a/THIRD-PARTY-NOTICES.md b/THIRD-PARTY-NOTICES.md index f973402..5198f0e 100644 --- a/THIRD-PARTY-NOTICES.md +++ b/THIRD-PARTY-NOTICES.md @@ -1,15 +1,19 @@ # Third-Party Notices -Light.PortableResults incorporates adapted floating-point number-formatting code from: +Light.PortableResults incorporates adapted primitive-formatting code from: - Project: .NET Runtime - Repository: https://github.com/dotnet/runtime - Source line: `release/6.0` - Immutable source: tag `v6.0.36`, commit `f1dd57165bfd91875761329ac3a8b17f6606ad18` -- Adapted files: `Number.Grisu3.cs`, `Number.DiyFp.cs`, `Number.Dragon4.cs`, - `Number.BigInteger.cs`, `Number.NumberBuffer.cs`, and `Number.Formatting.cs` +- Adapted files under `src/libraries/System.Private.CoreLib/src/System`: + `Number.Grisu3.cs`, `Number.DiyFp.cs`, `Number.Dragon4.cs`, `Number.BigInteger.cs`, + `Number.NumberBuffer.cs`, `Number.Formatting.cs`, `Decimal.DecCalc.cs`, + `Globalization/DateTimeFormat.cs`, and `Guid.cs` +- Adapted XML file: `src/libraries/System.Private.Xml/src/System/Xml/Schema/XsdDuration.cs` - Local files: the adapted implementation files under - `src/Light.PortableResults/Numbers`, as detailed in that folder's `README.md` + `src/Light.PortableResults/Numbers` and `src/Light.PortableResults/Text`, as detailed in those + folders' `README.md` files ## .NET Foundation MIT License diff --git a/ai-plans/0070-try-format-canonical-zero-allocations.md b/ai-plans/0070-0-try-format-canonical-zero-allocations.md similarity index 94% rename from ai-plans/0070-try-format-canonical-zero-allocations.md rename to ai-plans/0070-0-try-format-canonical-zero-allocations.md index 0fbeb13..61b08fd 100644 --- a/ai-plans/0070-try-format-canonical-zero-allocations.md +++ b/ai-plans/0070-0-try-format-canonical-zero-allocations.md @@ -10,23 +10,23 @@ Add allocation-free canonical formatters for both encodings, expose them through ## Acceptance Criteria -- [ ] `MetadataValue` exposes `TryFormatCanonical` and `TryFormatCanonicalUtf8`; neither allocates for any primitive kind, including copying or transcoding `String`, `Char`, and `Uri` into the caller's destination. -- [ ] One canonical renderer serves both package assets and encodings. Only decimal field extraction and UTF-16-to-UTF-8 transcoding vary by target, and neither decides the output text. There is no target-specific formatter, renderer, cross-encoding route, or intermediate output buffer; every method writes directly to the requested destination. One expected-output corpus passes unchanged against both assets. -- [ ] A public, XML-documented `CanonicalTextFormatter` exposes the new primitives in both encodings and per-type maximum lengths bounding both encodings on both assets. -- [ ] `MetadataValue` exposes a documented public bound for every bounded primitive in either encoding; `JsonCloudEventsExtensions` uses it instead of a private magic number. -- [ ] UTF-16 output exactly matches the previous `ToCanonicalString` text and count. UTF-8 output is that text's replacement-fallback encoding—lossy only for malformed UTF-16—with equal byte/character counts whenever the canonical text is ASCII. -- [ ] In both encodings, insufficient capacity returns `false`, reports zero, and leaves the destination unchanged; `Array` and `Object` still throw `InvalidOperationException`; corrupt `DateTime`, `DateOnly`, and `TimeOnly` payloads still throw `InvalidOperationException`. -- [ ] `CanonicalTextFormatter.TryFormat(DateTime, …)` normalizes `Local` to UTC like `MetadataValue.FromDateTime`, so all accepted values fit `MaximumDateTimeLength`; `Utc` and `Unspecified` are unchanged. A direct test covers this. -- [ ] `TryGetXxx` round-trip validators compare span-formatted text without allocating a candidate string. -- [ ] `ToCanonicalString` remains allocation-free for `Null`, `Boolean`, `String`, and `Uri`, and remains textually unchanged for every kind. -- [ ] The JSON metadata writer materializes no canonical string for string-shaped or `Double`/`Single` number-shaped values. Its bytes remain unchanged under the default and `UnsafeRelaxedJsonEscaping` encoders, including non-ASCII text and unpaired surrogates. -- [ ] Tests cover every kind and encoding: canonical output/count, exact and one-short capacity, boundaries, invalid-payload exceptions, and allocations. Metadata tests pass against the `net10.0` and `netstandard2.0` library assets. -- [ ] A `net10.0` microbenchmark compares the new `Guid` and `DateTime` formatters with framework `TryFormat`; results are recorded in the pull request as evidence for a deferred follow-up, with no target-specific implementation added here regardless of outcome. -- [ ] CloudEvents and HTTP write benchmarks contain affected metadata kinds; before/after allocations for both are recorded in the pull request. -- [ ] `THIRD-PARTY-NOTICES.md` and the folder README identify all adapted upstream files and adaptations; every adapted source retains the .NET Foundation MIT header. -- [ ] The `netstandard2.0` decimal path documents and allocation-freely enforces its runtime-layout assumption. A violation throws `PlatformNotSupportedException` from decimal formatting itself without disabling other formatters. -- [ ] Package release notes mention the new APIs and removed allocations. -- [ ] Both targets build in Release with warnings as errors, package validation succeeds, the Native AOT sample publishes, and coverage remains above 95%. +- [x] `MetadataValue` exposes `TryFormatCanonical` and `TryFormatCanonicalUtf8`; neither allocates for any primitive kind, including copying or transcoding `String`, `Char`, and `Uri` into the caller's destination. +- [x] One canonical renderer serves both package assets and encodings. Only decimal field extraction and UTF-16-to-UTF-8 transcoding vary by target, and neither decides the output text. There is no target-specific formatter, renderer, cross-encoding route, or intermediate output buffer; every method writes directly to the requested destination. One expected-output corpus passes unchanged against both assets. +- [x] A public, XML-documented `CanonicalTextFormatter` exposes the new primitives in both encodings and per-type maximum lengths bounding both encodings on both assets. +- [x] `MetadataValue` exposes a documented public bound for every bounded primitive in either encoding; `JsonCloudEventsExtensions` uses it instead of a private magic number. +- [x] UTF-16 output exactly matches the previous `ToCanonicalString` text and count. UTF-8 output is that text's replacement-fallback encoding—lossy only for malformed UTF-16—with equal byte/character counts whenever the canonical text is ASCII. +- [x] In both encodings, insufficient capacity returns `false`, reports zero, and leaves the destination unchanged; `Array` and `Object` still throw `InvalidOperationException`; corrupt `DateTime`, `DateOnly`, and `TimeOnly` payloads still throw `InvalidOperationException`. +- [x] `CanonicalTextFormatter.TryFormat(DateTime, …)` normalizes `Local` to UTC like `MetadataValue.FromDateTime`, so all accepted values fit `MaximumDateTimeLength`; `Utc` and `Unspecified` are unchanged. A direct test covers this. +- [x] `TryGetXxx` round-trip validators compare span-formatted text without allocating a candidate string. +- [x] `ToCanonicalString` remains allocation-free for `Null`, `Boolean`, `String`, and `Uri`, and remains textually unchanged for every kind. +- [x] The JSON metadata writer materializes no canonical string for string-shaped or `Double`/`Single` number-shaped values. Its bytes remain unchanged under the default and `UnsafeRelaxedJsonEscaping` encoders, including non-ASCII text and unpaired surrogates. +- [x] Tests cover every kind and encoding: canonical output/count, exact and one-short capacity, boundaries, invalid-payload exceptions, and allocations. Metadata tests pass against the `net10.0` and `netstandard2.0` library assets. +- [x] A `net10.0` microbenchmark compares the new `Guid` and `DateTime` formatters with framework `TryFormat`; results are recorded in the pull request as evidence for a deferred follow-up, with no target-specific implementation added here regardless of outcome. +- [x] CloudEvents and HTTP write benchmarks contain affected metadata kinds; before/after allocations for both are recorded in the pull request. +- [x] `THIRD-PARTY-NOTICES.md` and the folder README identify all adapted upstream files and adaptations; every adapted source retains the .NET Foundation MIT header. +- [x] The `netstandard2.0` decimal path documents and allocation-freely enforces its runtime-layout assumption. A violation throws `PlatformNotSupportedException` from decimal formatting itself without disabling other formatters. +- [x] Package release notes mention the new APIs and removed allocations. +- [x] Both targets build in Release with warnings as errors, package validation succeeds, the Native AOT sample publishes, and coverage remains above 95%. ## Technical Details diff --git a/ai-plans/0070-1-plan-deviations.md b/ai-plans/0070-1-plan-deviations.md new file mode 100644 index 0000000..7d6569b --- /dev/null +++ b/ai-plans/0070-1-plan-deviations.md @@ -0,0 +1,49 @@ +# Plan Deviations for Allocation-Free Canonical Formatting + +## Referenced Plans + +- `0058-fix-runtime-specific-number-metadata.md` introduced the public + `CanonicalFloatingPointFormatter` for runtime-independent `double` and `float` text. +- `0061-add-utf-8-floating-point-formatting.md` added its UTF-8 overloads and established the + private per-call Dragon4 test seam. +- `0070-0-try-format-canonical-zero-allocations.md` introduced `CanonicalTextFormatter` for the + remaining primitive metadata kinds and explicitly retained the existing floating-point formatter. + +## Deviations + +### Unified formatter surface + +The implemented API no longer exposes a separate `CanonicalFloatingPointFormatter`. Its public +constants and its `Format`, `TryFormat`, and `TryFormatUtf8` overloads for `double` and `float` are now +members of the partial `CanonicalTextFormatter` in `Light.PortableResults.Text`. The floating-point +surface resides in `CanonicalTextFormatter.FloatingPoint.cs`; the Grisu3, Dragon4, number-buffer, and +compatibility implementations remain internal types in `Light.PortableResults.Numbers`. + +Production call sites, tests, and benchmarks use the unified formatter. The floating-point tests +continue to locate the two private generic `TryFormatCore` overloads for their forced-Dragon4 corpus, +so the test seam required by plans 0058 and 0061 is unchanged apart from its declaring type. + +## Rationale + +Once plan 0070 added canonical formatting for every other primitive kind, retaining a second public +formatter made the API harder to discover and forced consumers such as `MetadataValue` to dispatch +between two classes with the same destination, atomicity, and allocation contracts. A partial class +keeps the large floating-point implementation in its own source file without creating a runtime or +performance boundary. The library is not yet stable and permits breaking API changes, so consolidating +the surface now is preferable to preserving the historical split through forwarding APIs. + +This deviation changes API ownership and source organization only. Floating-point text, exceptions, +capacity behavior, allocation behavior, UTF-8 output, algorithm selection, and wire formats remain +unchanged. + +### Public code-unit helper + +Plan 0070 originally required the shared code-unit helper to remain unexposed. The implemented +`CanonicalCodeUnit` is instead a public, top-level type in the focused `Light.PortableResults.Text` +namespace. This follows the repository's hide-in-plain-sight approach: advanced implementation-oriented +types remain accessible without adding them to the main namespace or nesting them inside a facade. + +The type and its `FromAscii` method are XML-documented. The public method supports +`byte` and `char`, matching the formatter's UTF-8 and UTF-16 destinations, and rejects other unmanaged +types with `NotSupportedException`. The supported generic instantiations retain their direct +allocation-free reinterpretation paths. diff --git a/benchmarks/Benchmarks/CanonicalTextFormatterBenchmarks.cs b/benchmarks/Benchmarks/CanonicalTextFormatterBenchmarks.cs new file mode 100644 index 0000000..05825ea --- /dev/null +++ b/benchmarks/Benchmarks/CanonicalTextFormatterBenchmarks.cs @@ -0,0 +1,85 @@ +using System; +using System.Globalization; +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Engines; +using Light.PortableResults.Text; + +namespace Benchmarks; + +[MemoryDiagnoser] +[SimpleJob] +[GroupBenchmarksBy(BenchmarkDotNet.Configs.BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class CanonicalTextFormatterBenchmarks +{ + private readonly Consumer _consumer = new(); + private readonly DateTime[] _dateTimes = + { + DateTime.SpecifyKind(DateTime.MinValue, DateTimeKind.Utc), + new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Utc).AddTicks(1_234_567), + DateTime.SpecifyKind(DateTime.MaxValue, DateTimeKind.Utc) + }; + private readonly Guid[] _guids = + { + Guid.Empty, + new ("a1b2c3d4-e5f6-7890-abcd-ef1234567890"), + new ("ffffffff-ffff-ffff-ffff-ffffffffffff") + }; + + [Benchmark(Baseline = true)] + [BenchmarkCategory("Guid")] + public void FrameworkGuidTryFormat() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumGuidLength]; + foreach (var value in _guids) + { + value.TryFormat(destination, out var charsWritten, "D"); + _consumer.Consume(charsWritten); + _consumer.Consume(destination[0]); + } + } + + [Benchmark] + [BenchmarkCategory("Guid")] + public void CanonicalGuidTryFormat() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumGuidLength]; + foreach (var value in _guids) + { + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten); + _consumer.Consume(charsWritten); + _consumer.Consume(destination[0]); + } + } + + [Benchmark(Baseline = true)] + [BenchmarkCategory("DateTime")] + public void FrameworkDateTimeTryFormat() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + foreach (var value in _dateTimes) + { + value.TryFormat( + destination, + out var charsWritten, + "yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK", + CultureInfo.InvariantCulture + ); + _consumer.Consume(charsWritten); + _consumer.Consume(destination[0]); + } + } + + [Benchmark] + [BenchmarkCategory("DateTime")] + public void CanonicalDateTimeTryFormat() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + foreach (var value in _dateTimes) + { + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten); + _consumer.Consume(charsWritten); + _consumer.Consume(destination[0]); + } + } +} diff --git a/benchmarks/Benchmarks/CanonicalFloatingPointFormatterBenchmarks.cs b/benchmarks/Benchmarks/CanonicalTextFormatterFloatingPointBenchmarks.cs similarity index 92% rename from benchmarks/Benchmarks/CanonicalFloatingPointFormatterBenchmarks.cs rename to benchmarks/Benchmarks/CanonicalTextFormatterFloatingPointBenchmarks.cs index 7c00d43..ddeff49 100644 --- a/benchmarks/Benchmarks/CanonicalFloatingPointFormatterBenchmarks.cs +++ b/benchmarks/Benchmarks/CanonicalTextFormatterFloatingPointBenchmarks.cs @@ -3,14 +3,14 @@ using System.Globalization; using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Engines; -using Light.PortableResults.Numbers; +using Light.PortableResults.Text; namespace Benchmarks; [MemoryDiagnoser] [GroupBenchmarksBy(BenchmarkDotNet.Configs.BenchmarkLogicalGroupRule.ByCategory)] [CategoriesColumn] -public class CanonicalFloatingPointFormatterBenchmarks +public class CanonicalTextFormatterFloatingPointBenchmarks { private const int ValueCount = 1_024; @@ -87,7 +87,7 @@ public int CanonicalDoubleTryFormat() var totalLength = 0; foreach (var value in _doubleValues) { - CanonicalFloatingPointFormatter.TryFormat(value, destination, out var charsWritten); + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten); totalLength += charsWritten; } @@ -116,7 +116,7 @@ public int CanonicalSingleTryFormat() var totalLength = 0; foreach (var value in _singleValues) { - CanonicalFloatingPointFormatter.TryFormat(value, destination, out var charsWritten); + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten); totalLength += charsWritten; } @@ -141,7 +141,7 @@ public void CanonicalDoubleFormat() { foreach (var value in _doubleValues) { - _consumer.Consume(CanonicalFloatingPointFormatter.Format(value)); + _consumer.Consume(CanonicalTextFormatter.Format(value)); } } @@ -163,7 +163,7 @@ public void CanonicalSingleFormat() { foreach (var value in _singleValues) { - _consumer.Consume(CanonicalFloatingPointFormatter.Format(value)); + _consumer.Consume(CanonicalTextFormatter.Format(value)); } } diff --git a/benchmarks/Benchmarks/CloudEventsWritingBenchmarks.cs b/benchmarks/Benchmarks/CloudEventsWritingBenchmarks.cs index 2d2d506..22b87eb 100644 --- a/benchmarks/Benchmarks/CloudEventsWritingBenchmarks.cs +++ b/benchmarks/Benchmarks/CloudEventsWritingBenchmarks.cs @@ -70,6 +70,34 @@ public void Setup() "traceid", MetadataValue.FromString("trace-456", MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes) ); + metadataBuilder.Add( + "correlationid", + MetadataValue.FromGuid( + new Guid("a1b2c3d4-e5f6-7890-abcd-ef1234567890"), + MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes + ) + ); + metadataBuilder.Add( + "recordedat", + MetadataValue.FromDateTime( + new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Utc).AddTicks(1_234_567), + MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes + ) + ); + metadataBuilder.Add( + "measurement", + MetadataValue.FromDouble( + 36_028_797_018_963_968.0, + MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes + ) + ); + metadataBuilder.Add( + "sequence", + MetadataValue.FromInt64( + (long) int.MaxValue + 1, + MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes + ) + ); var metadata = metadataBuilder.Build(); _genericSuccessWithMetadataResult = Result.Ok( @@ -459,4 +487,6 @@ public sealed class ContactDto } [JsonSerializable(typeof(CloudEventsWritingBenchmarks.ContactDto))] +[JsonSerializable(typeof(CloudEventsEnvelopeForWriting))] +[JsonSerializable(typeof(CloudEventsEnvelopeForWriting))] internal partial class CloudEventsWritingBenchmarksJsonContext : JsonSerializerContext; diff --git a/benchmarks/Benchmarks/HttpWriteSerializationBenchmarks.cs b/benchmarks/Benchmarks/HttpWriteSerializationBenchmarks.cs index fdef3f2..dd66df4 100644 --- a/benchmarks/Benchmarks/HttpWriteSerializationBenchmarks.cs +++ b/benchmarks/Benchmarks/HttpWriteSerializationBenchmarks.cs @@ -77,6 +77,26 @@ public void Setup() var metadata = MetadataObject.Create( ("correlationId", MetadataValue.FromString("corr-123", MetadataValueAnnotation.SerializeInHttpResponseBody)), + ("requestId", + MetadataValue.FromGuid( + new Guid("a1b2c3d4-e5f6-7890-abcd-ef1234567890"), + MetadataValueAnnotation.SerializeInHttpResponseBody + )), + ("recordedAt", + MetadataValue.FromDateTime( + new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Utc).AddTicks(1_234_567), + MetadataValueAnnotation.SerializeInHttpResponseBody + )), + ("measurement", + MetadataValue.FromDouble( + 36_028_797_018_963_968.0, + MetadataValueAnnotation.SerializeInHttpResponseBody + )), + ("sequence", + MetadataValue.FromInt64( + (long) int.MaxValue + 1, + MetadataValueAnnotation.SerializeInHttpResponseBody + )), ("traceId", MetadataValue.FromString("trace-456", MetadataValueAnnotation.SerializeInHttpHeader)) ); _genericSuccessWithMetadata = Result.Ok( diff --git a/src/Light.PortableResults/CloudEvents/Writing/Json/JsonCloudEventsExtensions.cs b/src/Light.PortableResults/CloudEvents/Writing/Json/JsonCloudEventsExtensions.cs index fa9e532..ff00bfe 100644 --- a/src/Light.PortableResults/CloudEvents/Writing/Json/JsonCloudEventsExtensions.cs +++ b/src/Light.PortableResults/CloudEvents/Writing/Json/JsonCloudEventsExtensions.cs @@ -11,10 +11,6 @@ namespace Light.PortableResults.CloudEvents.Writing.Json; /// public static class JsonCloudEventsExtensions { - // Guid's canonical D format is the longest bounded primitive encoding. Arbitrarily long String and Uri - // values reuse their existing text when this buffer is insufficient. - private const int CanonicalTextBufferLength = 36; - /// /// Serializes the contents of a into the provided /// using the supplied serializer options. @@ -375,7 +371,7 @@ private static void WriteStringAttribute( MetadataValue value ) { - Span canonicalText = stackalloc char[CanonicalTextBufferLength]; + Span canonicalText = stackalloc char[MetadataValue.MaximumPrimitiveCanonicalLength]; if (value.TryFormatCanonical(canonicalText, out var charsWritten)) { writer.WritePropertyName(attributeName); diff --git a/src/Light.PortableResults/Light.PortableResults.csproj b/src/Light.PortableResults/Light.PortableResults.csproj index 9c32266..abf731b 100644 --- a/src/Light.PortableResults/Light.PortableResults.csproj +++ b/src/Light.PortableResults/Light.PortableResults.csproj @@ -19,6 +19,10 @@ hosts and corrected on .NET Framework and legacy Mono hosts. - Adds allocation-free UTF-8 span formatting for canonical Double and Single values, plus public maximum-length constants shared by the UTF-16 and UTF-8 formatting APIs. + - Adds CanonicalTextFormatter with allocation-free UTF-16 and UTF-8 formatters for all primitive + metadata types, MetadataValue.TryFormatCanonicalUtf8, and public cross-encoding length bounds. + - Removes canonical-string allocations from metadata round-trip validation, JSON metadata writing, + and CloudEvents extension-attribute writing for bounded primitive values. - Emits primitive metadata arrays as ordered, separate HTTP header values. Custom header converters can reuse the new public HttpHeaderValueFormatter. @@ -28,6 +32,8 @@ - Values of the ten newly typed BCL metadata types no longer flatten to MetadataKind.String. Their JSON representations now use the canonical encoding of their dedicated kind. - Whole-number Double and Single metadata values serialize with a trailing .0. + - CanonicalFloatingPointFormatter has been consolidated into the partial CanonicalTextFormatter; + its floating-point constants and formatting overloads now reside on CanonicalTextFormatter. - Default HTTP header conversion no longer surrounds string metadata values with quotes. - Float metadata values now use MetadataKind.Single rather than MetadataKind.Double. - Decimal metadata values now have the dedicated kind MetadataKind.Decimal instead of MetadataKind.String, diff --git a/src/Light.PortableResults/Metadata/MetadataValue.cs b/src/Light.PortableResults/Metadata/MetadataValue.cs index d5bb530..30b759b 100644 --- a/src/Light.PortableResults/Metadata/MetadataValue.cs +++ b/src/Light.PortableResults/Metadata/MetadataValue.cs @@ -1,7 +1,7 @@ using System; using System.Globalization; using System.Xml; -using Light.PortableResults.Numbers; +using Light.PortableResults.Text; namespace Light.PortableResults.Metadata; @@ -13,16 +13,26 @@ namespace Light.PortableResults.Metadata; /// public readonly struct MetadataValue : IEquatable { - private const string DateTimeFormat = "yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK"; - private const string DateTimeOffsetFormat = "yyyy-MM-dd'T'HH:mm:ss.FFFFFFFzzz"; private const string DateOnlyFormat = "yyyy-MM-dd"; - private const string TimeOnlyFormat = "HH:mm:ss.FFFFFFF"; + private const string NullCanonicalText = "null"; + private const string TrueCanonicalText = "true"; + private const string FalseCanonicalText = "false"; /// /// Gets the default annotation for metadata values, which is . /// public const MetadataValueAnnotation DefaultAnnotation = MetadataValueAnnotation.SerializeInBodies; + /// + /// A destination size that holds the canonical text of every bounded primitive metadata value, + /// in UTF-16 characters or UTF-8 bytes. + /// + /// + /// and are unbounded and may + /// require a larger destination. Every other primitive kind fits this bound. + /// + public const int MaximumPrimitiveCanonicalLength = CanonicalTextFormatter.MaximumGuidLength; + private readonly MetadataPayload _payload; private MetadataValue( @@ -54,13 +64,13 @@ private MetadataValue( /// /// Gets a representing a null value. /// - public static MetadataValue Null => new (MetadataKind.Null, default); + public static MetadataValue Null => new(MetadataKind.Null, default); /// /// Creates a null metadata value. /// public static MetadataValue FromNull(MetadataValueAnnotation annotation = DefaultAnnotation) => - new (MetadataKind.Null, default, annotation); + new(MetadataKind.Null, default, annotation); /// /// Creates a metadata value from a Boolean. @@ -69,7 +79,7 @@ public static MetadataValue FromBoolean( bool value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Boolean, new MetadataPayload(value ? 1L : 0L), annotation); + new(MetadataKind.Boolean, new MetadataPayload(value ? 1L : 0L), annotation); /// /// Creates a metadata value from a signed 64-bit integer. @@ -78,7 +88,7 @@ public static MetadataValue FromInt64( long value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Int64, new MetadataPayload(value), annotation); + new(MetadataKind.Int64, new MetadataPayload(value), annotation); /// /// Creates a metadata value from a double-precision floating-point number. @@ -115,7 +125,7 @@ public static MetadataValue FromDecimal( decimal value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Decimal, new MetadataPayload(value), annotation); + new(MetadataKind.Decimal, new MetadataPayload(value), annotation); /// /// Creates a metadata value from an unsigned 64-bit integer. @@ -124,7 +134,7 @@ public static MetadataValue FromUInt64( ulong value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.UInt64, MetadataPayload.FromUInt64(value), annotation); + new(MetadataKind.UInt64, MetadataPayload.FromUInt64(value), annotation); /// /// Creates a metadata value from a single-precision floating-point number. @@ -146,7 +156,7 @@ public static MetadataValue FromChar( char value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Char, new MetadataPayload(value), annotation); + new(MetadataKind.Char, new MetadataPayload(value), annotation); /// /// @@ -182,7 +192,7 @@ public static MetadataValue FromDateTimeOffset( DateTimeOffset value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.DateTimeOffset, new MetadataPayload(value), annotation); + new(MetadataKind.DateTimeOffset, new MetadataPayload(value), annotation); #if NET10_0_OR_GREATER /// @@ -211,7 +221,7 @@ public static MetadataValue FromTimeSpan( TimeSpan value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.TimeSpan, new MetadataPayload(value.Ticks), annotation); + new(MetadataKind.TimeSpan, new MetadataPayload(value.Ticks), annotation); /// /// Creates a metadata value from a globally unique identifier. @@ -220,7 +230,7 @@ public static MetadataValue FromGuid( Guid value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Guid, new MetadataPayload(value), annotation); + new(MetadataKind.Guid, new MetadataPayload(value), annotation); /// /// Creates a metadata value from a URI. A null URI becomes a null metadata value. @@ -392,7 +402,7 @@ public bool TryGetInt64(out long value) if (TryGetRawString(out var text) && long.TryParse(text, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out value) && - string.Equals(value.ToString(CultureInfo.InvariantCulture), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -490,7 +500,7 @@ public bool TryGetUInt64(out ulong value) if (TryGetRawString(out var text) && ulong.TryParse(text, NumberStyles.None, CultureInfo.InvariantCulture, out value) && - string.Equals(FormatUInt64(value), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -512,7 +522,7 @@ public bool TryGetSingle(out float value) float.TryParse(text, NumberStyles.Float, CultureInfo.InvariantCulture, out value) && !float.IsNaN(value) && !float.IsInfinity(value) && - string.Equals(FormatSingle(value), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -560,7 +570,7 @@ public bool TryGetDateTime(out DateTime value) out value ) && value.Kind != DateTimeKind.Local && - string.Equals(FormatDateTime(value), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -600,13 +610,13 @@ out value } private static bool IsCanonicalDateTimeOffsetText(DateTimeOffset value, string text) => - string.Equals(FormatDateTimeOffset(value), text, StringComparison.Ordinal) || + IsCanonical(value, text) || // A zero offset is also written as "Z" by RFC 3339 emitters everywhere, including this library's own // DateTime kind, so it is accepted although the DateTimeOffset writer never produces it. Text without // any designator is still rejected: it parses against the reading machine's local offset and matches // neither form. (value.Offset == TimeSpan.Zero && - string.Equals(FormatDateTime(value.UtcDateTime), text, StringComparison.Ordinal)); + IsCanonical(value.UtcDateTime, text)); #if NET10_0_OR_GREATER /// Attempts to get a date or parse its canonical RFC 3339 full-date encoding. @@ -634,7 +644,7 @@ public bool TryGetDateOnly(out DateOnly value) DateTimeStyles.None, out value ) && - string.Equals(FormatDateOnly(value.DayNumber), text, StringComparison.Ordinal)) + IsCanonicalDate(value.DayNumber, text)) { return true; } @@ -667,7 +677,7 @@ public bool TryGetTimeOnly(out TimeOnly value) DateTimeStyles.None, out value ) && - string.Equals(FormatTimeOnly(value.Ticks), text, StringComparison.Ordinal)) + IsCanonicalTime(value.Ticks, text)) { return true; } @@ -691,7 +701,7 @@ public bool TryGetTimeSpan(out TimeSpan value) try { value = XmlConvert.ToTimeSpan(text); - if (string.Equals(FormatTimeSpan(value), text, StringComparison.Ordinal)) + if (IsCanonical(value, text)) { return true; } @@ -719,7 +729,7 @@ public bool TryGetGuid(out Guid value) if (TryGetRawString(out var text) && Guid.TryParseExact(text, "D", out value) && - string.Equals(FormatGuid(value), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -806,24 +816,13 @@ public MetadataObject AsObject() => public string ToCanonicalString() => Kind switch { - MetadataKind.Null => "null", - MetadataKind.Boolean => _payload.Int64 != 0 ? "true" : "false", - MetadataKind.Int64 => _payload.Int64.ToString(CultureInfo.InvariantCulture), - MetadataKind.Double => FormatDouble(_payload.Float64), + MetadataKind.Null => NullCanonicalText, + MetadataKind.Boolean => _payload.Int64 != 0 ? TrueCanonicalText : FalseCanonicalText, MetadataKind.String => GetRequiredReference(), - MetadataKind.Decimal => GetRequiredReference().ToString(CultureInfo.InvariantCulture), - MetadataKind.UInt64 => FormatUInt64(_payload.UInt64), - MetadataKind.Single => FormatSingle((float) _payload.Float64), - MetadataKind.Char => ((char) _payload.Int64).ToString(), - MetadataKind.DateTime => FormatStoredDateTime(_payload.Int64), - MetadataKind.DateTimeOffset => FormatDateTimeOffset(GetRequiredReference()), - MetadataKind.DateOnly => FormatDateOnly(_payload.Int64), - MetadataKind.TimeOnly => FormatTimeOnly(_payload.Int64), - MetadataKind.TimeSpan => FormatTimeSpan(new TimeSpan(_payload.Int64)), - MetadataKind.Guid => FormatGuid(GetRequiredReference()), MetadataKind.Uri => GetRequiredReference().OriginalString, MetadataKind.Array => ThrowComplexCanonicalText(MetadataKind.Array), - MetadataKind.Object => ThrowComplexCanonicalText(MetadataKind.Object) + MetadataKind.Object => ThrowComplexCanonicalText(MetadataKind.Object), + _ => FormatBoundedCanonical(this) }; /// @@ -831,40 +830,187 @@ public string ToCanonicalString() => /// /// The destination for the encoded characters. /// The number of characters written. - /// when the destination was large enough; otherwise, . + /// + /// when the destination was large enough; otherwise, . + /// On failure, is zero and the destination is unchanged. + /// + /// The value is complex or has a malformed payload. public bool TryFormatCanonical(Span destination, out int charsWritten) { - if (Kind == MetadataKind.Double) - { - return CanonicalFloatingPointFormatter.TryFormat( - _payload.Float64, - destination, - out charsWritten - ); - } - - if (Kind == MetadataKind.Single) + switch (Kind) { - return CanonicalFloatingPointFormatter.TryFormat( - (float) _payload.Float64, - destination, - out charsWritten - ); + case MetadataKind.Null: + return TryCopyAscii(NullCanonicalText, destination, out charsWritten); + case MetadataKind.Boolean: + return TryCopyAscii( + _payload.Int64 != 0 ? TrueCanonicalText : FalseCanonicalText, + destination, + out charsWritten + ); + case MetadataKind.Int64: + return CanonicalTextFormatter.TryFormat(_payload.Int64, destination, out charsWritten); + case MetadataKind.Double: + return CanonicalTextFormatter.TryFormat( + _payload.Float64, + destination, + out charsWritten + ); + case MetadataKind.String: + return TryCopyText(GetRequiredReference(), destination, out charsWritten); + case MetadataKind.Decimal: + return CanonicalTextFormatter.TryFormat( + GetRequiredReference(), + destination, + out charsWritten + ); + case MetadataKind.UInt64: + return CanonicalTextFormatter.TryFormat(_payload.UInt64, destination, out charsWritten); + case MetadataKind.Single: + return CanonicalTextFormatter.TryFormat( + (float) _payload.Float64, + destination, + out charsWritten + ); + case MetadataKind.Char: + return CanonicalTextFormatter.TryFormat( + (char) _payload.Int64, + destination, + out charsWritten + ); + case MetadataKind.DateTime: + return TryFormatStoredDateTime(_payload.Int64, destination, out charsWritten); + case MetadataKind.DateTimeOffset: + return CanonicalTextFormatter.TryFormat( + GetRequiredReference(), + destination, + out charsWritten + ); + case MetadataKind.DateOnly: + return TryFormatStoredDate(_payload.Int64, destination, out charsWritten); + case MetadataKind.TimeOnly: + return TryFormatStoredTime(_payload.Int64, destination, out charsWritten); + case MetadataKind.TimeSpan: + return CanonicalTextFormatter.TryFormat( + new TimeSpan(_payload.Int64), + destination, + out charsWritten + ); + case MetadataKind.Guid: + return CanonicalTextFormatter.TryFormat( + GetRequiredReference(), + destination, + out charsWritten + ); + case MetadataKind.Uri: + return TryCopyText( + GetRequiredReference().OriginalString, + destination, + out charsWritten + ); + case MetadataKind.Array: + case MetadataKind.Object: + return ThrowComplexCanonicalFormat(Kind, out charsWritten); + default: + throw new InvalidOperationException($"Kind '{Kind}' does not have a canonical text encoding."); } + } - var canonicalText = ToCanonicalString(); - if (canonicalText.AsSpan().TryCopyTo(destination)) + /// + /// Attempts to write the replacement-fallback UTF-8 encoding of the canonical text to the supplied destination. + /// + /// The destination for the encoded bytes. + /// The number of bytes written. + /// + /// when the destination was large enough; otherwise, . + /// On failure, is zero and the destination is unchanged. + /// + /// The value is complex or has a malformed payload. + public bool TryFormatCanonicalUtf8(Span destination, out int bytesWritten) + { + switch (Kind) { - charsWritten = canonicalText.Length; - return true; + case MetadataKind.Null: + return TryCopyAscii(NullCanonicalText, destination, out bytesWritten); + case MetadataKind.Boolean: + return TryCopyAscii( + _payload.Int64 != 0 ? TrueCanonicalText : FalseCanonicalText, + destination, + out bytesWritten + ); + case MetadataKind.Int64: + return CanonicalTextFormatter.TryFormatUtf8(_payload.Int64, destination, out bytesWritten); + case MetadataKind.Double: + return CanonicalTextFormatter.TryFormatUtf8( + _payload.Float64, + destination, + out bytesWritten + ); + case MetadataKind.String: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference().AsSpan(), + destination, + out bytesWritten + ); + case MetadataKind.Decimal: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference(), + destination, + out bytesWritten + ); + case MetadataKind.UInt64: + return CanonicalTextFormatter.TryFormatUtf8(_payload.UInt64, destination, out bytesWritten); + case MetadataKind.Single: + return CanonicalTextFormatter.TryFormatUtf8( + (float) _payload.Float64, + destination, + out bytesWritten + ); + case MetadataKind.Char: + return CanonicalTextFormatter.TryFormatUtf8( + (char) _payload.Int64, + destination, + out bytesWritten + ); + case MetadataKind.DateTime: + return TryFormatStoredDateTimeUtf8(_payload.Int64, destination, out bytesWritten); + case MetadataKind.DateTimeOffset: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference(), + destination, + out bytesWritten + ); + case MetadataKind.DateOnly: + return TryFormatStoredDateUtf8(_payload.Int64, destination, out bytesWritten); + case MetadataKind.TimeOnly: + return TryFormatStoredTimeUtf8(_payload.Int64, destination, out bytesWritten); + case MetadataKind.TimeSpan: + return CanonicalTextFormatter.TryFormatUtf8( + new TimeSpan(_payload.Int64), + destination, + out bytesWritten + ); + case MetadataKind.Guid: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference(), + destination, + out bytesWritten + ); + case MetadataKind.Uri: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference().OriginalString.AsSpan(), + destination, + out bytesWritten + ); + case MetadataKind.Array: + case MetadataKind.Object: + return ThrowComplexCanonicalFormat(Kind, out bytesWritten); + default: + throw new InvalidOperationException($"Kind '{Kind}' does not have a canonical text encoding."); } - - charsWritten = 0; - return false; } internal MetadataValue WithAnnotation(MetadataValueAnnotation annotation) => - new (Kind, _payload, annotation); + new(Kind, _payload, annotation); /// public bool Equals(MetadataValue other) @@ -1017,11 +1163,113 @@ private static void ValidateFinite(double value, string parameterName) } } - private static string FormatUInt64(ulong value) => value.ToString(CultureInfo.InvariantCulture); + private static string FormatBoundedCanonical(MetadataValue value) + { + Span destination = stackalloc char[MaximumPrimitiveCanonicalLength]; + value.TryFormatCanonical(destination, out var charsWritten); + return destination.Slice(0, charsWritten).ToString(); + } + + private static bool TryCopyText( + string text, + Span destination, + out int charsWritten + ) + { + if (!text.AsSpan().TryCopyTo(destination)) + { + charsWritten = 0; + return false; + } + + charsWritten = text.Length; + return true; + } + + private static bool TryCopyAscii( + string text, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if (destination.Length < text.Length) + { + unitsWritten = 0; + return false; + } + + for (var index = 0; index < text.Length; index++) + { + destination[index] = CanonicalCodeUnit.FromAscii((byte) text[index]); + } + + unitsWritten = text.Length; + return true; + } + + private static bool IsCanonical(long value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumInt64Length]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(ulong value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumUInt64Length]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(float value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(DateTime value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(DateTimeOffset value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateTimeOffsetLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } - private static string FormatSingle(float value) => CanonicalFloatingPointFormatter.Format(value); + private static bool IsCanonicalDate(int dayNumber, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateLength]; + return CanonicalTextFormatter.TryFormatDate(dayNumber, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } - private static string FormatDouble(double value) => CanonicalFloatingPointFormatter.Format(value); + private static bool IsCanonicalTime(long ticks, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumTimeLength]; + return CanonicalTextFormatter.TryFormatTime(ticks, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(TimeSpan value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumTimeSpanLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(Guid value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumGuidLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } private static bool TryReadStoredDateTime(long binaryValue, out DateTime value) { @@ -1047,47 +1295,88 @@ private static bool TryReadStoredDateTime(long binaryValue, out DateTime value) } } - private static string FormatStoredDateTime(long binaryValue) => + private static bool TryFormatStoredDateTime( + long binaryValue, + Span destination, + out int charsWritten + ) => TryReadStoredDateTime(binaryValue, out var value) ? - FormatDateTime(value) : + CanonicalTextFormatter.TryFormat(value, destination, out charsWritten) : throw new InvalidOperationException("The DateTime metadata payload is invalid."); - private static string FormatDateTime(DateTime value) => - value.ToString(DateTimeFormat, CultureInfo.InvariantCulture); - - private static string FormatDateTimeOffset(DateTimeOffset value) => - value.ToString(DateTimeOffsetFormat, CultureInfo.InvariantCulture); + private static bool TryFormatStoredDateTimeUtf8( + long binaryValue, + Span destination, + out int bytesWritten + ) => + TryReadStoredDateTime(binaryValue, out var value) ? + CanonicalTextFormatter.TryFormatUtf8(value, destination, out bytesWritten) : + throw new InvalidOperationException("The DateTime metadata payload is invalid."); - private static string FormatDateOnly(long dayNumber) + private static bool TryFormatStoredDate( + long dayNumber, + Span destination, + out int charsWritten + ) { - try + if ((ulong) dayNumber > CanonicalTextFormatter.MaximumDayNumber) { - var ticks = checked(dayNumber * TimeSpan.TicksPerDay); - return new DateTime(ticks, DateTimeKind.Unspecified).ToString( - DateOnlyFormat, - CultureInfo.InvariantCulture - ); + throw new InvalidOperationException("The DateOnly metadata payload is invalid."); } - catch (Exception exception) when (exception is ArgumentOutOfRangeException or OverflowException) + + return CanonicalTextFormatter.TryFormatDate((int) dayNumber, destination, out charsWritten); + } + + private static bool TryFormatStoredDateUtf8( + long dayNumber, + Span destination, + out int bytesWritten + ) + { + if ((ulong) dayNumber > CanonicalTextFormatter.MaximumDayNumber) { - throw new InvalidOperationException("The DateOnly metadata payload is invalid.", exception); + throw new InvalidOperationException("The DateOnly metadata payload is invalid."); } + + return CanonicalTextFormatter.TryFormatDateUtf8((int) dayNumber, destination, out bytesWritten); } - private static string FormatTimeOnly(long ticks) + private static bool TryFormatStoredTime( + long ticks, + Span destination, + out int charsWritten + ) { - if (ticks < 0 || ticks >= TimeSpan.TicksPerDay) + if ((ulong) ticks > (ulong) CanonicalTextFormatter.MaximumTimeOfDayTicks) { throw new InvalidOperationException("The TimeOnly metadata payload is invalid."); } - return new DateTime(ticks, DateTimeKind.Unspecified).ToString(TimeOnlyFormat, CultureInfo.InvariantCulture); + return CanonicalTextFormatter.TryFormatTime(ticks, destination, out charsWritten); } - private static string FormatTimeSpan(TimeSpan value) => XmlConvert.ToString(value); + private static bool TryFormatStoredTimeUtf8( + long ticks, + Span destination, + out int bytesWritten + ) + { + if ((ulong) ticks > (ulong) CanonicalTextFormatter.MaximumTimeOfDayTicks) + { + throw new InvalidOperationException("The TimeOnly metadata payload is invalid."); + } + + return CanonicalTextFormatter.TryFormatTimeUtf8(ticks, destination, out bytesWritten); + } - // The "D" format is specified to produce lowercase hexadecimal digits, so no additional lowering is needed. - private static string FormatGuid(Guid value) => value.ToString("D", CultureInfo.InvariantCulture); + private static bool ThrowComplexCanonicalFormat( + MetadataKind kind, + out TCodeUnit unitsWritten + ) + { + unitsWritten = default!; + throw new InvalidOperationException($"Kind '{kind}' does not have a primitive canonical text encoding."); + } private static string ThrowComplexCanonicalText(MetadataKind kind) => throw new InvalidOperationException($"Kind '{kind}' does not have a primitive canonical text encoding."); diff --git a/src/Light.PortableResults/Numbers/README.md b/src/Light.PortableResults/Numbers/README.md index 00d53c5..3ffd686 100644 --- a/src/Light.PortableResults/Numbers/README.md +++ b/src/Light.PortableResults/Numbers/README.md @@ -15,8 +15,9 @@ The following upstream files under - `Number.NumberBuffer.cs` - `Number.Formatting.cs` (IEEE bit extraction and the floating-point-to-number dispatch) -Files containing adapted runtime code retain the .NET Foundation MIT header. The public -`CanonicalFloatingPointFormatter` and its invariant renderer are Light.PortableResults code. +Files containing adapted runtime code retain the .NET Foundation MIT header. The floating-point +portion of the public `CanonicalTextFormatter` and its invariant renderer are +Light.PortableResults code. ## Adaptations @@ -47,8 +48,8 @@ Files containing adapted runtime code retain the .NET Foundation MIT header. The ## Retained code that shortest-unique mode cannot reach -Two small regions survive from upstream that no input can execute through -`CanonicalFloatingPointFormatter`. They are kept rather than deleted because they belong to the +Two small regions survive from upstream that no input can execute through the floating-point +overloads on `CanonicalTextFormatter`. They are kept rather than deleted because they belong to the published shape of these algorithms, and cutting into them would make the remaining source harder to compare against its pinned origin. Neither is excluded from code coverage. The reasoning below records why the test suite cannot cover them, so a later reader does not mistake them for a gap. diff --git a/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs b/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs index f7ee235..d31159d 100644 --- a/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs +++ b/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs @@ -66,7 +66,7 @@ MetadataValueAnnotation requiredAnnotation WriteNumberValue(writer, value); break; case MetadataJsonShape.String: - writer.WriteStringValue(value.ToCanonicalString()); + WriteStringValue(writer, value); break; case MetadataJsonShape.Array: value.TryGetArray(out var arrayMetadataValue); @@ -98,7 +98,9 @@ private static void WriteNumberValue(Utf8JsonWriter writer, MetadataValue value) // back as an Int64. The canonical text is therefore built and written raw. Validation is // skipped because the text comes from our own formatter and non-finite values are rejected at // construction, so it is always a well-formed JSON number. - writer.WriteRawValue(value.ToCanonicalString(), skipInputValidation: true); + Span canonicalNumber = stackalloc byte[MetadataValue.MaximumPrimitiveCanonicalLength]; + value.TryFormatCanonicalUtf8(canonicalNumber, out var bytesWritten); + writer.WriteRawValue(canonicalNumber.Slice(0, bytesWritten), skipInputValidation: true); return; case MetadataNumberEncoding.None: throw new InvalidOperationException( @@ -107,6 +109,31 @@ private static void WriteNumberValue(Utf8JsonWriter writer, MetadataValue value) } } + private static void WriteStringValue(Utf8JsonWriter writer, MetadataValue value) + { + switch (value.Kind) + { + case MetadataKind.String: + case MetadataKind.Uri: + // Keep text-bearing values on the UTF-16 writer route. Besides avoiding a redundant + // transcode, this preserves the configured encoder's handling of malformed UTF-16. + writer.WriteStringValue(value.ToCanonicalString()); + return; + case MetadataKind.Char: + value.TryGetChar(out var character); + Span characterText = stackalloc char[1]; + characterText[0] = character; + writer.WriteStringValue(characterText); + return; + default: + // Every remaining string-shaped kind has bounded, entirely ASCII canonical text. + Span canonicalText = stackalloc byte[MetadataValue.MaximumPrimitiveCanonicalLength]; + value.TryFormatCanonicalUtf8(canonicalText, out var bytesWritten); + writer.WriteStringValue(canonicalText.Slice(0, bytesWritten)); + return; + } + } + /// /// Writes the JSON representation for the specified metadata array. /// diff --git a/src/Light.PortableResults/Text/CanonicalCodeUnit.cs b/src/Light.PortableResults/Text/CanonicalCodeUnit.cs new file mode 100644 index 0000000..0f0252a --- /dev/null +++ b/src/Light.PortableResults/Text/CanonicalCodeUnit.cs @@ -0,0 +1,39 @@ +using System; +using System.Runtime.CompilerServices; + +namespace Light.PortableResults.Text; + +/// +/// Converts ASCII bytes to the UTF-8 or UTF-16 code units used by canonical text formatters. +/// +/// +/// This low-level helper lets one generic renderer target either or +/// destinations. Other code-unit types are not supported. +/// +public static class CanonicalCodeUnit +{ + /// Converts an ASCII byte to a UTF-8 byte or UTF-16 character code unit. + /// The destination code-unit type, either or . + /// The ASCII byte to convert. + /// The equivalent code unit. + /// + /// is neither nor . + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TCodeUnit FromAscii(byte value) + where TCodeUnit : unmanaged + { + if (typeof(TCodeUnit) == typeof(byte)) + { + return Unsafe.As(ref value); + } + + if (typeof(TCodeUnit) == typeof(char)) + { + var character = (char) value; + return Unsafe.As(ref character); + } + + throw new NotSupportedException("Only byte and char code units are supported."); + } +} diff --git a/src/Light.PortableResults/Numbers/CanonicalFloatingPointFormatter.cs b/src/Light.PortableResults/Text/CanonicalTextFormatter.FloatingPoint.cs similarity index 86% rename from src/Light.PortableResults/Numbers/CanonicalFloatingPointFormatter.cs rename to src/Light.PortableResults/Text/CanonicalTextFormatter.FloatingPoint.cs index 3e9db0e..eab5be4 100644 --- a/src/Light.PortableResults/Numbers/CanonicalFloatingPointFormatter.cs +++ b/src/Light.PortableResults/Text/CanonicalTextFormatter.FloatingPoint.cs @@ -1,7 +1,7 @@ using System; -using System.Runtime.CompilerServices; +using Light.PortableResults.Numbers; -namespace Light.PortableResults.Numbers; +namespace Light.PortableResults.Text; /// /// Formats finite IEEE 754 binary floating-point values with one runtime-independent invariant encoding. @@ -19,7 +19,7 @@ namespace Light.PortableResults.Numbers; /// including zero, end in .0 so their floating-point JSON shape is unambiguous. /// /// -public static class CanonicalFloatingPointFormatter +public static partial class CanonicalTextFormatter { /// /// A destination size that is always large enough to hold the canonical encoding of any finite @@ -226,30 +226,32 @@ out int unitsWritten var index = 0; if (isNegative) { - destination[index++] = ToCodeUnit((byte) '-'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); } if (useScientificNotation) { - destination[index++] = ToCodeUnit(digits[0]); + destination[index++] = CanonicalCodeUnit.FromAscii(digits[0]); if (digits.Length > 1) { - destination[index++] = ToCodeUnit((byte) '.'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); CopyDigits(digits[1..], destination, ref index); } - destination[index++] = ToCodeUnit((byte) 'E'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'E'); var exponent = scale - 1; - destination[index++] = ToCodeUnit(exponent < 0 ? (byte) '-' : (byte) '+'); + destination[index++] = CanonicalCodeUnit.FromAscii( + exponent < 0 ? (byte) '-' : (byte) '+' + ); WriteExponent(exponent < 0 ? -exponent : exponent, destination, ref index); } else if (scale <= 0) { - destination[index++] = ToCodeUnit((byte) '0'); - destination[index++] = ToCodeUnit((byte) '.'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); for (var zeroIndex = 0; zeroIndex < -scale; zeroIndex++) { - destination[index++] = ToCodeUnit((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); } CopyDigits(digits, destination, ref index); @@ -257,7 +259,7 @@ out int unitsWritten else if (scale < digits.Length) { CopyDigits(digits[..scale], destination, ref index); - destination[index++] = ToCodeUnit((byte) '.'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); CopyDigits(digits[scale..], destination, ref index); } else @@ -265,11 +267,11 @@ out int unitsWritten CopyDigits(digits, destination, ref index); for (var zeroIndex = digits.Length; zeroIndex < scale; zeroIndex++) { - destination[index++] = ToCodeUnit((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); } - destination[index++] = ToCodeUnit((byte) '.'); - destination[index++] = ToCodeUnit((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); } unitsWritten = index; @@ -285,7 +287,7 @@ ref int destinationIndex { for (var index = 0; index < digits.Length; index++) { - destination[destinationIndex++] = ToCodeUnit(digits[index]); + destination[destinationIndex++] = CanonicalCodeUnit.FromAscii(digits[index]); } } @@ -298,24 +300,17 @@ ref int index { if (exponent >= 100) { - destination[index++] = ToCodeUnit((byte) ('0' + exponent / 100)); + destination[index++] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + exponent / 100) + ); exponent %= 100; } - destination[index++] = ToCodeUnit((byte) ('0' + exponent / 10)); - destination[index++] = ToCodeUnit((byte) ('0' + exponent % 10)); - } - - [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static TCodeUnit ToCodeUnit(byte value) - where TCodeUnit : unmanaged - { - if (typeof(TCodeUnit) == typeof(byte)) - { - return Unsafe.As(ref value); - } - - var character = (char) value; - return Unsafe.As(ref character); + destination[index++] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + exponent / 10) + ); + destination[index++] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + exponent % 10) + ); } } diff --git a/src/Light.PortableResults/Text/CanonicalTextFormatter.cs b/src/Light.PortableResults/Text/CanonicalTextFormatter.cs new file mode 100644 index 0000000..ee38f70 --- /dev/null +++ b/src/Light.PortableResults/Text/CanonicalTextFormatter.cs @@ -0,0 +1,1096 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +using System; +using System.Runtime.CompilerServices; +#if !NET10_0_OR_GREATER +using System.Text; +#endif +#if NET10_0_OR_GREATER +using System.Buffers; +using System.Text.Unicode; +#endif + +namespace Light.PortableResults.Text; + +/// +/// Formats primitive values with the culture-independent canonical encodings used by metadata. +/// +/// +/// Every maximum-length constant bounds both UTF-16 characters and UTF-8 bytes. Except for +/// , the corresponding canonical alphabet is ASCII, so the counts +/// in both encodings are equal. +/// +public static partial class CanonicalTextFormatter +{ + /// The maximum canonical length of a signed 64-bit integer. + public const int MaximumInt64Length = 20; + + /// The maximum canonical length of an unsigned 64-bit integer. + public const int MaximumUInt64Length = 20; + + /// The maximum canonical length of a decimal value. + public const int MaximumDecimalLength = 31; + + /// The maximum canonical length of one UTF-16 code unit in either encoding. + public const int MaximumCharLength = 3; + + /// The largest valid day number, equal to DateOnly.MaxValue.DayNumber. + public const int MaximumDayNumber = 3_652_058; + + /// The largest valid time-of-day tick count. + public const long MaximumTimeOfDayTicks = TimeSpan.TicksPerDay - 1; + + /// The maximum canonical length of a . + public const int MaximumDateTimeLength = 28; + + /// The maximum canonical length of a . + public const int MaximumDateTimeOffsetLength = 33; + + /// The canonical length of a date. + public const int MaximumDateLength = 10; + + /// The maximum canonical length of a time of day. + public const int MaximumTimeLength = 16; + + /// The maximum canonical length of a . + public const int MaximumTimeSpanLength = 27; + + /// The canonical length of a GUID in lowercase D format. + public const int MaximumGuidLength = 36; + + private const uint OneBillion = 1_000_000_000; + +#if !NET10_0_OR_GREATER + // The netstandard2.0 asset has no allocation-free decimal.GetBits overload. Its fallback relies + // on the historical little-endian decimal layout used by .NET Framework, .NET, and Mono: + // flags, high, low, middle. Keep the check non-throwing so unrelated formatters remain usable. + private static readonly bool IsLegacyDecimalLayoutSupported = ValidateLegacyDecimalLayout(); +#endif + + /// Attempts to format one UTF-16 code unit. + /// on success; otherwise . + public static bool TryFormat(char value, Span destination, out int charsWritten) + { + if (destination.IsEmpty) + { + charsWritten = 0; + return false; + } + + destination[0] = value; + charsWritten = 1; + return true; + } + + /// Attempts to format a signed 64-bit integer. + /// on success; otherwise . + public static bool TryFormat(long value, Span destination, out int charsWritten) => + TryFormatInt64(value, destination, out charsWritten); + + /// Attempts to format an unsigned 64-bit integer. + /// on success; otherwise . + public static bool TryFormat(ulong value, Span destination, out int charsWritten) => + TryFormatUInt64(value, destination, out charsWritten); + + /// Attempts to format a decimal while preserving its scale. + /// on success; otherwise . + /// + /// The netstandard2.0 asset is running on a platform whose decimal layout is unsupported. + /// + public static bool TryFormat(decimal value, Span destination, out int charsWritten) => + TryFormatDecimal(value, destination, out charsWritten); + + /// + /// Attempts to format a date and time. Local values are converted to UTC; UTC and unspecified + /// values are unchanged. + /// + /// Local conversion depends on the machine's local time zone. + /// on success; otherwise . + public static bool TryFormat(DateTime value, Span destination, out int charsWritten) => + TryFormatDateTime(value, destination, out charsWritten); + + /// Attempts to format a date and time with an offset. + /// on success; otherwise . + public static bool TryFormat( + DateTimeOffset value, + Span destination, + out int charsWritten + ) => + TryFormatDateTimeOffset(value, destination, out charsWritten); + + /// Attempts to format an XML Schema duration. + /// on success; otherwise . + public static bool TryFormat(TimeSpan value, Span destination, out int charsWritten) => + TryFormatTimeSpan(value, destination, out charsWritten); + + /// Attempts to format a GUID in lowercase D format. + /// on success; otherwise . + public static bool TryFormat(Guid value, Span destination, out int charsWritten) => + TryFormatGuid(value, destination, out charsWritten); + + /// Attempts to format a zero-based Gregorian day number as a date. + /// on success; otherwise . + /// is invalid. + public static bool TryFormatDate( + int dayNumber, + Span destination, + out int charsWritten + ) => + TryFormatDateNumber(dayNumber, destination, out charsWritten); + + /// Attempts to format a time-of-day tick count. + /// on success; otherwise . + /// is invalid. + public static bool TryFormatTime( + long ticks, + Span destination, + out int charsWritten + ) => + TryFormatTimeOfDay(ticks, destination, out charsWritten); + + /// Attempts to format one UTF-16 code unit as replacement-fallback UTF-8. + /// on success; otherwise . + public static unsafe bool TryFormatUtf8( + char value, + Span destination, + out int bytesWritten + ) + { + var pointer = &value; + return TryFormatUtf8(new ReadOnlySpan(pointer, 1), destination, out bytesWritten); + } + + /// Attempts to format a signed 64-bit integer as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8(long value, Span destination, out int bytesWritten) => + TryFormatInt64(value, destination, out bytesWritten); + + /// Attempts to format an unsigned 64-bit integer as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8(ulong value, Span destination, out int bytesWritten) => + TryFormatUInt64(value, destination, out bytesWritten); + + /// Attempts to format a decimal as UTF-8 while preserving its scale. + /// on success; otherwise . + /// + /// The netstandard2.0 asset is running on a platform whose decimal layout is unsupported. + /// + public static bool TryFormatUtf8(decimal value, Span destination, out int bytesWritten) => + TryFormatDecimal(value, destination, out bytesWritten); + + /// + /// Attempts to format a date and time as UTF-8. Local values are converted to UTC; UTC and + /// unspecified values are unchanged. + /// + /// Local conversion depends on the machine's local time zone. + /// on success; otherwise . + public static bool TryFormatUtf8( + DateTime value, + Span destination, + out int bytesWritten + ) => + TryFormatDateTime(value, destination, out bytesWritten); + + /// Attempts to format a date and time with an offset as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8( + DateTimeOffset value, + Span destination, + out int bytesWritten + ) => + TryFormatDateTimeOffset(value, destination, out bytesWritten); + + /// Attempts to format an XML Schema duration as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8( + TimeSpan value, + Span destination, + out int bytesWritten + ) => + TryFormatTimeSpan(value, destination, out bytesWritten); + + /// Attempts to format a GUID in lowercase D format as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8(Guid value, Span destination, out int bytesWritten) => + TryFormatGuid(value, destination, out bytesWritten); + + /// Attempts to format a zero-based Gregorian day number as a UTF-8 date. + /// on success; otherwise . + /// is invalid. + public static bool TryFormatDateUtf8( + int dayNumber, + Span destination, + out int bytesWritten + ) => + TryFormatDateNumber(dayNumber, destination, out bytesWritten); + + /// Attempts to format a time-of-day tick count as UTF-8. + /// on success; otherwise . + /// is invalid. + public static bool TryFormatTimeUtf8( + long ticks, + Span destination, + out int bytesWritten + ) => + TryFormatTimeOfDay(ticks, destination, out bytesWritten); + + /// + /// Attempts to transcode UTF-16 text to UTF-8 with replacement fallback for malformed sequences. + /// + /// + /// on success; otherwise . On failure, + /// is zero and the destination is unchanged. + /// + public static unsafe bool TryFormatUtf8( + ReadOnlySpan text, + Span destination, + out int bytesWritten + ) + { + var requiredLength = GetUtf8ByteCountWithReplacement(text); + if (destination.Length < requiredLength) + { + bytesWritten = 0; + return false; + } + +#if NET10_0_OR_GREATER + var status = Utf8.FromUtf16( + text, + destination, + out var charsRead, + out bytesWritten, + replaceInvalidSequences: true, + isFinalBlock: true + ); + return status == OperationStatus.Done && charsRead == text.Length; +#else + fixed (char* textPointer = text) + { + fixed (byte* destinationPointer = destination) + { + bytesWritten = TranscodeUtf16WithReplacement( + textPointer, + text.Length, + destinationPointer, + destination.Length + ); + return true; + } + } +#endif + } + + private static int GetUtf8ByteCountWithReplacement(ReadOnlySpan text) + { + long count = 0; + for (var index = 0; index < text.Length; index++) + { + var character = text[index]; + if (character <= 0x7F) + { + count++; + } + else if (character <= 0x7FF) + { + count += 2; + } + else if (character is >= '\uD800' and <= '\uDBFF' && + index + 1 < text.Length && + text[index + 1] is >= '\uDC00' and <= '\uDFFF') + { + count += 4; + index++; + } + else + { + count += 3; + } + } + + return count <= int.MaxValue ? (int) count : int.MaxValue; + } + +#if !NET10_0_OR_GREATER + private static unsafe int TranscodeUtf16WithReplacement( + char* text, + int textLength, + byte* destination, + int destinationLength + ) + { + var segmentStart = 0; + var destinationIndex = 0; + for (var index = 0; index < textLength; index++) + { + var scalar = (int) text[index]; + if (scalar is >= '\uD800' and <= '\uDBFF' && + index + 1 < textLength && + text[index + 1] is >= '\uDC00' and <= '\uDFFF') + { + scalar = 0x10000 + ((scalar - 0xD800) << 10) + text[index + 1] - 0xDC00; + index++; + continue; + } + + if (scalar is not (>= '\uD800' and <= '\uDFFF')) + { + continue; + } + + destinationIndex += Encoding.UTF8.GetBytes( + text + segmentStart, + index - segmentStart, + destination + destinationIndex, + destinationLength - destinationIndex + ); + destination[destinationIndex++] = 0xEF; + destination[destinationIndex++] = 0xBF; + destination[destinationIndex++] = 0xBD; + segmentStart = index + 1; + } + + destinationIndex += Encoding.UTF8.GetBytes( + text + segmentStart, + textLength - segmentStart, + destination + destinationIndex, + destinationLength - destinationIndex + ); + return destinationIndex; + } +#endif + + private static bool TryFormatInt64( + long value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + var isNegative = value < 0; + var magnitude = isNegative ? unchecked((ulong) -value) : (ulong) value; + var requiredLength = CountDigits(magnitude) + (isNegative ? 1 : 0); + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + WriteUInt64Digits(magnitude, destination, requiredLength, isNegative ? 1 : 0); + if (isNegative) + { + destination[0] = CanonicalCodeUnit.FromAscii((byte) '-'); + } + + unitsWritten = requiredLength; + return true; + } + + private static bool TryFormatUInt64( + ulong value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + var requiredLength = CountDigits(value); + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + WriteUInt64Digits(value, destination, requiredLength, 0); + unitsWritten = requiredLength; + return true; + } + + private static bool TryFormatDecimal( + decimal value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + GetDecimalFields(value, out var low, out var middle, out var high, out var flags); + var scale = (int) ((flags >> 16) & 0xFF); + var isZero = (low | middle | high) == 0; + var isNegative = !isZero && (flags & 0x80000000U) != 0; + var digitCount = isZero ? 1 : CountDecimalDigits(low, middle, high); + + int requiredLength; + if (scale == 0) + { + requiredLength = digitCount + (isNegative ? 1 : 0); + } + else if (scale >= digitCount) + { + requiredLength = scale + 2 + (isNegative ? 1 : 0); + } + else + { + requiredLength = digitCount + 1 + (isNegative ? 1 : 0); + } + + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + var prefixLength = isNegative ? 1 : 0; + if (isNegative) + { + destination[0] = CanonicalCodeUnit.FromAscii((byte) '-'); + } + + int digitStart; + var decimalPointIndex = -1; + if (scale == 0) + { + digitStart = prefixLength; + } + else if (scale >= digitCount) + { + destination[prefixLength] = CanonicalCodeUnit.FromAscii((byte) '0'); + destination[prefixLength + 1] = CanonicalCodeUnit.FromAscii((byte) '.'); + digitStart = prefixLength + 2 + scale - digitCount; + for (var index = prefixLength + 2; index < digitStart; index++) + { + destination[index] = CanonicalCodeUnit.FromAscii((byte) '0'); + } + } + else + { + decimalPointIndex = prefixLength + digitCount - scale; + destination[decimalPointIndex] = CanonicalCodeUnit.FromAscii((byte) '.'); + digitStart = prefixLength; + } + + WriteDecimalDigits( + low, + middle, + high, + destination, + digitStart, + digitCount, + decimalPointIndex + ); + unitsWritten = requiredLength; + return true; + } + + private static bool TryFormatDateTime( + DateTime value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if (value.Kind == DateTimeKind.Local) + { + value = value.ToUniversalTime(); + } + + return TryFormatDateAndTime( + value, + value.Kind == DateTimeKind.Utc, + default, + hasOffset: false, + destination, + out unitsWritten + ); + } + + private static bool TryFormatDateTimeOffset( + DateTimeOffset value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged => + TryFormatDateAndTime( + value.DateTime, + appendUtcDesignator: false, + value.Offset, + hasOffset: true, + destination, + out unitsWritten + ); + + private static bool TryFormatDateAndTime( + DateTime value, + bool appendUtcDesignator, + TimeSpan offset, + bool hasOffset, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + var fraction = value.Ticks % TimeSpan.TicksPerSecond; + var fractionDigits = CountFractionDigits(fraction); + var requiredLength = 19 + + (fractionDigits == 0 ? 0 : fractionDigits + 1) + + (appendUtcDesignator ? 1 : 0) + + (hasOffset ? 6 : 0); + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + WriteFourDigits((uint) value.Year, destination, 0); + destination[4] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteTwoDigits((uint) value.Month, destination, 5); + destination[7] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteTwoDigits((uint) value.Day, destination, 8); + destination[10] = CanonicalCodeUnit.FromAscii((byte) 'T'); + WriteTwoDigits((uint) value.Hour, destination, 11); + destination[13] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) value.Minute, destination, 14); + destination[16] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) value.Second, destination, 17); + + var index = 19; + if (fractionDigits != 0) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); + WriteFraction(fraction, fractionDigits, destination, index); + index += fractionDigits; + } + + if (appendUtcDesignator) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'Z'); + } + else if (hasOffset) + { + var totalMinutes = (int) (offset.Ticks / TimeSpan.TicksPerMinute); + destination[index++] = CanonicalCodeUnit.FromAscii( + totalMinutes < 0 ? (byte) '-' : (byte) '+' + ); + if (totalMinutes < 0) + { + totalMinutes = -totalMinutes; + } + + WriteTwoDigits((uint) (totalMinutes / 60), destination, index); + index += 2; + destination[index++] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) (totalMinutes % 60), destination, index); + index += 2; + } + + unitsWritten = index; + return true; + } + + private static bool TryFormatDateNumber( + int dayNumber, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if ((uint) dayNumber > MaximumDayNumber) + { + throw new ArgumentOutOfRangeException(nameof(dayNumber)); + } + + if (destination.Length < MaximumDateLength) + { + unitsWritten = 0; + return false; + } + + var value = new DateTime((long) dayNumber * TimeSpan.TicksPerDay, DateTimeKind.Unspecified); + WriteFourDigits((uint) value.Year, destination, 0); + destination[4] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteTwoDigits((uint) value.Month, destination, 5); + destination[7] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteTwoDigits((uint) value.Day, destination, 8); + unitsWritten = MaximumDateLength; + return true; + } + + private static bool TryFormatTimeOfDay( + long ticks, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if ((ulong) ticks > (ulong) MaximumTimeOfDayTicks) + { + throw new ArgumentOutOfRangeException(nameof(ticks)); + } + + var fraction = ticks % TimeSpan.TicksPerSecond; + var fractionDigits = CountFractionDigits(fraction); + var requiredLength = 8 + (fractionDigits == 0 ? 0 : fractionDigits + 1); + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + var hours = ticks / TimeSpan.TicksPerHour; + var minutes = ticks / TimeSpan.TicksPerMinute % 60; + var seconds = ticks / TimeSpan.TicksPerSecond % 60; + WriteTwoDigits((uint) hours, destination, 0); + destination[2] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) minutes, destination, 3); + destination[5] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) seconds, destination, 6); + + var index = 8; + if (fractionDigits != 0) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); + WriteFraction(fraction, fractionDigits, destination, index); + index += fractionDigits; + } + + unitsWritten = index; + return true; + } + + private static bool TryFormatTimeSpan( + TimeSpan value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + var ticks = value.Ticks; + var isNegative = ticks < 0; + var magnitude = isNegative ? unchecked((ulong) -ticks) : (ulong) ticks; + var days = magnitude / (ulong) TimeSpan.TicksPerDay; + var hours = magnitude / (ulong) TimeSpan.TicksPerHour % 24; + var minutes = magnitude / (ulong) TimeSpan.TicksPerMinute % 60; + var seconds = magnitude / (ulong) TimeSpan.TicksPerSecond % 60; + var fraction = (long) (magnitude % (ulong) TimeSpan.TicksPerSecond); + var fractionDigits = CountFractionDigits(fraction); + var hasTime = hours != 0 || minutes != 0 || seconds != 0 || fraction != 0; + + var requiredLength = (isNegative ? 1 : 0) + 1; + if (days != 0) + { + requiredLength += CountDigits(days) + 1; + } + + if (hasTime) + { + requiredLength++; + if (hours != 0) + { + requiredLength += CountDigits(hours) + 1; + } + + if (minutes != 0) + { + requiredLength += CountDigits(minutes) + 1; + } + + if (seconds != 0 || fraction != 0) + { + requiredLength += CountDigits(seconds) + 1; + if (fractionDigits != 0) + { + requiredLength += fractionDigits + 1; + } + } + } + else if (days == 0) + { + requiredLength += 3; + } + + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + var index = 0; + if (isNegative) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + } + + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'P'); + if (days != 0) + { + WriteUnsignedComponent(days, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'D'); + } + + if (hasTime) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'T'); + if (hours != 0) + { + WriteUnsignedComponent(hours, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'H'); + } + + if (minutes != 0) + { + WriteUnsignedComponent(minutes, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'M'); + } + + if (seconds != 0 || fraction != 0) + { + WriteUnsignedComponent(seconds, destination, ref index); + if (fractionDigits != 0) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); + WriteFraction(fraction, fractionDigits, destination, index); + index += fractionDigits; + } + + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'S'); + } + } + else if (days == 0) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'T'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'S'); + } + + unitsWritten = index; + return true; + } + + private static bool TryFormatGuid( + Guid value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if (destination.Length < MaximumGuidLength) + { + unitsWritten = 0; + return false; + } + + ref var fields = ref Unsafe.As(ref value); + var index = 0; + WriteHexByte((byte) ((uint) fields.A >> 24), destination, ref index); + WriteHexByte((byte) ((uint) fields.A >> 16), destination, ref index); + WriteHexByte((byte) ((uint) fields.A >> 8), destination, ref index); + WriteHexByte((byte) fields.A, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteHexByte((byte) ((ushort) fields.B >> 8), destination, ref index); + WriteHexByte((byte) fields.B, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteHexByte((byte) ((ushort) fields.C >> 8), destination, ref index); + WriteHexByte((byte) fields.C, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteHexByte(fields.D, destination, ref index); + WriteHexByte(fields.E, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteHexByte(fields.F, destination, ref index); + WriteHexByte(fields.G, destination, ref index); + WriteHexByte(fields.H, destination, ref index); + WriteHexByte(fields.I, destination, ref index); + WriteHexByte(fields.J, destination, ref index); + WriteHexByte(fields.K, destination, ref index); + unitsWritten = index; + return true; + } + + private static void WriteUInt64Digits( + ulong value, + Span destination, + int end, + int start + ) + where TCodeUnit : unmanaged + { + var index = end; + while (value >= OneBillion) + { + var remainder = (uint) (value % OneBillion); + value /= OneBillion; + WriteUInt32Digits(remainder, destination, ref index, 9); + } + + WriteUInt32Digits((uint) value, destination, ref index, index - start); + } + + private static void WriteUInt32Digits( + uint value, + Span destination, + ref int end, + int digits + ) + where TCodeUnit : unmanaged + { + while (digits-- > 0) + { + var quotient = value / 10; + destination[--end] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + value - quotient * 10) + ); + value = quotient; + } + } + + private static void WriteDecimalDigits( + uint low, + uint middle, + uint high, + Span destination, + int digitStart, + int digitCount, + int decimalPointIndex + ) + where TCodeUnit : unmanaged + { + var index = digitStart + digitCount + (decimalPointIndex >= 0 ? 1 : 0); + var remainingDigits = digitCount; + do + { + var group = DivideDecimalByOneBillion(ref low, ref middle, ref high); + var groupDigits = remainingDigits > 9 ? 9 : remainingDigits; + for (var digit = 0; digit < groupDigits; digit++) + { + if (index - 1 == decimalPointIndex) + { + index--; + } + + var quotient = group / 10; + destination[--index] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + group - quotient * 10) + ); + group = quotient; + } + + remainingDigits -= groupDigits; + } + while (remainingDigits != 0); + } + + private static int CountDecimalDigits(uint low, uint middle, uint high) + { + var groupCount = 0; + uint mostSignificantGroup; + do + { + mostSignificantGroup = DivideDecimalByOneBillion(ref low, ref middle, ref high); + groupCount++; + } + while ((low | middle | high) != 0); + + return (groupCount - 1) * 9 + CountDigits(mostSignificantGroup); + } + + private static uint DivideDecimalByOneBillion(ref uint low, ref uint middle, ref uint high) + { + var high64 = ((ulong) high << 32) + middle; + var quotient64 = high64 / OneBillion; + high = (uint) (quotient64 >> 32); + middle = (uint) quotient64; + + var value = ((high64 - (uint) quotient64 * OneBillion) << 32) + low; + var quotient = (uint) (value / OneBillion); + low = quotient; + return (uint) value - quotient * OneBillion; + } + + private static void GetDecimalFields( + decimal value, + out uint low, + out uint middle, + out uint high, + out uint flags + ) + { +#if NET10_0_OR_GREATER + Span bits = stackalloc int[4]; + decimal.GetBits(value, bits); + low = (uint) bits[0]; + middle = (uint) bits[1]; + high = (uint) bits[2]; + flags = (uint) bits[3]; +#else + if (!IsLegacyDecimalLayoutSupported) + { + throw new PlatformNotSupportedException( + "Allocation-free decimal formatting requires a little-endian runtime with the historical decimal field layout." + ); + } + + ref var layout = ref Unsafe.As(ref value); + low = layout.Low; + middle = layout.Middle; + high = layout.High; + flags = layout.Flags; +#endif + } + +#if !NET10_0_OR_GREATER + private static bool ValidateLegacyDecimalLayout() + { + var probe = new decimal( + lo: 0x11111111, + mid: 0x22222222, + hi: 0x33333333, + isNegative: true, + scale: 5 + ); + ref var layout = ref Unsafe.As(ref probe); + return BitConverter.IsLittleEndian && + layout.Flags == 0x80050000U && + layout.High == 0x33333333U && + layout.Low == 0x11111111U && + layout.Middle == 0x22222222U; + } + +#pragma warning disable CS0649 // Fields are populated by reinterpreting decimal storage via Unsafe.As. + private struct DecimalLayout + { + public uint Flags; + public uint High; + public uint Low; + public uint Middle; + } +#pragma warning restore CS0649 +#endif + + private static int CountDigits(ulong value) + { + var count = 1; + while (value >= 10) + { + value /= 10; + count++; + } + + return count; + } + + private static int CountFractionDigits(long fraction) + { + if (fraction == 0) + { + return 0; + } + + var digits = 7; + while (fraction % 10 == 0) + { + fraction /= 10; + digits--; + } + + return digits; + } + + private static void WriteFraction( + long fraction, + int digits, + Span destination, + int start + ) + where TCodeUnit : unmanaged + { + for (var index = 7; index > digits; index--) + { + fraction /= 10; + } + + var end = start + digits; + while (end > start) + { + var quotient = fraction / 10; + destination[--end] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + fraction - quotient * 10) + ); + fraction = quotient; + } + } + + private static void WriteUnsignedComponent( + ulong value, + Span destination, + ref int index + ) + where TCodeUnit : unmanaged + { + var digits = CountDigits(value); + var end = index + digits; + WriteUInt64Digits(value, destination, end, index); + index = end; + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void WriteTwoDigits( + uint value, + Span destination, + int index + ) + where TCodeUnit : unmanaged + { + var quotient = value / 10; + destination[index] = CanonicalCodeUnit.FromAscii((byte) ('0' + quotient)); + destination[index + 1] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + value - quotient * 10) + ); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void WriteFourDigits( + uint value, + Span destination, + int index + ) + where TCodeUnit : unmanaged + { + for (var offset = 3; offset >= 0; offset--) + { + var quotient = value / 10; + destination[index + offset] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + value - quotient * 10) + ); + value = quotient; + } + } + + private static void WriteHexByte( + byte value, + Span destination, + ref int index + ) + where TCodeUnit : unmanaged + { + destination[index++] = CanonicalCodeUnit.FromAscii(ToLowerHex(value >> 4)); + destination[index++] = CanonicalCodeUnit.FromAscii(ToLowerHex(value & 0xF)); + } + + private static byte ToLowerHex(int value) => + (byte) (value < 10 ? '0' + value : 'a' + value - 10); + +#pragma warning disable CS0649 // Fields are populated by reinterpreting Guid storage via Unsafe.As. + private struct GuidFields + { + public int A; + public short B; + public short C; + public byte D; + public byte E; + public byte F; + public byte G; + public byte H; + public byte I; + public byte J; + public byte K; + } +#pragma warning restore CS0649 +} diff --git a/src/Light.PortableResults/Text/README.md b/src/Light.PortableResults/Text/README.md new file mode 100644 index 0000000..c0d1d6d --- /dev/null +++ b/src/Light.PortableResults/Text/README.md @@ -0,0 +1,41 @@ +# Runtime canonical-text provenance + +The scalar formatting routines in this folder are adapted from the +[dotnet/runtime](https://github.com/dotnet/runtime) repository's `release/6.0` servicing line, +using immutable tag `v6.0.36` at commit +`f1dd57165bfd91875761329ac3a8b17f6606ad18`. + +The following upstream files supplied the implementation: + +- `src/libraries/System.Private.CoreLib/src/System/Number.Formatting.cs` +- `src/libraries/System.Private.CoreLib/src/System/Decimal.DecCalc.cs` +- `src/libraries/System.Private.CoreLib/src/System/Globalization/DateTimeFormat.cs` +- `src/libraries/System.Private.CoreLib/src/System/Guid.cs` +- `src/libraries/System.Private.Xml/src/System/Xml/Schema/XsdDuration.cs` + +## Adaptations + +- Retained only invariant signed and unsigned 64-bit decimal formatting and made the renderer generic + over UTF-16 characters and UTF-8 bytes. +- Retained decimal's 96-bit division by one billion, added a scale-preserving direct renderer, and + split field extraction by target. `net10.0` uses `decimal.GetBits(decimal, Span)`; + `netstandard2.0` validates and uses the historical little-endian decimal layout without allocating. +- Reduced round-trip date/time formatting to the metadata canonical forms, omitted zero fractions, + trimmed trailing fractional zeros, normalized local `DateTime` values to UTC, and retained explicit + offsets for `DateTimeOffset`. +- Reduced `XsdDuration` to the `TimeSpan` constructor and duration renderer, including the unchecked + `TimeSpan.MinValue` magnitude conversion and XML Schema component-omission rules. +- Retained GUID's lowercase `D` branch and hexadecimal conversion, reading the sequential numeric + fields without an intermediate byte or string allocation. +- Added all-or-nothing capacity checks and direct generic output so both encodings share each renderer. + Only UTF-16-to-UTF-8 transcoding differs by target. + +The complete upstream license is reproduced in the repository-root `THIRD-PARTY-NOTICES.md`, which +is packed at the root of the `Light.PortableResults` NuGet package. + +## Floating-point formatting + +`CanonicalTextFormatter.FloatingPoint.cs` supplies the `double` and `float` portions of the partial +formatter. Its invariant renderer is Light.PortableResults code, while its shortest-digit generation +uses the internal Grisu3 and Dragon4 implementation under `Numbers/`. See `Numbers/README.md` for that +implementation's provenance and adaptations. diff --git a/tests/Light.PortableResults.Tests/CloudEvents/Writing/JsonCloudEventsExtensionsTests.cs b/tests/Light.PortableResults.Tests/CloudEvents/Writing/JsonCloudEventsExtensionsTests.cs index c59f4d1..b457dde 100644 --- a/tests/Light.PortableResults.Tests/CloudEvents/Writing/JsonCloudEventsExtensionsTests.cs +++ b/tests/Light.PortableResults.Tests/CloudEvents/Writing/JsonCloudEventsExtensionsTests.cs @@ -159,31 +159,15 @@ public void WriteCloudEventsExtensionAttributeShouldOmitNullAtomically() [Fact] public void FloatingPointExtensionAttributeWritingShouldAllocateNothingAfterWarmup() { - var output = new ArrayBufferWriter(1024 * 1024); - using var writer = new Utf8JsonWriter(output); var doubleValue = MetadataValue.FromDouble(36_028_797_018_963_968.0); var singleValue = MetadataValue.FromSingle(123_456_789f); - writer.WriteStartObject(); - - for (var index = 0; index < 100; index++) - { - writer.WriteCloudEventsExtensionAttribute("doublevalue", doubleValue); - writer.WriteCloudEventsExtensionAttribute("singlevalue", singleValue); - } - var before = GC.GetAllocatedBytesForCurrentThread(); - for (var index = 0; index < 1_000; index++) - { - writer.WriteCloudEventsExtensionAttribute("doublevalue", doubleValue); - writer.WriteCloudEventsExtensionAttribute("singlevalue", singleValue); - } - - var after = GC.GetAllocatedBytesForCurrentThread(); - after.Should().Be(before); + MeasureMinimumWriterAllocations(doubleValue).Should().Be(0); + MeasureMinimumWriterAllocations(singleValue).Should().Be(0); } [Fact] - public void OtherStringMappedKindsShouldNotAllocateMoreThanCanonicalFormatting() + public void OtherStringMappedKindsShouldNotAllocateCanonicalText() { var values = new[] { @@ -212,11 +196,8 @@ public void OtherStringMappedKindsShouldNotAllocateMoreThanCanonicalFormatting() var canonicalAllocations = MeasureMinimumCanonicalFormattingAllocations(value); var writerAllocations = MeasureMinimumWriterAllocations(value); - writerAllocations.Should().BeLessThanOrEqualTo( - canonicalAllocations, - "writing {0} should materialize at most its existing canonical string", - value.Kind - ); + canonicalAllocations.Should().Be(0, "formatting {0} is span-based", value.Kind); + writerAllocations.Should().Be(0, "writing {0} should not materialize canonical text", value.Kind); } } diff --git a/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs b/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs new file mode 100644 index 0000000..87a682f --- /dev/null +++ b/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs @@ -0,0 +1,504 @@ +using System; +using System.Collections.Generic; +using System.Globalization; +using System.Runtime.CompilerServices; +using System.Text; +using System.Xml; +using FluentAssertions; +using Light.PortableResults.Metadata; +using Light.PortableResults.Text; +using Xunit; + +namespace Light.PortableResults.Tests.Metadata; + +public sealed class CanonicalTextFormatterTests +{ + private const int AllocationIterations = 1_000; + + public static TheoryData CanonicalValues + { + get + { + var data = new TheoryData + { + { "null", MetadataValue.Null, "null" }, + { "boolean-false", MetadataValue.FromBoolean(false), "false" }, + { "boolean-true", MetadataValue.FromBoolean(true), "true" }, + { "int64-minimum", MetadataValue.FromInt64(long.MinValue), Invariant(long.MinValue) }, + { "int64-maximum", MetadataValue.FromInt64(long.MaxValue), Invariant(long.MaxValue) }, + { "double", MetadataValue.FromDouble(-0.0), "-0.0" }, + { "string-ascii", MetadataValue.FromString("canonical text"), "canonical text" }, + { "string-non-ascii", MetadataValue.FromString("Grüße 日本語 😀"), "Grüße 日本語 😀" }, + { "string-unpaired-high", MetadataValue.FromString("a\uD800b"), "a\uD800b" }, + { "string-unpaired-low", MetadataValue.FromString("a\uDC00b"), "a\uDC00b" }, + { "decimal-scale", MetadataValue.FromDecimal(19.50m), Invariant(19.50m) }, + { "decimal-negative-zero", MetadataValue.FromDecimal(-0.000m), Invariant(-0.000m) }, + { "decimal-minimum", MetadataValue.FromDecimal(decimal.MinValue), Invariant(decimal.MinValue) }, + { "decimal-maximum", MetadataValue.FromDecimal(decimal.MaxValue), Invariant(decimal.MaxValue) }, + { + "decimal-smallest-scaled", + MetadataValue.FromDecimal(0.0000000000000000000000000001m), + Invariant(0.0000000000000000000000000001m) + }, + { "uint64-maximum", MetadataValue.FromUInt64(ulong.MaxValue), Invariant(ulong.MaxValue) }, + { "single", MetadataValue.FromSingle(float.Epsilon), "1E-45" }, + { "char-ascii", MetadataValue.FromChar('x'), "x" }, + { "char-non-ascii", MetadataValue.FromChar('ß'), "ß" }, + { "char-unpaired", MetadataValue.FromChar('\uD800'), "\uD800" } + }; + + var utcMinimum = DateTime.SpecifyKind(DateTime.MinValue, DateTimeKind.Utc); + var utcMaximum = DateTime.SpecifyKind(DateTime.MaxValue, DateTimeKind.Utc); + var unspecified = new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Unspecified).AddTicks(1_230_000); + data.Add("datetime-utc-minimum", MetadataValue.FromDateTime(utcMinimum), Format(utcMinimum)); + data.Add("datetime-utc-maximum", MetadataValue.FromDateTime(utcMaximum), Format(utcMaximum)); + data.Add("datetime-unspecified-fraction", MetadataValue.FromDateTime(unspecified), Format(unspecified)); + + var offsetMinimum = DateTimeOffset.MinValue; + var offsetMaximum = DateTimeOffset.MaxValue; + var offsetFraction = new DateTimeOffset( + 2026, + 7, + 26, + 13, + 45, + 30, + TimeSpan.FromHours(-3.5) + ).AddTicks(1_234_500); + data.Add("datetimeoffset-minimum", MetadataValue.FromDateTimeOffset(offsetMinimum), Format(offsetMinimum)); + data.Add("datetimeoffset-maximum", MetadataValue.FromDateTimeOffset(offsetMaximum), Format(offsetMaximum)); + data.Add( + "datetimeoffset-fraction", + MetadataValue.FromDateTimeOffset(offsetFraction), + Format(offsetFraction) + ); + +#if !TESTING_NETSTANDARD_ASSET + data.Add("date-minimum", MetadataValue.FromDateOnly(DateOnly.MinValue), Format(DateOnly.MinValue)); + data.Add("date-maximum", MetadataValue.FromDateOnly(DateOnly.MaxValue), Format(DateOnly.MaxValue)); + data.Add("time-minimum", MetadataValue.FromTimeOnly(TimeOnly.MinValue), Format(TimeOnly.MinValue)); + data.Add("time-maximum", MetadataValue.FromTimeOnly(TimeOnly.MaxValue), Format(TimeOnly.MaxValue)); +#endif + + var duration = TimeSpan.FromDays(2) + + TimeSpan.FromHours(3) + + TimeSpan.FromMinutes(4) + + TimeSpan.FromSeconds(5) + + TimeSpan.FromTicks(600_000); + data.Add("timespan-zero", MetadataValue.FromTimeSpan(TimeSpan.Zero), XmlConvert.ToString(TimeSpan.Zero)); + data.Add( + "timespan-days-only", + MetadataValue.FromTimeSpan(TimeSpan.FromDays(2)), + XmlConvert.ToString(TimeSpan.FromDays(2)) + ); + data.Add("timespan-components", MetadataValue.FromTimeSpan(duration), XmlConvert.ToString(duration)); + data.Add( + "timespan-minimum", + MetadataValue.FromTimeSpan(TimeSpan.MinValue), + XmlConvert.ToString(TimeSpan.MinValue) + ); + data.Add( + "timespan-maximum", + MetadataValue.FromTimeSpan(TimeSpan.MaxValue), + XmlConvert.ToString(TimeSpan.MaxValue) + ); + + var guid = new Guid("a1b2c3d4-e5f6-7890-abcd-ef1234567890"); + data.Add("guid-empty", MetadataValue.FromGuid(Guid.Empty), Guid.Empty.ToString("D")); + data.Add("guid", MetadataValue.FromGuid(guid), guid.ToString("D")); + data.Add( + "uri", + MetadataValue.FromUri(new Uri("https://example.com/Grüße?q=日本語#😀")), + "https://example.com/Grüße?q=日本語#😀" + ); + + return data; + } + } + + public static TheoryData AllocationValues + { + get + { + var data = new TheoryData + { + MetadataValue.Null, + MetadataValue.FromBoolean(true), + MetadataValue.FromInt64(long.MinValue), + MetadataValue.FromDouble(double.MaxValue), + MetadataValue.FromString("Grüße 日本語 😀"), + MetadataValue.FromString("a\uD800b"), + MetadataValue.FromDecimal(0.0000000000000000000000000001m), + MetadataValue.FromUInt64(ulong.MaxValue), + MetadataValue.FromSingle(float.Epsilon), + MetadataValue.FromChar('\uD800'), + MetadataValue.FromDateTime(DateTime.SpecifyKind(DateTime.MaxValue, DateTimeKind.Utc)), + MetadataValue.FromDateTimeOffset(DateTimeOffset.MaxValue) + }; +#if !TESTING_NETSTANDARD_ASSET + data.Add(MetadataValue.FromDateOnly(DateOnly.MaxValue)); + data.Add(MetadataValue.FromTimeOnly(TimeOnly.MaxValue)); +#endif + data.Add(MetadataValue.FromTimeSpan(TimeSpan.MinValue)); + data.Add(MetadataValue.FromGuid(Guid.Empty)); + data.Add(MetadataValue.FromUri(new Uri("https://example.com/Grüße?q=日本語"))); + + return data; + } + } + + public static TheoryData ValidatorValues + { + get + { + var data = new TheoryData + { + { 0, Invariant(long.MinValue) }, + { 1, Invariant(ulong.MaxValue) }, + { 2, "0.1" }, + { 3, "2026-07-26T13:45:30.123Z" }, + { 4, "2026-07-26T13:45:30.123+02:00" } + }; +#if !TESTING_NETSTANDARD_ASSET + data.Add(5, "2026-07-26"); + data.Add(6, "13:45:30.123"); +#endif + data.Add(7, "P2DT3H4M5.06S"); + data.Add(8, "a1b2c3d4-e5f6-7890-abcd-ef1234567890"); + + return data; + } + } + + [Theory] + [MemberData(nameof(CanonicalValues))] + public void MetadataCanonicalFormattingShouldMatchTheExistingTextInBothEncodings( + string caseName, + MetadataValue value, + string expected + ) + { + var expectedUtf8 = Encoding.UTF8.GetBytes(expected); + var chars = new char[expected.Length]; + var bytes = new byte[expectedUtf8.Length]; + + value.TryFormatCanonical(chars, out var charsWritten).Should().BeTrue(caseName); + value.TryFormatCanonicalUtf8(bytes, out var bytesWritten).Should().BeTrue(caseName); + + charsWritten.Should().Be(expected.Length, caseName); + bytesWritten.Should().Be(expectedUtf8.Length, caseName); + chars.AsSpan().SequenceEqual(expected.AsSpan()).Should().BeTrue(caseName); + bytes.Should().Equal(expectedUtf8, caseName); + value.ToCanonicalString().Should().Be(expected, caseName); + if (IsAscii(expected)) + { + bytesWritten.Should().Be(charsWritten, caseName); + } + } + + [Theory] + [MemberData(nameof(CanonicalValues))] + public void MetadataCanonicalFormattingShouldBeAtomicWhenCapacityIsOneShort( + string caseName, + MetadataValue value, + string expected + ) + { + var expectedUtf8Length = Encoding.UTF8.GetByteCount(expected); + var chars = CreateFilledArray(Math.Max(expected.Length, 1), '\uA55A'); + var bytes = CreateFilledArray(Math.Max(expectedUtf8Length, 1), (byte) 0xA5); + + value.TryFormatCanonical(chars.AsSpan(0, expected.Length - 1), out var charsWritten) + .Should() + .BeFalse(caseName); + value.TryFormatCanonicalUtf8(bytes.AsSpan(0, expectedUtf8Length - 1), out var bytesWritten) + .Should() + .BeFalse(caseName); + + charsWritten.Should().Be(0, caseName); + bytesWritten.Should().Be(0, caseName); + chars.Should().OnlyContain(character => character == '\uA55A', caseName); + bytes.Should().OnlyContain(value => value == 0xA5, caseName); + } + + [Theory] + [MemberData(nameof(AllocationValues))] + public void MetadataCanonicalSpanFormattingShouldNotAllocate(MetadataValue value) + { + Span chars = stackalloc char[128]; + Span bytes = stackalloc byte[256]; + for (var index = 0; index < 100; index++) + { + value.TryFormatCanonical(chars, out _); + value.TryFormatCanonicalUtf8(bytes, out _); + } + + var minimumAllocations = long.MaxValue; + for (var sample = 0; sample < 5; sample++) + { + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var index = 0; index < AllocationIterations; index++) + { + value.TryFormatCanonical(chars, out _); + value.TryFormatCanonicalUtf8(bytes, out _); + } + + minimumAllocations = Math.Min( + minimumAllocations, + GC.GetAllocatedBytesForCurrentThread() - before + ); + } + + minimumAllocations.Should().Be(0, value.Kind.ToString()); + } + + [Theory] + [MemberData(nameof(ValidatorValues))] + public void CanonicalRoundTripValidatorsShouldNotAllocateCandidateText(int validator, string text) + { + var value = MetadataValue.FromString(text); + for (var index = 0; index < 100; index++) + { + Validate(value, validator).Should().BeTrue(); + } + + var minimumAllocations = long.MaxValue; + var allValid = true; + for (var sample = 0; sample < 5; sample++) + { + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var index = 0; index < AllocationIterations; index++) + { + allValid &= Validate(value, validator); + } + + minimumAllocations = Math.Min( + minimumAllocations, + GC.GetAllocatedBytesForCurrentThread() - before + ); + } + + allValid.Should().BeTrue(); + minimumAllocations.Should().Be(0); + } + + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(2)] + [InlineData(3)] + public void ExistingAllocationFreeCanonicalStringsShouldReuseTheirText(int caseIndex) + { + var value = caseIndex switch + { + 0 => MetadataValue.Null, + 1 => MetadataValue.FromBoolean(true), + 2 => MetadataValue.FromString("stored string"), + _ => MetadataValue.FromUri(new Uri("https://example.com/stored")) + }; + var expected = value.ToCanonicalString(); + + for (var index = 0; index < 100; index++) + { + _ = value.ToCanonicalString(); + } + + var before = GC.GetAllocatedBytesForCurrentThread(); + var allReferencesMatch = true; + for (var index = 0; index < AllocationIterations; index++) + { + allReferencesMatch &= ReferenceEquals(value.ToCanonicalString(), expected); + } + + var after = GC.GetAllocatedBytesForCurrentThread(); + allReferencesMatch.Should().BeTrue(); + after.Should().Be(before); + } + + [Fact] + public void CanonicalTextFormatterShouldResolveACharLiteralToTheCharOverload() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumCharLength]; + + CanonicalTextFormatter.TryFormat('x', destination, out var charsWritten).Should().BeTrue(); + + destination.Slice(0, charsWritten).ToString().Should().Be("x"); + + CanonicalTextFormatter.TryFormat(42, destination, out charsWritten).Should().BeTrue(); + destination.Slice(0, charsWritten).ToString().Should().Be("42"); + } + + [Fact] + public void CanonicalCodeUnitShouldConvertAsciiToBothSupportedCodeUnits() + { + const byte value = (byte) 'x'; + + CanonicalCodeUnit.FromAscii(value).Should().Be(value); + CanonicalCodeUnit.FromAscii(value).Should().Be('x'); + } + + [Fact] + public void CanonicalCodeUnitShouldRejectUnsupportedCodeUnitTypes() + { + var act = () => CanonicalCodeUnit.FromAscii((byte) 'x'); + + act.Should().Throw(); + } + + [Fact] + public void CanonicalTextFormatterShouldNormalizeOnlyLocalDateTimes() + { + var local = new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Local).AddTicks(1_230_000); + var unspecified = DateTime.SpecifyKind(local, DateTimeKind.Unspecified); + Span localText = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + Span unspecifiedText = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + + CanonicalTextFormatter.TryFormat(local, localText, out var localLength).Should().BeTrue(); + CanonicalTextFormatter.TryFormat(unspecified, unspecifiedText, out var unspecifiedLength).Should().BeTrue(); + + localText.Slice(0, localLength).ToString().Should().Be(Format(local.ToUniversalTime())); + localLength.Should().BeLessThanOrEqualTo(CanonicalTextFormatter.MaximumDateTimeLength); + unspecifiedText.Slice(0, unspecifiedLength).ToString().Should().Be(Format(unspecified)); + } + + [Theory] + [InlineData(-1)] + [InlineData(CanonicalTextFormatter.MaximumDayNumber + 1)] + public void CanonicalDateFormatterShouldRejectInvalidDayNumbers(int dayNumber) + { + var chars = CreateFilledArray(CanonicalTextFormatter.MaximumDateLength, '\uA55A'); + var bytes = CreateFilledArray(CanonicalTextFormatter.MaximumDateLength, (byte) 0xA5); + + var charAct = () => CanonicalTextFormatter.TryFormatDate(dayNumber, chars, out _); + var byteAct = () => CanonicalTextFormatter.TryFormatDateUtf8(dayNumber, bytes, out _); + + charAct.Should().Throw(); + byteAct.Should().Throw(); + chars.Should().OnlyContain(value => value == '\uA55A'); + bytes.Should().OnlyContain(value => value == 0xA5); + } + + [Theory] + [InlineData(-1L)] + [InlineData(CanonicalTextFormatter.MaximumTimeOfDayTicks + 1)] + public void CanonicalTimeFormatterShouldRejectInvalidTicks(long ticks) + { + var chars = CreateFilledArray(CanonicalTextFormatter.MaximumTimeLength, '\uA55A'); + var bytes = CreateFilledArray(CanonicalTextFormatter.MaximumTimeLength, (byte) 0xA5); + + var charAct = () => CanonicalTextFormatter.TryFormatTime(ticks, chars, out _); + var byteAct = () => CanonicalTextFormatter.TryFormatTimeUtf8(ticks, bytes, out _); + + charAct.Should().Throw(); + byteAct.Should().Throw(); + chars.Should().OnlyContain(value => value == '\uA55A'); + bytes.Should().OnlyContain(value => value == 0xA5); + } + + [Fact] + public void ComplexAndMalformedMetadataShouldThrowInBothEncodings() + { + var values = new List + { + MetadataValue.FromArray(MetadataArray.Empty), + MetadataValue.FromObject(MetadataObject.Empty), + MetadataValueTestFactory.CreateWithInt64Payload(MetadataKind.DateTime, long.MaxValue), + MetadataValueTestFactory.CreateWithInt64Payload(MetadataKind.DateTime, -1L), + MetadataValueTestFactory.CreateWithInt64Payload(MetadataKind.DateOnly, long.MaxValue), + MetadataValueTestFactory.CreateWithInt64Payload(MetadataKind.TimeOnly, long.MaxValue) + }; + + foreach (var value in values) + { + var chars = new char[MetadataValue.MaximumPrimitiveCanonicalLength]; + var bytes = new byte[MetadataValue.MaximumPrimitiveCanonicalLength]; + var charAct = () => value.TryFormatCanonical(chars, out _); + var byteAct = () => value.TryFormatCanonicalUtf8(bytes, out _); + + charAct.Should().Throw(value.Kind.ToString()); + byteAct.Should().Throw(value.Kind.ToString()); + } + } + + [Fact] + public void LegacyDecimalFieldOrderShouldMatchTheContractualBits() + { + var value = new decimal( + lo: 0x11111111, + mid: 0x22222222, + hi: 0x33333333, + isNegative: true, + scale: 5 + ); + var expected = decimal.GetBits(value); + ref var layout = ref Unsafe.As(ref value); + + new[] { layout.Low, layout.Middle, layout.High, layout.Flags } + .Should() + .Equal((uint) expected[0], (uint) expected[1], (uint) expected[2], (uint) expected[3]); + } + + private static bool IsAscii(string text) + { + foreach (var character in text) + { + if (character > 0x7F) + { + return false; + } + } + + return true; + } + + private static bool Validate(MetadataValue value, int validator) => + validator switch + { + 0 => value.TryGetInt64(out _), + 1 => value.TryGetUInt64(out _), + 2 => value.TryGetSingle(out _), + 3 => value.TryGetDateTime(out _), + 4 => value.TryGetDateTimeOffset(out _), +#if !TESTING_NETSTANDARD_ASSET + 5 => value.TryGetDateOnly(out _), + 6 => value.TryGetTimeOnly(out _), +#endif + 7 => value.TryGetTimeSpan(out _), + _ => value.TryGetGuid(out _) + }; + + private static string Invariant(T value) + where T : IFormattable => + value.ToString(null, CultureInfo.InvariantCulture); + + private static string Format(DateTime value) => + value.ToString("yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK", CultureInfo.InvariantCulture); + + private static string Format(DateTimeOffset value) => + value.ToString("yyyy-MM-dd'T'HH:mm:ss.FFFFFFFzzz", CultureInfo.InvariantCulture); + + private static T[] CreateFilledArray(int length, T value) + { + var array = new T[length]; + Array.Fill(array, value); + return array; + } + +#pragma warning disable CS0649 // Fields are populated by reinterpreting decimal storage via Unsafe.As. + private struct DecimalLayout + { + public uint Flags; + public uint High; + public uint Low; + public uint Middle; + } +#pragma warning restore CS0649 + +#if !TESTING_NETSTANDARD_ASSET + private static string Format(DateOnly value) => + value.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture); + + private static string Format(TimeOnly value) => + value.ToString("HH:mm:ss.FFFFFFF", CultureInfo.InvariantCulture); +#endif +} diff --git a/tests/Light.PortableResults.Tests/Numbers/CanonicalFloatingPointFormatterTests.cs b/tests/Light.PortableResults.Tests/Numbers/CanonicalTextFormatterFloatingPointTests.cs similarity index 80% rename from tests/Light.PortableResults.Tests/Numbers/CanonicalFloatingPointFormatterTests.cs rename to tests/Light.PortableResults.Tests/Numbers/CanonicalTextFormatterFloatingPointTests.cs index 3551928..c055e46 100644 --- a/tests/Light.PortableResults.Tests/Numbers/CanonicalFloatingPointFormatterTests.cs +++ b/tests/Light.PortableResults.Tests/Numbers/CanonicalTextFormatterFloatingPointTests.cs @@ -6,12 +6,12 @@ using System.Text; using FluentAssertions; using Light.PortableResults.Metadata; -using Light.PortableResults.Numbers; +using Light.PortableResults.Text; using Xunit; namespace Light.PortableResults.Tests.Numbers; -public sealed class CanonicalFloatingPointFormatterTests +public sealed class CanonicalTextFormatterFloatingPointTests { private const int CorpusSize = 50_000; private const int CorpusSeed = 0x5EED_0058; @@ -31,7 +31,7 @@ public sealed class CanonicalFloatingPointFormatterTests CreateSingleDragon4Formatter(); public static TheoryData DoubleScenarios => - new () + new() { { 0x3F1A36E2EB1C432DUL, "0.0001" }, { 0x3EE4F8B588E368F1UL, "1E-05" }, @@ -63,7 +63,7 @@ public sealed class CanonicalFloatingPointFormatterTests }; public static TheoryData SingleScenarios => - new () + new() { { 0x38D1B717U, "0.0001" }, { 0x3727C5ACU, "1E-05" }, @@ -113,7 +113,7 @@ public void DoubleNamedScenariosShouldUseTheCanonicalEncoding(ulong bits, string { var value = BitConverter.Int64BitsToDouble((long) bits); - CanonicalFloatingPointFormatter.Format(value).Should().Be(expected); + CanonicalTextFormatter.Format(value).Should().Be(expected); Format(value).Should().Be(expected); FormatWithDragon4(value).Should().Be(expected); AssertUtf8Matches(value, expected, forceDragon4: false); @@ -127,7 +127,7 @@ public void SingleNamedScenariosShouldUseTheCanonicalEncoding(uint bits, string { var value = BitConverter.Int32BitsToSingle((int) bits); - CanonicalFloatingPointFormatter.Format(value).Should().Be(expected); + CanonicalTextFormatter.Format(value).Should().Be(expected); Format(value).Should().Be(expected); FormatWithDragon4(value).Should().Be(expected); AssertUtf8Matches(value, expected, forceDragon4: false); @@ -140,17 +140,17 @@ public void NonFiniteDoubleValuesShouldBeRejectedByEveryOverload() { foreach (var value in new[] { double.NaN, double.PositiveInfinity, double.NegativeInfinity }) { - var format = () => CanonicalFloatingPointFormatter.Format(value); + var format = () => CanonicalTextFormatter.Format(value); var tryFormat = () => - CanonicalFloatingPointFormatter.TryFormat( + CanonicalTextFormatter.TryFormat( value, - new char[CanonicalFloatingPointFormatter.MaximumDoubleLength], + new char[CanonicalTextFormatter.MaximumDoubleLength], out _ ); var tryFormatUtf8 = () => - CanonicalFloatingPointFormatter.TryFormatUtf8( + CanonicalTextFormatter.TryFormatUtf8( value, - new byte[CanonicalFloatingPointFormatter.MaximumDoubleLength], + new byte[CanonicalTextFormatter.MaximumDoubleLength], out _ ); @@ -165,17 +165,17 @@ public void NonFiniteSingleValuesShouldBeRejectedByEveryOverload() { foreach (var value in new[] { float.NaN, float.PositiveInfinity, float.NegativeInfinity }) { - var format = () => CanonicalFloatingPointFormatter.Format(value); + var format = () => CanonicalTextFormatter.Format(value); var tryFormat = () => - CanonicalFloatingPointFormatter.TryFormat( + CanonicalTextFormatter.TryFormat( value, - new char[CanonicalFloatingPointFormatter.MaximumSingleLength], + new char[CanonicalTextFormatter.MaximumSingleLength], out _ ); var tryFormatUtf8 = () => - CanonicalFloatingPointFormatter.TryFormatUtf8( + CanonicalTextFormatter.TryFormatUtf8( value, - new byte[CanonicalFloatingPointFormatter.MaximumSingleLength], + new byte[CanonicalTextFormatter.MaximumSingleLength], out _ ); @@ -197,28 +197,28 @@ public void InsufficientDestinationsShouldRemainUnmodified() doubleUtf8Destination.Fill(0xAA); singleUtf8Destination.Fill(0xBB); - CanonicalFloatingPointFormatter.TryFormat( + CanonicalTextFormatter.TryFormat( double.MaxValue, doubleDestination, out var doubleCharsWritten ) .Should() .BeFalse(); - CanonicalFloatingPointFormatter.TryFormat( + CanonicalTextFormatter.TryFormat( float.MaxValue, singleDestination, out var singleCharsWritten ) .Should() .BeFalse(); - CanonicalFloatingPointFormatter.TryFormatUtf8( + CanonicalTextFormatter.TryFormatUtf8( double.MaxValue, doubleUtf8Destination, out var doubleBytesWritten ) .Should() .BeFalse(); - CanonicalFloatingPointFormatter.TryFormatUtf8( + CanonicalTextFormatter.TryFormatUtf8( float.MaxValue, singleUtf8Destination, out var singleBytesWritten @@ -255,7 +255,7 @@ public void RandomFiniteBitPatternsShouldMatchTheRuntimeOracle() var value = BitConverter.Int64BitsToDouble((long) bits); var expected = CanonicalizeRuntimeText(value.ToString("R", CultureInfo.InvariantCulture)); - CanonicalFloatingPointFormatter.Format(value) + CanonicalTextFormatter.Format(value) .Should() .Be(expected, "binary64 bits 0x{0:X16} must be deterministic", bits); FormatWithDragon4(value) @@ -277,7 +277,7 @@ public void RandomFiniteBitPatternsShouldMatchTheRuntimeOracle() var value = BitConverter.Int32BitsToSingle((int) bits); var expected = CanonicalizeRuntimeText(value.ToString("R", CultureInfo.InvariantCulture)); - CanonicalFloatingPointFormatter.Format(value) + CanonicalTextFormatter.Format(value) .Should() .Be(expected, "binary32 bits 0x{0:X8} must be deterministic", bits); FormatWithDragon4(value) @@ -316,15 +316,15 @@ public void FloatingPointSpanFormattingShouldAllocateNothingAfterWarmup() const float singleValue = 123_456_789f; var doubleMetadata = MetadataValue.FromDouble(doubleValue); var singleMetadata = MetadataValue.FromSingle(singleValue); - Span charDestination = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; - Span byteDestination = stackalloc byte[CanonicalFloatingPointFormatter.MaximumDoubleLength]; + Span charDestination = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; + Span byteDestination = stackalloc byte[CanonicalTextFormatter.MaximumDoubleLength]; for (var index = 0; index < 100; index++) { - CanonicalFloatingPointFormatter.TryFormat(doubleValue, charDestination, out _); - CanonicalFloatingPointFormatter.TryFormat(singleValue, charDestination, out _); - CanonicalFloatingPointFormatter.TryFormatUtf8(doubleValue, byteDestination, out _); - CanonicalFloatingPointFormatter.TryFormatUtf8(singleValue, byteDestination, out _); + CanonicalTextFormatter.TryFormat(doubleValue, charDestination, out _); + CanonicalTextFormatter.TryFormat(singleValue, charDestination, out _); + CanonicalTextFormatter.TryFormatUtf8(doubleValue, byteDestination, out _); + CanonicalTextFormatter.TryFormatUtf8(singleValue, byteDestination, out _); ForceDragon4DoubleChars(doubleValue, charDestination, out _, true); ForceDragon4SingleChars(singleValue, charDestination, out _, true); ForceDragon4DoubleBytes(doubleValue, byteDestination, out _, true); @@ -336,10 +336,10 @@ public void FloatingPointSpanFormattingShouldAllocateNothingAfterWarmup() var before = GC.GetAllocatedBytesForCurrentThread(); for (var index = 0; index < 1_000; index++) { - CanonicalFloatingPointFormatter.TryFormat(doubleValue, charDestination, out _); - CanonicalFloatingPointFormatter.TryFormat(singleValue, charDestination, out _); - CanonicalFloatingPointFormatter.TryFormatUtf8(doubleValue, byteDestination, out _); - CanonicalFloatingPointFormatter.TryFormatUtf8(singleValue, byteDestination, out _); + CanonicalTextFormatter.TryFormat(doubleValue, charDestination, out _); + CanonicalTextFormatter.TryFormat(singleValue, charDestination, out _); + CanonicalTextFormatter.TryFormatUtf8(doubleValue, byteDestination, out _); + CanonicalTextFormatter.TryFormatUtf8(singleValue, byteDestination, out _); ForceDragon4DoubleChars(doubleValue, charDestination, out _, true); ForceDragon4SingleChars(singleValue, charDestination, out _, true); ForceDragon4DoubleBytes(doubleValue, byteDestination, out _, true); @@ -356,15 +356,15 @@ public void StringFormattingShouldAllocateOnlyTheReturnedStrings() { const double doubleValue = 36_028_797_018_963_968.0; const float singleValue = 123_456_789f; - var doubleText = CanonicalFloatingPointFormatter.Format(doubleValue); - var singleText = CanonicalFloatingPointFormatter.Format(singleValue); + var doubleText = CanonicalTextFormatter.Format(doubleValue); + var singleText = CanonicalTextFormatter.Format(singleValue); var doubleMetadata = MetadataValue.FromDouble(doubleValue); var singleMetadata = MetadataValue.FromSingle(singleValue); for (var index = 0; index < 100; index++) { - _ = CanonicalFloatingPointFormatter.Format(doubleValue); - _ = CanonicalFloatingPointFormatter.Format(singleValue); + _ = CanonicalTextFormatter.Format(doubleValue); + _ = CanonicalTextFormatter.Format(singleValue); _ = doubleMetadata.ToCanonicalString(); _ = singleMetadata.ToCanonicalString(); } @@ -372,13 +372,13 @@ public void StringFormattingShouldAllocateOnlyTheReturnedStrings() var doubleBaseline = MeasureStringAllocations(() => new string(doubleText.AsSpan())); var singleBaseline = MeasureStringAllocations(() => new string(singleText.AsSpan())); - MeasureStringAllocations(() => CanonicalFloatingPointFormatter.Format(doubleValue)) + MeasureStringAllocations(() => CanonicalTextFormatter.Format(doubleValue)) .Should() .Be(doubleBaseline); MeasureStringAllocations(() => doubleMetadata.ToCanonicalString()) .Should() .Be(doubleBaseline); - MeasureStringAllocations(() => CanonicalFloatingPointFormatter.Format(singleValue)) + MeasureStringAllocations(() => CanonicalTextFormatter.Format(singleValue)) .Should() .Be(singleBaseline); MeasureStringAllocations(() => singleMetadata.ToCanonicalString()) @@ -390,13 +390,13 @@ private static void AssertFitsIntoMaximumLength(double value) { // The destinations are sized exactly at the constant, so a successful call is itself the // proof that the constant bounds this value in this encoding. - Span chars = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; - Span bytes = stackalloc byte[CanonicalFloatingPointFormatter.MaximumDoubleLength]; + Span chars = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; + Span bytes = stackalloc byte[CanonicalTextFormatter.MaximumDoubleLength]; - CanonicalFloatingPointFormatter.TryFormat(value, chars, out var charsWritten) + CanonicalTextFormatter.TryFormat(value, chars, out var charsWritten) .Should() .BeTrue(); - CanonicalFloatingPointFormatter.TryFormatUtf8(value, bytes, out var bytesWritten) + CanonicalTextFormatter.TryFormatUtf8(value, bytes, out var bytesWritten) .Should() .BeTrue(); ForceDragon4DoubleChars(value, chars, out var dragon4CharsWritten, true).Should().BeTrue(); @@ -409,13 +409,13 @@ private static void AssertFitsIntoMaximumLength(double value) private static void AssertFitsIntoMaximumLength(float value) { - Span chars = stackalloc char[CanonicalFloatingPointFormatter.MaximumSingleLength]; - Span bytes = stackalloc byte[CanonicalFloatingPointFormatter.MaximumSingleLength]; + Span chars = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; + Span bytes = stackalloc byte[CanonicalTextFormatter.MaximumSingleLength]; - CanonicalFloatingPointFormatter.TryFormat(value, chars, out var charsWritten) + CanonicalTextFormatter.TryFormat(value, chars, out var charsWritten) .Should() .BeTrue(); - CanonicalFloatingPointFormatter.TryFormatUtf8(value, bytes, out var bytesWritten) + CanonicalTextFormatter.TryFormatUtf8(value, bytes, out var bytesWritten) .Should() .BeTrue(); ForceDragon4SingleChars(value, chars, out var dragon4CharsWritten, true).Should().BeTrue(); @@ -429,7 +429,7 @@ private static void AssertFitsIntoMaximumLength(float value) private static void AssertMatchesOracle(double value) { var expected = CanonicalizeRuntimeText(value.ToString("R", CultureInfo.InvariantCulture)); - CanonicalFloatingPointFormatter.Format(value).Should().Be(expected); + CanonicalTextFormatter.Format(value).Should().Be(expected); FormatWithDragon4(value).Should().Be(expected); AssertUtf8Matches(value, expected, forceDragon4: false); AssertUtf8Matches(value, expected, forceDragon4: true); @@ -438,7 +438,7 @@ private static void AssertMatchesOracle(double value) private static void AssertMatchesOracle(float value) { var expected = CanonicalizeRuntimeText(value.ToString("R", CultureInfo.InvariantCulture)); - CanonicalFloatingPointFormatter.Format(value).Should().Be(expected); + CanonicalTextFormatter.Format(value).Should().Be(expected); FormatWithDragon4(value).Should().Be(expected); AssertUtf8Matches(value, expected, forceDragon4: false); AssertUtf8Matches(value, expected, forceDragon4: true); @@ -449,8 +449,8 @@ private static string CanonicalizeRuntimeText(string value) => private static string Format(double value) { - Span destination = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; - CanonicalFloatingPointFormatter.TryFormat(value, destination, out var charsWritten) + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) .Should() .BeTrue(); return new string(destination[..charsWritten]); @@ -458,8 +458,8 @@ private static string Format(double value) private static string Format(float value) { - Span destination = stackalloc char[CanonicalFloatingPointFormatter.MaximumSingleLength]; - CanonicalFloatingPointFormatter.TryFormat(value, destination, out var charsWritten) + Span destination = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) .Should() .BeTrue(); return new string(destination[..charsWritten]); @@ -467,30 +467,30 @@ private static string Format(float value) private static string FormatWithDragon4(double value) { - Span destination = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; ForceDragon4DoubleChars(value, destination, out var charsWritten, true).Should().BeTrue(); return new string(destination[..charsWritten]); } private static string FormatWithDragon4(float value) { - Span destination = stackalloc char[CanonicalFloatingPointFormatter.MaximumSingleLength]; + Span destination = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; ForceDragon4SingleChars(value, destination, out var charsWritten, true).Should().BeTrue(); return new string(destination[..charsWritten]); } private static void AssertUtf8Matches(double value, string expected, bool forceDragon4) { - Span chars = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; - Span bytes = stackalloc byte[CanonicalFloatingPointFormatter.MaximumDoubleLength + 1]; + Span chars = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; + Span bytes = stackalloc byte[CanonicalTextFormatter.MaximumDoubleLength + 1]; bytes.Fill(UnwrittenByte); var charsSucceeded = forceDragon4 ? ForceDragon4DoubleChars(value, chars, out var charsWritten, true) : - CanonicalFloatingPointFormatter.TryFormat(value, chars, out charsWritten); + CanonicalTextFormatter.TryFormat(value, chars, out charsWritten); var bytesSucceeded = forceDragon4 ? ForceDragon4DoubleBytes(value, bytes, out var bytesWritten, true) : - CanonicalFloatingPointFormatter.TryFormatUtf8(value, bytes, out bytesWritten); + CanonicalTextFormatter.TryFormatUtf8(value, bytes, out bytesWritten); charsSucceeded.Should().BeTrue(); bytesSucceeded.Should().BeTrue(); @@ -502,16 +502,16 @@ private static void AssertUtf8Matches(double value, string expected, bool forceD private static void AssertUtf8Matches(float value, string expected, bool forceDragon4) { - Span chars = stackalloc char[CanonicalFloatingPointFormatter.MaximumSingleLength]; - Span bytes = stackalloc byte[CanonicalFloatingPointFormatter.MaximumSingleLength + 1]; + Span chars = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; + Span bytes = stackalloc byte[CanonicalTextFormatter.MaximumSingleLength + 1]; bytes.Fill(UnwrittenByte); var charsSucceeded = forceDragon4 ? ForceDragon4SingleChars(value, chars, out var charsWritten, true) : - CanonicalFloatingPointFormatter.TryFormat(value, chars, out charsWritten); + CanonicalTextFormatter.TryFormat(value, chars, out charsWritten); var bytesSucceeded = forceDragon4 ? ForceDragon4SingleBytes(value, bytes, out var bytesWritten, true) : - CanonicalFloatingPointFormatter.TryFormatUtf8(value, bytes, out bytesWritten); + CanonicalTextFormatter.TryFormatUtf8(value, bytes, out bytesWritten); charsSucceeded.Should().BeTrue(); bytesSucceeded.Should().BeTrue(); @@ -536,7 +536,7 @@ private static TFormatter CreateSingleDragon4Formatter() .CreateDelegate(typeof(TFormatter)); private static MethodInfo GetTryFormatCore(Type numberType) => - typeof(CanonicalFloatingPointFormatter) + typeof(CanonicalTextFormatter) .GetMethods(BindingFlags.NonPublic | BindingFlags.Static) .Single( method => diff --git a/tests/Light.PortableResults.Tests/SharedJsonSerialization/Writing/SharedWritingExtensionsTests.cs b/tests/Light.PortableResults.Tests/SharedJsonSerialization/Writing/SharedWritingExtensionsTests.cs index 2184864..a63bdc7 100644 --- a/tests/Light.PortableResults.Tests/SharedJsonSerialization/Writing/SharedWritingExtensionsTests.cs +++ b/tests/Light.PortableResults.Tests/SharedJsonSerialization/Writing/SharedWritingExtensionsTests.cs @@ -1,7 +1,9 @@ using System; +using System.Buffers; using System.Globalization; using System.IO; using System.Text; +using System.Text.Encodings.Web; using System.Text.Json; using System.Text.Json.Serialization.Metadata; using FluentAssertions; @@ -13,6 +15,8 @@ namespace Light.PortableResults.Tests.SharedJsonSerialization.Writing; public sealed class SharedWritingExtensionsTests { + private const int AllocationIterations = 1_000; + [Fact] public void WriteMetadataValue_ShouldThrow_WhenWriterIsNull() { @@ -162,6 +166,63 @@ public void WriteMetadataArray_ShouldWriteDecimalElementsAsUnquotedNumbers() json.Should().Be("[9.99,99.90]"); } + [Theory] + [InlineData(false)] + [InlineData(true)] + public void TextBearingMetadataShouldPreserveTheWriterUtf16RouteBytes(bool useUnsafeEncoder) + { + var values = new (MetadataValue Value, string DefaultJson, string UnsafeJson)[] + { + (MetadataValue.FromString("Grüße"), "\"Gr\\u00FC\\u00DFe\"", "\"Grüße\""), + (MetadataValue.FromString("a\uD800b"), "\"a\\uFFFDb\"", "\"a\\uFFFDb\""), + (MetadataValue.FromChar('\uD800'), "\"\\uFFFD\"", "\"\\uFFFD\"") + }; + var options = new JsonWriterOptions + { + Encoder = useUnsafeEncoder ? JavaScriptEncoder.UnsafeRelaxedJsonEscaping : null + }; + + foreach (var (value, defaultJson, unsafeJson) in values) + { + var actual = SerializeToUtf8( + writer => writer.WriteMetadataValue( + value, + MetadataValueAnnotation.SerializeInHttpResponseBody + ), + options + ); + + actual.Should().Equal( + Encoding.UTF8.GetBytes(useUnsafeEncoder ? unsafeJson : defaultJson), + value.Kind.ToString() + ); + } + } + + [Fact] + public void CanonicallyFormattedJsonValuesShouldNotAllocateStrings() + { + var values = new[] + { + MetadataValue.FromDouble(36_028_797_018_963_968.0), + MetadataValue.FromSingle(123_456_789f), + MetadataValue.FromUInt64(ulong.MaxValue), + MetadataValue.FromDateTime( + new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Utc).AddTicks(1_234_567) + ), + MetadataValue.FromTimeSpan(TimeSpan.MaxValue), + MetadataValue.FromGuid(new Guid("a1b2c3d4-e5f6-7890-abcd-ef1234567890")), + MetadataValue.FromString("Grüße 日本語 😀"), + MetadataValue.FromChar('ß'), + MetadataValue.FromUri(new Uri("https://example.com/Grüße?q=日本語")) + }; + + foreach (var value in values) + { + MeasureWriterAllocations(value).Should().Be(0, value.Kind.ToString()); + } + } + [Fact] public void WriteRichErrors_ShouldThrow_WhenWriterIsNull() { @@ -229,4 +290,44 @@ private static string Serialize(Action writeAction) return Encoding.UTF8.GetString(stream.ToArray()); } + + private static byte[] SerializeToUtf8( + Action writeAction, + JsonWriterOptions options + ) + { + var output = new ArrayBufferWriter(); + using var writer = new Utf8JsonWriter(output, options); + writeAction(writer); + writer.Flush(); + return output.WrittenSpan.ToArray(); + } + + private static long MeasureWriterAllocations(MetadataValue value) + { + var minimumAllocations = long.MaxValue; + for (var sample = 0; sample < 5; sample++) + { + var output = new ArrayBufferWriter(1024 * 1024); + using var writer = new Utf8JsonWriter(output); + writer.WriteStartArray(); + for (var index = 0; index < 100; index++) + { + writer.WriteMetadataValue(value, MetadataValueAnnotation.SerializeInHttpResponseBody); + } + + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var index = 0; index < AllocationIterations; index++) + { + writer.WriteMetadataValue(value, MetadataValueAnnotation.SerializeInHttpResponseBody); + } + + minimumAllocations = Math.Min( + minimumAllocations, + GC.GetAllocatedBytesForCurrentThread() - before + ); + } + + return minimumAllocations; + } } From 22a9b51e1988bd456be374d5fa3425027ba9bd26 Mon Sep 17 00:00:00 2001 From: Kenny Pflug Date: Sun, 2 Aug 2026 13:38:03 +0200 Subject: [PATCH 6/7] fix(metadata): harden canonical formatting contracts Handle empty UTF-16 input consistently across package assets and keep UTF-8 capacity checks atomic for every representable span. Assert fixed-buffer invariants, consolidate unreachable failure paths, and extend the shared canonical corpus with empty text and negative scaled decimals. --- .../Metadata/MetadataValue.cs | 16 ++++++++-------- .../Writing/MetadataExtensions.cs | 7 +++++-- .../Text/CanonicalTextFormatter.cs | 18 +++++++++++------- .../Metadata/CanonicalTextFormatterTests.cs | 19 +++++++++++++++++++ 4 files changed, 43 insertions(+), 17 deletions(-) diff --git a/src/Light.PortableResults/Metadata/MetadataValue.cs b/src/Light.PortableResults/Metadata/MetadataValue.cs index 30b759b..6a5576f 100644 --- a/src/Light.PortableResults/Metadata/MetadataValue.cs +++ b/src/Light.PortableResults/Metadata/MetadataValue.cs @@ -1,4 +1,5 @@ using System; +using System.Diagnostics; using System.Globalization; using System.Xml; using Light.PortableResults.Text; @@ -909,9 +910,8 @@ out charsWritten ); case MetadataKind.Array: case MetadataKind.Object: - return ThrowComplexCanonicalFormat(Kind, out charsWritten); default: - throw new InvalidOperationException($"Kind '{Kind}' does not have a canonical text encoding."); + return ThrowComplexCanonicalFormat(Kind, out charsWritten); } } @@ -1003,9 +1003,8 @@ out bytesWritten ); case MetadataKind.Array: case MetadataKind.Object: - return ThrowComplexCanonicalFormat(Kind, out bytesWritten); default: - throw new InvalidOperationException($"Kind '{Kind}' does not have a canonical text encoding."); + return ThrowComplexCanonicalFormat(Kind, out bytesWritten); } } @@ -1166,7 +1165,8 @@ private static void ValidateFinite(double value, string parameterName) private static string FormatBoundedCanonical(MetadataValue value) { Span destination = stackalloc char[MaximumPrimitiveCanonicalLength]; - value.TryFormatCanonical(destination, out var charsWritten); + var formatted = value.TryFormatCanonical(destination, out var charsWritten); + Debug.Assert(formatted); return destination.Slice(0, charsWritten).ToString(); } @@ -1369,12 +1369,12 @@ out int bytesWritten return CanonicalTextFormatter.TryFormatTimeUtf8(ticks, destination, out bytesWritten); } - private static bool ThrowComplexCanonicalFormat( + private static bool ThrowComplexCanonicalFormat( MetadataKind kind, - out TCodeUnit unitsWritten + out int unitsWritten ) { - unitsWritten = default!; + unitsWritten = 0; throw new InvalidOperationException($"Kind '{kind}' does not have a primitive canonical text encoding."); } diff --git a/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs b/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs index d31159d..7cdd557 100644 --- a/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs +++ b/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs @@ -1,4 +1,5 @@ using System; +using System.Diagnostics; using System.Text.Json; using Light.PortableResults.Metadata; @@ -99,7 +100,8 @@ private static void WriteNumberValue(Utf8JsonWriter writer, MetadataValue value) // skipped because the text comes from our own formatter and non-finite values are rejected at // construction, so it is always a well-formed JSON number. Span canonicalNumber = stackalloc byte[MetadataValue.MaximumPrimitiveCanonicalLength]; - value.TryFormatCanonicalUtf8(canonicalNumber, out var bytesWritten); + var formatted = value.TryFormatCanonicalUtf8(canonicalNumber, out var bytesWritten); + Debug.Assert(formatted); writer.WriteRawValue(canonicalNumber.Slice(0, bytesWritten), skipInputValidation: true); return; case MetadataNumberEncoding.None: @@ -128,7 +130,8 @@ private static void WriteStringValue(Utf8JsonWriter writer, MetadataValue value) default: // Every remaining string-shaped kind has bounded, entirely ASCII canonical text. Span canonicalText = stackalloc byte[MetadataValue.MaximumPrimitiveCanonicalLength]; - value.TryFormatCanonicalUtf8(canonicalText, out var bytesWritten); + var formatted = value.TryFormatCanonicalUtf8(canonicalText, out var bytesWritten); + Debug.Assert(formatted); writer.WriteStringValue(canonicalText.Slice(0, bytesWritten)); return; } diff --git a/src/Light.PortableResults/Text/CanonicalTextFormatter.cs b/src/Light.PortableResults/Text/CanonicalTextFormatter.cs index ee38f70..c40ac76 100644 --- a/src/Light.PortableResults/Text/CanonicalTextFormatter.cs +++ b/src/Light.PortableResults/Text/CanonicalTextFormatter.cs @@ -7,7 +7,6 @@ using System.Text; #endif #if NET10_0_OR_GREATER -using System.Buffers; using System.Text.Unicode; #endif @@ -248,6 +247,12 @@ public static unsafe bool TryFormatUtf8( out int bytesWritten ) { + if (text.IsEmpty) + { + bytesWritten = 0; + return true; + } + var requiredLength = GetUtf8ByteCountWithReplacement(text); if (destination.Length < requiredLength) { @@ -256,15 +261,15 @@ out int bytesWritten } #if NET10_0_OR_GREATER - var status = Utf8.FromUtf16( + Utf8.FromUtf16( text, destination, - out var charsRead, + out _, out bytesWritten, replaceInvalidSequences: true, isFinalBlock: true ); - return status == OperationStatus.Done && charsRead == text.Length; + return true; #else fixed (char* textPointer = text) { @@ -282,7 +287,7 @@ out int bytesWritten #endif } - private static int GetUtf8ByteCountWithReplacement(ReadOnlySpan text) + private static long GetUtf8ByteCountWithReplacement(ReadOnlySpan text) { long count = 0; for (var index = 0; index < text.Length; index++) @@ -309,7 +314,7 @@ private static int GetUtf8ByteCountWithReplacement(ReadOnlySpan text) } } - return count <= int.MaxValue ? (int) count : int.MaxValue; + return count; } #if !NET10_0_OR_GREATER @@ -329,7 +334,6 @@ int destinationLength index + 1 < textLength && text[index + 1] is >= '\uDC00' and <= '\uDFFF') { - scalar = 0x10000 + ((scalar - 0xD800) << 10) + text[index + 1] - 0xDC00; index++; continue; } diff --git a/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs b/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs index 87a682f..1083161 100644 --- a/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs +++ b/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs @@ -27,11 +27,14 @@ public static TheoryData CanonicalValues { "int64-minimum", MetadataValue.FromInt64(long.MinValue), Invariant(long.MinValue) }, { "int64-maximum", MetadataValue.FromInt64(long.MaxValue), Invariant(long.MaxValue) }, { "double", MetadataValue.FromDouble(-0.0), "-0.0" }, + { "string-empty", MetadataValue.FromString(string.Empty), string.Empty }, { "string-ascii", MetadataValue.FromString("canonical text"), "canonical text" }, { "string-non-ascii", MetadataValue.FromString("Grüße 日本語 😀"), "Grüße 日本語 😀" }, { "string-unpaired-high", MetadataValue.FromString("a\uD800b"), "a\uD800b" }, { "string-unpaired-low", MetadataValue.FromString("a\uDC00b"), "a\uDC00b" }, { "decimal-scale", MetadataValue.FromDecimal(19.50m), Invariant(19.50m) }, + { "decimal-negative-scale", MetadataValue.FromDecimal(-19.50m), Invariant(-19.50m) }, + { "decimal-negative-fraction", MetadataValue.FromDecimal(-0.5m), Invariant(-0.5m) }, { "decimal-negative-zero", MetadataValue.FromDecimal(-0.000m), Invariant(-0.000m) }, { "decimal-minimum", MetadataValue.FromDecimal(decimal.MinValue), Invariant(decimal.MinValue) }, { "decimal-maximum", MetadataValue.FromDecimal(decimal.MaxValue), Invariant(decimal.MaxValue) }, @@ -208,6 +211,22 @@ string expected var chars = CreateFilledArray(Math.Max(expected.Length, 1), '\uA55A'); var bytes = CreateFilledArray(Math.Max(expectedUtf8Length, 1), (byte) 0xA5); + if (expected.Length == 0) + { + value.TryFormatCanonical(chars.AsSpan(0, 0), out var emptyCharsWritten) + .Should() + .BeTrue(caseName); + value.TryFormatCanonicalUtf8(bytes.AsSpan(0, 0), out var emptyBytesWritten) + .Should() + .BeTrue(caseName); + + emptyCharsWritten.Should().Be(0, caseName); + emptyBytesWritten.Should().Be(0, caseName); + chars.Should().OnlyContain(character => character == '\uA55A', caseName); + bytes.Should().OnlyContain(element => element == 0xA5, caseName); + return; + } + value.TryFormatCanonical(chars.AsSpan(0, expected.Length - 1), out var charsWritten) .Should() .BeFalse(caseName); From 63aaf4becf60e2fc53b8c51d8471a1e47dfc3a2f Mon Sep 17 00:00:00 2001 From: Kenny Pflug Date: Sun, 2 Aug 2026 13:58:21 +0200 Subject: [PATCH 7/7] chore: add benchmarks results to the original plan Signed-off-by: Kenny Pflug --- ...0-try-format-canonical-zero-allocations.md | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/ai-plans/0070-0-try-format-canonical-zero-allocations.md b/ai-plans/0070-0-try-format-canonical-zero-allocations.md index 61b08fd..a2bd1ac 100644 --- a/ai-plans/0070-0-try-format-canonical-zero-allocations.md +++ b/ai-plans/0070-0-try-format-canonical-zero-allocations.md @@ -1,5 +1,46 @@ # Allocation-Free Canonical Formatting for MetadataValue +> **Benchmark results.** BenchmarkDotNet v0.15.8, macOS Tahoe 26.6, Apple M3 Max (16 logical cores), +> .NET SDK 10.0.302, .NET 10.0.10 Arm64 RyuJIT, `DefaultJob`, Release. +> +> Owned formatters against framework `TryFormat` (`CanonicalTextFormatterBenchmarks`). Each invocation +> formats three values into a stack buffer, so the per-value cost is one third of the mean. Neither +> side allocates. +> +> | Method | Category | Mean | Ratio | +> | --- | --- | ---: | ---: | +> | `FrameworkDateTimeTryFormat` | DateTime | 229.59 ns | 1.00 | +> | `CanonicalDateTimeTryFormat` | DateTime | 47.06 ns | 0.20 | +> | `FrameworkGuidTryFormat` | Guid | 10.52 ns | 1.00 | +> | `CanonicalGuidTryFormat` | Guid | 28.50 ns | 2.71 | +> +> The split matches this plan's expectation. The owned `DateTime` renderer wins by 4.9× because the +> canonical custom pattern drives `DateTime.TryFormat` into the general format interpreter instead of +> fixed-shape `TryFormatO`; the same reasoning covers `DateTimeOffset`, `DateOnly`, and `TimeOnly`, +> which share that renderer. `Guid.TryFormat` wins by 2.7× because its `D` branch is vectorized in +> current runtimes while the `v6.0.36` port predates that rewrite. That gap is the evidence for the +> deferred follow-up: roughly 6 ns per GUID against the ~900 ns cost of writing a whole CloudEvent, +> to be weighed against reintroducing an asset-divergent path in the area #58 unified. No +> target-specific implementation was added here. +> +> End-to-end write allocations. "Before" was measured in a worktree at `7c0e1b9`, the commit preceding +> this change, with the new benchmark fixtures copied in verbatim, so both columns serialize identical +> payloads and the delta isolates the library change rather than the fixture change. +> +> | Benchmark | Mean before | Mean after | Alloc before | Alloc after | Δ alloc | +> | --- | ---: | ---: | ---: | ---: | ---: | +> | `ToCloudEvents_GenericSuccessWithMetadata` | 1.043 µs | 916.6 ns | 1.31 KB | 1.09 KB | −16.8 % | +> | `ToCloudEvents_GenericSuccessWithMetadata_LegacyDirect` | 1.004 µs | 871.9 ns | 3.35 KB | 3.12 KB | −6.9 % | +> | `HttpWriteSerialization.GenericSuccessWithMetadata` | 430.3 ns | 328.4 ns | 432 B | 192 B | −55.6 % | +> +> Both paths drop ≈225–240 B, the sum of the canonical strings no longer materialized: a 36-character +> `Guid` (96 B), a 28-character `DateTime` (80 B), and a 22-character `Double` (72 B). CloudEvents +> shows the smaller relative win only because envelope construction dominates its larger baseline. The +> 12–24 % throughput improvement follows from the removed allocations and is an observation, not a +> goal. The header path is excluded from any claim: `HttpHeaderValueFormatter` returns `StringValues` +> and still materializes text by construction. The remaining benchmarks in both classes contain no +> affected metadata kinds and were not re-run. + ## Rationale `MetadataValue.TryFormatCanonical` is genuinely span-based only for `Double` and `Single`; every other kind calls `ToCanonicalString()` and copies the result. `Null`, `Boolean`, `String`, and `Uri` already return a literal or stored reference, but the other ten kinds allocate. This affects `JsonCloudEventsExtensions.WriteStringAttribute`, which still creates a throwaway string for values such as `Guid`, `DateTime`, and `Int64`, and the nine `TryGetXxx` validators that reformat candidates (`Int64`, `UInt64`, `Single`, four date/time kinds, `TimeSpan`, and `Guid`). `TryGetDecimal`, `TryGetChar`, and `TryGetUri` do not reformat and are unaffected.