diff --git a/THIRD-PARTY-NOTICES.md b/THIRD-PARTY-NOTICES.md index f973402..5198f0e 100644 --- a/THIRD-PARTY-NOTICES.md +++ b/THIRD-PARTY-NOTICES.md @@ -1,15 +1,19 @@ # Third-Party Notices -Light.PortableResults incorporates adapted floating-point number-formatting code from: +Light.PortableResults incorporates adapted primitive-formatting code from: - Project: .NET Runtime - Repository: https://github.com/dotnet/runtime - Source line: `release/6.0` - Immutable source: tag `v6.0.36`, commit `f1dd57165bfd91875761329ac3a8b17f6606ad18` -- Adapted files: `Number.Grisu3.cs`, `Number.DiyFp.cs`, `Number.Dragon4.cs`, - `Number.BigInteger.cs`, `Number.NumberBuffer.cs`, and `Number.Formatting.cs` +- Adapted files under `src/libraries/System.Private.CoreLib/src/System`: + `Number.Grisu3.cs`, `Number.DiyFp.cs`, `Number.Dragon4.cs`, `Number.BigInteger.cs`, + `Number.NumberBuffer.cs`, `Number.Formatting.cs`, `Decimal.DecCalc.cs`, + `Globalization/DateTimeFormat.cs`, and `Guid.cs` +- Adapted XML file: `src/libraries/System.Private.Xml/src/System/Xml/Schema/XsdDuration.cs` - Local files: the adapted implementation files under - `src/Light.PortableResults/Numbers`, as detailed in that folder's `README.md` + `src/Light.PortableResults/Numbers` and `src/Light.PortableResults/Text`, as detailed in those + folders' `README.md` files ## .NET Foundation MIT License diff --git a/ai-plans/0070-0-try-format-canonical-zero-allocations.md b/ai-plans/0070-0-try-format-canonical-zero-allocations.md new file mode 100644 index 0000000..a2bd1ac --- /dev/null +++ b/ai-plans/0070-0-try-format-canonical-zero-allocations.md @@ -0,0 +1,231 @@ +# Allocation-Free Canonical Formatting for MetadataValue + +> **Benchmark results.** BenchmarkDotNet v0.15.8, macOS Tahoe 26.6, Apple M3 Max (16 logical cores), +> .NET SDK 10.0.302, .NET 10.0.10 Arm64 RyuJIT, `DefaultJob`, Release. +> +> Owned formatters against framework `TryFormat` (`CanonicalTextFormatterBenchmarks`). Each invocation +> formats three values into a stack buffer, so the per-value cost is one third of the mean. Neither +> side allocates. +> +> | Method | Category | Mean | Ratio | +> | --- | --- | ---: | ---: | +> | `FrameworkDateTimeTryFormat` | DateTime | 229.59 ns | 1.00 | +> | `CanonicalDateTimeTryFormat` | DateTime | 47.06 ns | 0.20 | +> | `FrameworkGuidTryFormat` | Guid | 10.52 ns | 1.00 | +> | `CanonicalGuidTryFormat` | Guid | 28.50 ns | 2.71 | +> +> The split matches this plan's expectation. The owned `DateTime` renderer wins by 4.9× because the +> canonical custom pattern drives `DateTime.TryFormat` into the general format interpreter instead of +> fixed-shape `TryFormatO`; the same reasoning covers `DateTimeOffset`, `DateOnly`, and `TimeOnly`, +> which share that renderer. `Guid.TryFormat` wins by 2.7× because its `D` branch is vectorized in +> current runtimes while the `v6.0.36` port predates that rewrite. That gap is the evidence for the +> deferred follow-up: roughly 6 ns per GUID against the ~900 ns cost of writing a whole CloudEvent, +> to be weighed against reintroducing an asset-divergent path in the area #58 unified. No +> target-specific implementation was added here. +> +> End-to-end write allocations. "Before" was measured in a worktree at `7c0e1b9`, the commit preceding +> this change, with the new benchmark fixtures copied in verbatim, so both columns serialize identical +> payloads and the delta isolates the library change rather than the fixture change. +> +> | Benchmark | Mean before | Mean after | Alloc before | Alloc after | Δ alloc | +> | --- | ---: | ---: | ---: | ---: | ---: | +> | `ToCloudEvents_GenericSuccessWithMetadata` | 1.043 µs | 916.6 ns | 1.31 KB | 1.09 KB | −16.8 % | +> | `ToCloudEvents_GenericSuccessWithMetadata_LegacyDirect` | 1.004 µs | 871.9 ns | 3.35 KB | 3.12 KB | −6.9 % | +> | `HttpWriteSerialization.GenericSuccessWithMetadata` | 430.3 ns | 328.4 ns | 432 B | 192 B | −55.6 % | +> +> Both paths drop ≈225–240 B, the sum of the canonical strings no longer materialized: a 36-character +> `Guid` (96 B), a 28-character `DateTime` (80 B), and a 22-character `Double` (72 B). CloudEvents +> shows the smaller relative win only because envelope construction dominates its larger baseline. The +> 12–24 % throughput improvement follows from the removed allocations and is an observation, not a +> goal. The header path is excluded from any claim: `HttpHeaderValueFormatter` returns `StringValues` +> and still materializes text by construction. The remaining benchmarks in both classes contain no +> affected metadata kinds and were not re-run. + +## Rationale + +`MetadataValue.TryFormatCanonical` is genuinely span-based only for `Double` and `Single`; every other kind calls `ToCanonicalString()` and copies the result. `Null`, `Boolean`, `String`, and `Uri` already return a literal or stored reference, but the other ten kinds allocate. This affects `JsonCloudEventsExtensions.WriteStringAttribute`, which still creates a throwaway string for values such as `Guid`, `DateTime`, and `Int64`, and the nine `TryGetXxx` validators that reformat candidates (`Int64`, `UInt64`, `Single`, four date/time kinds, `TimeSpan`, and `Guid`). `TryGetDecimal`, `TryGetChar`, and `TryGetUri` do not reformat and are unaffected. + +The serializers ultimately write UTF-8. `CanonicalFloatingPointFormatter` already renders either encoding from one generic implementation, while #61 deferred its serializer integration: `MetadataExtensions.WriteNumberValue` still creates a `Double` string that System.Text.Json transcodes back to ASCII. The remaining non-text formats are also ASCII; only `String`, `Char`, and `Uri` require transcoding. + +Add allocation-free canonical formatters for both encodings, expose them through `MetadataValue`, make them the source for `ToCanonicalString` and validation, and adopt them in the JSON and CloudEvents writers. + +## Acceptance Criteria + +- [x] `MetadataValue` exposes `TryFormatCanonical` and `TryFormatCanonicalUtf8`; neither allocates for any primitive kind, including copying or transcoding `String`, `Char`, and `Uri` into the caller's destination. +- [x] One canonical renderer serves both package assets and encodings. Only decimal field extraction and UTF-16-to-UTF-8 transcoding vary by target, and neither decides the output text. There is no target-specific formatter, renderer, cross-encoding route, or intermediate output buffer; every method writes directly to the requested destination. One expected-output corpus passes unchanged against both assets. +- [x] A public, XML-documented `CanonicalTextFormatter` exposes the new primitives in both encodings and per-type maximum lengths bounding both encodings on both assets. +- [x] `MetadataValue` exposes a documented public bound for every bounded primitive in either encoding; `JsonCloudEventsExtensions` uses it instead of a private magic number. +- [x] UTF-16 output exactly matches the previous `ToCanonicalString` text and count. UTF-8 output is that text's replacement-fallback encoding—lossy only for malformed UTF-16—with equal byte/character counts whenever the canonical text is ASCII. +- [x] In both encodings, insufficient capacity returns `false`, reports zero, and leaves the destination unchanged; `Array` and `Object` still throw `InvalidOperationException`; corrupt `DateTime`, `DateOnly`, and `TimeOnly` payloads still throw `InvalidOperationException`. +- [x] `CanonicalTextFormatter.TryFormat(DateTime, …)` normalizes `Local` to UTC like `MetadataValue.FromDateTime`, so all accepted values fit `MaximumDateTimeLength`; `Utc` and `Unspecified` are unchanged. A direct test covers this. +- [x] `TryGetXxx` round-trip validators compare span-formatted text without allocating a candidate string. +- [x] `ToCanonicalString` remains allocation-free for `Null`, `Boolean`, `String`, and `Uri`, and remains textually unchanged for every kind. +- [x] The JSON metadata writer materializes no canonical string for string-shaped or `Double`/`Single` number-shaped values. Its bytes remain unchanged under the default and `UnsafeRelaxedJsonEscaping` encoders, including non-ASCII text and unpaired surrogates. +- [x] Tests cover every kind and encoding: canonical output/count, exact and one-short capacity, boundaries, invalid-payload exceptions, and allocations. Metadata tests pass against the `net10.0` and `netstandard2.0` library assets. +- [x] A `net10.0` microbenchmark compares the new `Guid` and `DateTime` formatters with framework `TryFormat`; results are recorded in the pull request as evidence for a deferred follow-up, with no target-specific implementation added here regardless of outcome. +- [x] CloudEvents and HTTP write benchmarks contain affected metadata kinds; before/after allocations for both are recorded in the pull request. +- [x] `THIRD-PARTY-NOTICES.md` and the folder README identify all adapted upstream files and adaptations; every adapted source retains the .NET Foundation MIT header. +- [x] The `netstandard2.0` decimal path documents and allocation-freely enforces its runtime-layout assumption. A violation throws `PlatformNotSupportedException` from decimal formatting itself without disabling other formatters. +- [x] Package release notes mention the new APIs and removed allocations. +- [x] Both targets build in Release with warnings as errors, package validation succeeds, the Native AOT sample publishes, and coverage remains above 95%. + +## Technical Details + +### Architecture and upstream sources + +`netstandard2.0` has no affected span-formatting APIs; `Polyfill` extensions call `ToString` and copy, and `IUtf8SpanFormattable` is unavailable. Ship one owned implementation for both assets and encodings, with no `NET10_0_OR_GREATER` branch to framework formatting. This avoids the asset wire-format divergence fixed for floating point in #58 and keeps one implementation and test surface. + +Framework fast paths remain a follow-up informed by the required benchmark. `Guid.TryFormat` has a vectorized `D` path and is plausible; the date kinds are less likely because their canonical custom pattern enters the general format interpreter instead of fixed-shape `TryFormatO`. Do not branch in this issue even if a framework API wins. + +Adapt—not rederive—the scalar implementations from the repository's existing `dotnet/runtime` baseline, tag `v6.0.36`, commit `f1dd57165bfd91875761329ac3a8b17f6606ad18`. This line predates the `System.Runtime.Intrinsics` rewrites and compiles for `netstandard2.0`; newer sources would require de-vectorization. `Numbers/` already establishes the pattern: adapted upstream code under the .NET Foundation MIT header, with provenance in `THIRD-PARTY-NOTICES.md` and a folder README. Rederiving carries the real risk—the `TryGetXxx` validators pin these encodings, so a subtly wrong hand-derived rule breaks round-tripping of data already on the wire. + +| Kind | Upstream source and adaptation | +| --- | --- | +| `Int64`, `UInt64` | `Number.Formatting.cs`: `TryUInt64ToDecStr`, `UInt32ToDecChars`, `Int64DivMod1E9`; `FormattingHelpers.CountDigits`. Direct scalar port over a fixed destination. | +| `Decimal` | `Number.Formatting.cs`: `DecimalToNumber`; `Decimal.DecCalc.cs`: `DecDivMod1E9`. Reuse the existing `NumberBuffer`; add a scale-preserving renderer. | +| Date/time kinds | `DateTimeFormat.cs`: `TryFormatO`, `WriteTwoDecimalDigits`, `WriteFourDecimalDigits`, `WriteDigits`. Port `TryFormatO`, not `FormatCustomized`. | +| `TimeSpan` | `System.Private.Xml/XsdDuration.cs`: `TimeSpan` constructor and `ToString(DurationType)`, the normative implementation of `XmlConvert.ToString(TimeSpan)`. | +| `Guid` | `Guid.cs`: `TryFormat`'s `D` branch, `HexsToChars`, `HexConverter.ToCharLower`. Numeric-field formatting is endianness-independent. | + +`FormatCustomized` brings culture, calendars, Hebrew/Japanese cases, and `StringBuilderCache`; `TryFormatO` is a small culture-free fixed template. Adapt it to omit a zero fraction and otherwise trim trailing zeros; replace internal `GetDate`/`GetTimePrecise` with ported helpers or public component properties (accepting their repeated tick calculations); drop its local-offset branch for `DateTime` because local values are normalized, while retaining offsets for `DateTimeOffset`. + +All upstream renderers target `char`; route ASCII through the repository's `TCodeUnit` conversion pattern so one body writes either encoding. + +### Decimal extraction + +`decimal.GetBits(decimal, Span)` is unavailable on `netstandard2.0`; its array overload allocates. Upstream sidesteps this with `Unsafe.As`, but that guarantee is weaker than it looks: `GetBits` documents the *logical* representation, not the private field order, and upstream `DecCalc` carries an explicit `#if BIGENDIAN` layout—endianness is part of the assumption rather than incidental to it. Split only extraction: + +- `net10.0` uses the contractual, allocation-free, endian-independent span overload. +- `netstandard2.0` reinterprets the value, documented as requiring a little-endian runtime with historical `flags`, `hi`, `lo`, `mid` field order. Digit generation and rendering remain shared. + +Validate the legacy assumption once without calling the allocating array overload. Construct + +```csharp +var probe = new decimal( + lo: 0x11111111, + mid: 0x22222222, + hi: 0x33333333, + isNegative: true, + scale: 5 +); +``` + +then reinterpret and compare the fields with those arguments and flags `0x80050000`. The constructor defines the same logical representation as `GetBits`; this uses only a struct and integers. + +Keep the guard in a decimal-only private helper, store its result in `static readonly bool`, and throw `PlatformNotSupportedException` from decimal formatting—not a type initializer, which would wrap it in `TypeInitializationException` and disable unrelated formatters. The initialized flag folds away. A unit test should compare reinterpretation with `decimal.GetBits`, but both assets run on the .NET 10 host, so only the runtime guard covers .NET Framework or Mono. Per `tests/AGENTS.md`, mutants in the initializer are a known static-analysis blind spot. Warmed allocation tests also miss initialization, so first use must be allocation-free by construction. + +### Canonical contracts + +- **Decimal:** preserve scale (`19.50m` → `19.50`); omit the sign of negative zero, matching the framework. +- **TimeSpan:** zero is `PT0S`; `-` precedes `P`; omit zero days; include `T` only when a time component follows; trim fractional zeros (`P2DT3H4M5.06S`, `PT0.0000001S`). Preserve upstream's `unchecked((ulong)-ticks)` handling of `TimeSpan.MinValue`. +- **Date/time:** `DateTime` UTC ends in `Z`, Unspecified has no designator, matching `yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK`; `DateTimeOffset` uses `+hh:mm`/`-hh:mm`, never `Z`. Date/time fractions are omitted at zero and otherwise trimmed. +- **Guid:** lowercase `D` format. + +For `DateTime`, conditionally call `ToUniversalTime()` only for `Kind.Local` and render `Z`, matching `FromDateTime`. Document that this input depends on the machine's local time zone. Never normalize Unspecified: `ToUniversalTime()` would treat it as local and shift it. Conversion saturates at `DateTime.MinValue`/`MaxValue`, so it adds no exception. Normalization also keeps the maximum at 28 rather than the 33 characters required by a local `±hh:mm` form. + +`FromDateTime` already normalizes Local and `TryReadStoredDateTime` rejects a stored Local payload, so only direct formatter tests can reach this branch. Assert normalized text/count/bound plus an Unspecified value that is not shifted. + +### Public API and failure behavior + +```csharp +namespace Light.PortableResults.Text; + +public static class CanonicalTextFormatter +{ + public const int MaximumInt64Length = 20; + public const int MaximumUInt64Length = 20; + public const int MaximumDecimalLength = 31; + public const int MaximumCharLength = 3; + + public const int MaximumDayNumber = 3_652_058; // DateOnly.MaxValue.DayNumber + public const long MaximumTimeOfDayTicks = 863_999_999_999; // TimeSpan.TicksPerDay - 1 + public const int MaximumDateTimeLength = 28; + public const int MaximumDateTimeOffsetLength = 33; + public const int MaximumDateLength = 10; + public const int MaximumTimeLength = 16; + public const int MaximumTimeSpanLength = 27; + public const int MaximumGuidLength = 36; + + public static bool TryFormat(char value, Span destination, out int charsWritten); + public static bool TryFormat(long value, Span destination, out int charsWritten); + public static bool TryFormat(ulong value, Span destination, out int charsWritten); + public static bool TryFormat(decimal value, Span destination, out int charsWritten); + public static bool TryFormat(DateTime value, Span destination, out int charsWritten); + public static bool TryFormat(DateTimeOffset value, Span destination, out int charsWritten); + public static bool TryFormat(TimeSpan value, Span destination, out int charsWritten); + public static bool TryFormat(Guid value, Span destination, out int charsWritten); + public static bool TryFormatDate(int dayNumber, Span destination, out int charsWritten); + public static bool TryFormatTime(long ticks, Span destination, out int charsWritten); + + // One Span/bytesWritten TryFormatUtf8 counterpart for every overload above. + public static bool TryFormatUtf8( + ReadOnlySpan text, + Span destination, + out int bytesWritten + ); +} +``` + +These signatures are exact. The `char` overload prevents a char literal from binding to `long` (or, if integer methods were renamed, `decimal`) and rendering `'x'` as `120`. Other integral types may correctly bind to `long`. Test overload resolution with a char literal. + +`false` means only insufficient capacity. `TryFormatDate` accepts `[0, MaximumDayNumber]`; `TryFormatTime` accepts `[0, MaximumTimeOfDayTicks]`; invalid values throw `ArgumentOutOfRangeException`, consistent with the floating formatter reserving `false` for capacity and throwing for non-finite input. `MetadataValue` pre-validates with the same public constants to retain its existing `InvalidOperationException` and message; test the formatter's otherwise-unreachable range exceptions directly. + +`MaximumCharLength` is 3 because the constants bound both encodings: one BMP code unit, including an unpaired surrogate replaced by U+FFFD, needs at most three UTF-8 bytes. The UTF-16 method writes one. `TryFormatDate` and `TryFormatTime` accept stored payloads because `DateOnly`/`TimeOnly` BCL types are absent from `netstandard2.0`. The text-transcoding overload needs no UTF-16 peer because chars use `TryCopyTo`. + +Keep `CanonicalFloatingPointFormatter` and its API/constants in place. Add to `MetadataValue`: + +```csharp +public const int MaximumPrimitiveCanonicalLength = 36; +public bool TryFormatCanonicalUtf8(Span destination, out int bytesWritten); +``` + +The bound is `Guid`'s length and covers both encodings. Document that it excludes unbounded `String` and `Uri`; only they may still outgrow this buffer. `Char` remains bounded at three. CloudEvents therefore retains its materializing fallback. + +### Rendering, transcoding, and single sourcing + +Follow `CanonicalFloatingPointFormatter.TryRender`: private generic cores constrained to `unmanaged`, a shared internal conversion helper, and JIT-foldable `typeof(TCodeUnit) == typeof(byte)` ASCII writes. Do not expose the helper, duplicate it, or guard impossible non-`char`/`byte` instantiations (an unreachable throw is a coverage hole). Calculate required length before writing to preserve all-or-nothing behavior. + +`String`, `Char`, and `Uri` instead transcode with replacement fallback: unpaired surrogates accepted by `FromChar`/`FromString` become U+FFFD. This is intentionally lossy because malformed UTF-16 has no valid UTF-8 representation; `false` must still mean only insufficient capacity. Counts vary for non-ASCII/malformed text, but match for ASCII; UTF-8 round-trip identity is not promised. + +Use `Utf8.FromUtf16(replaceInvalidSequences: true)` on `net10.0`. On `netstandard2.0`, use `Encoding`'s pointer overload under `fixed`, whose default fallback also emits U+FFFD. Both paths count bytes before writing, requiring a second pass but preventing partial output. + +The serializers must not use pre-transcoded UTF-8 for these three kinds. With System.Text.Json 10.0.10 and `UnsafeRelaxedJsonEscaping`, malformed input differs bytewise: + +```text +input "a\uD800b" +char route: 22-61-5C-75-46-46-46-44-62-22 +UTF-8 route: 22-61-EF-BF-BD-62-22 +``` + +Both parse equivalently, and both routes converge for valid U+FFFD; the default encoder also hides the difference by escaping non-ASCII. Preserve bytes by letting the writer transcode text-bearing kinds. + +Make span formatting primary. `ToCanonicalString` stack-formats bounded values and creates one string from the written slice, but retains allocation-free fast paths: literals for `Null`/`Boolean`, stored references for `String`/`Uri`. Share literals as `private const string`; both span encodings write the same constant through the ASCII helper—do not add separate `u8` literals. + +Likewise, validator `FormatXxx` helpers stack-format and compare with `MemoryExtensions.SequenceEqual`; replace `TryGetInt64`'s inline invariant `ToString` too. + +### Serializer adoption + +The rule: the writer performs any UTF-16 transcoding itself; serializers hand it UTF-8 only for kinds whose canonical text is ASCII, where the two encodings cannot disagree. + +- `MetadataExtensions.WriteNumberValue` stack-formats `Double`/`Single` as UTF-8 and calls `WriteRawValue(ReadOnlySpan, skipInputValidation: true)`, completing #61. +- `WriteMetadataValue` stack-writes ASCII-bounded string shapes (date kinds, `TimeSpan`, `Guid`, `UInt64`) in either encoding. +- `String`/`Uri` pass their stored string to the writer; `Char` uses UTF-16 so the writer owns surrogate handling. +- `JsonCloudEventsExtensions.WriteStringAttribute` remains UTF-16, switches its stack size to the public constant, and retains fallback. CloudEvents rejects unpaired surrogates upstream. +- `HttpHeaderValueFormatter` remains unchanged because returning `StringValues` inherently requires strings. + +### Benchmarks and tests + +Current write fixtures contain only `FromString`, so their allocation delta is necessarily zero. Add `Guid`, `DateTime`, `Double`, and an `Int64` outside the inclusive 32-bit range: + +- CloudEvents: annotate with `SerializeInCloudEventsExtensionAttributes`; the range matters because an in-range `Int64` uses numeric `Integer` encoding and bypasses `WriteStringAttribute`. +- HTTP: annotate with `SerializeInHttpResponseBody`; do not claim a header improvement, because `StringValues` still materializes text. + +Test canonical behavior sociably through `MetadataValue`, using direct `CanonicalTextFormatter` tests only for inaccessible inputs such as Local `DateTime` and invalid raw date/time ranges. Derive expected text from the same framework `ToString` calls used before this change; derive UTF-8 expectations from those strings (ASCII where applicable), as in #61. + +Cover each kind's minimum/maximum, zero/negative zero, decimal scale, `TimeSpan.MinValue`/`MaxValue`/`Zero` and component omission, fractions present/absent for date/time values, and both storable `DateTimeKind` values. For text kinds cover non-ASCII, surrogate pairs, and unpaired surrogates in both encodings. At writer level assert exact bytes with both JSON encoders; default-only coverage is insufficient because it hides the malformed-input divergence. + +Measure allocations with `GC.GetAllocatedBytesForCurrentThread` around warmed loops for every affected kind and encoding, plus the four allocation-free `ToCanonicalString` kinds. Extend this to validators using `String`-kind canonical input; typed values return early and would miss the old allocation. Warm-up hides lazy initialization, hence the allocation-free-construction requirement above. + +All new methods have `out` parameters and therefore fall into Stryker Safe Mode's documented compile-failure blind spot. Mutation score cannot assess them; the pull request must manually map each formatter contract to its constraining tests. + +Framework target-specific calls and any resulting optimization remain deferred. Provenance, release notes, benchmark results, build/package/AOT validation, and coverage are required by the acceptance criteria above. diff --git a/ai-plans/0070-1-plan-deviations.md b/ai-plans/0070-1-plan-deviations.md new file mode 100644 index 0000000..7d6569b --- /dev/null +++ b/ai-plans/0070-1-plan-deviations.md @@ -0,0 +1,49 @@ +# Plan Deviations for Allocation-Free Canonical Formatting + +## Referenced Plans + +- `0058-fix-runtime-specific-number-metadata.md` introduced the public + `CanonicalFloatingPointFormatter` for runtime-independent `double` and `float` text. +- `0061-add-utf-8-floating-point-formatting.md` added its UTF-8 overloads and established the + private per-call Dragon4 test seam. +- `0070-0-try-format-canonical-zero-allocations.md` introduced `CanonicalTextFormatter` for the + remaining primitive metadata kinds and explicitly retained the existing floating-point formatter. + +## Deviations + +### Unified formatter surface + +The implemented API no longer exposes a separate `CanonicalFloatingPointFormatter`. Its public +constants and its `Format`, `TryFormat`, and `TryFormatUtf8` overloads for `double` and `float` are now +members of the partial `CanonicalTextFormatter` in `Light.PortableResults.Text`. The floating-point +surface resides in `CanonicalTextFormatter.FloatingPoint.cs`; the Grisu3, Dragon4, number-buffer, and +compatibility implementations remain internal types in `Light.PortableResults.Numbers`. + +Production call sites, tests, and benchmarks use the unified formatter. The floating-point tests +continue to locate the two private generic `TryFormatCore` overloads for their forced-Dragon4 corpus, +so the test seam required by plans 0058 and 0061 is unchanged apart from its declaring type. + +## Rationale + +Once plan 0070 added canonical formatting for every other primitive kind, retaining a second public +formatter made the API harder to discover and forced consumers such as `MetadataValue` to dispatch +between two classes with the same destination, atomicity, and allocation contracts. A partial class +keeps the large floating-point implementation in its own source file without creating a runtime or +performance boundary. The library is not yet stable and permits breaking API changes, so consolidating +the surface now is preferable to preserving the historical split through forwarding APIs. + +This deviation changes API ownership and source organization only. Floating-point text, exceptions, +capacity behavior, allocation behavior, UTF-8 output, algorithm selection, and wire formats remain +unchanged. + +### Public code-unit helper + +Plan 0070 originally required the shared code-unit helper to remain unexposed. The implemented +`CanonicalCodeUnit` is instead a public, top-level type in the focused `Light.PortableResults.Text` +namespace. This follows the repository's hide-in-plain-sight approach: advanced implementation-oriented +types remain accessible without adding them to the main namespace or nesting them inside a facade. + +The type and its `FromAscii` method are XML-documented. The public method supports +`byte` and `char`, matching the formatter's UTF-8 and UTF-16 destinations, and rejects other unmanaged +types with `NotSupportedException`. The supported generic instantiations retain their direct +allocation-free reinterpretation paths. diff --git a/benchmarks/Benchmarks/CanonicalTextFormatterBenchmarks.cs b/benchmarks/Benchmarks/CanonicalTextFormatterBenchmarks.cs new file mode 100644 index 0000000..05825ea --- /dev/null +++ b/benchmarks/Benchmarks/CanonicalTextFormatterBenchmarks.cs @@ -0,0 +1,85 @@ +using System; +using System.Globalization; +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Engines; +using Light.PortableResults.Text; + +namespace Benchmarks; + +[MemoryDiagnoser] +[SimpleJob] +[GroupBenchmarksBy(BenchmarkDotNet.Configs.BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class CanonicalTextFormatterBenchmarks +{ + private readonly Consumer _consumer = new(); + private readonly DateTime[] _dateTimes = + { + DateTime.SpecifyKind(DateTime.MinValue, DateTimeKind.Utc), + new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Utc).AddTicks(1_234_567), + DateTime.SpecifyKind(DateTime.MaxValue, DateTimeKind.Utc) + }; + private readonly Guid[] _guids = + { + Guid.Empty, + new ("a1b2c3d4-e5f6-7890-abcd-ef1234567890"), + new ("ffffffff-ffff-ffff-ffff-ffffffffffff") + }; + + [Benchmark(Baseline = true)] + [BenchmarkCategory("Guid")] + public void FrameworkGuidTryFormat() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumGuidLength]; + foreach (var value in _guids) + { + value.TryFormat(destination, out var charsWritten, "D"); + _consumer.Consume(charsWritten); + _consumer.Consume(destination[0]); + } + } + + [Benchmark] + [BenchmarkCategory("Guid")] + public void CanonicalGuidTryFormat() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumGuidLength]; + foreach (var value in _guids) + { + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten); + _consumer.Consume(charsWritten); + _consumer.Consume(destination[0]); + } + } + + [Benchmark(Baseline = true)] + [BenchmarkCategory("DateTime")] + public void FrameworkDateTimeTryFormat() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + foreach (var value in _dateTimes) + { + value.TryFormat( + destination, + out var charsWritten, + "yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK", + CultureInfo.InvariantCulture + ); + _consumer.Consume(charsWritten); + _consumer.Consume(destination[0]); + } + } + + [Benchmark] + [BenchmarkCategory("DateTime")] + public void CanonicalDateTimeTryFormat() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + foreach (var value in _dateTimes) + { + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten); + _consumer.Consume(charsWritten); + _consumer.Consume(destination[0]); + } + } +} diff --git a/benchmarks/Benchmarks/CanonicalFloatingPointFormatterBenchmarks.cs b/benchmarks/Benchmarks/CanonicalTextFormatterFloatingPointBenchmarks.cs similarity index 92% rename from benchmarks/Benchmarks/CanonicalFloatingPointFormatterBenchmarks.cs rename to benchmarks/Benchmarks/CanonicalTextFormatterFloatingPointBenchmarks.cs index 7c00d43..ddeff49 100644 --- a/benchmarks/Benchmarks/CanonicalFloatingPointFormatterBenchmarks.cs +++ b/benchmarks/Benchmarks/CanonicalTextFormatterFloatingPointBenchmarks.cs @@ -3,14 +3,14 @@ using System.Globalization; using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Engines; -using Light.PortableResults.Numbers; +using Light.PortableResults.Text; namespace Benchmarks; [MemoryDiagnoser] [GroupBenchmarksBy(BenchmarkDotNet.Configs.BenchmarkLogicalGroupRule.ByCategory)] [CategoriesColumn] -public class CanonicalFloatingPointFormatterBenchmarks +public class CanonicalTextFormatterFloatingPointBenchmarks { private const int ValueCount = 1_024; @@ -87,7 +87,7 @@ public int CanonicalDoubleTryFormat() var totalLength = 0; foreach (var value in _doubleValues) { - CanonicalFloatingPointFormatter.TryFormat(value, destination, out var charsWritten); + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten); totalLength += charsWritten; } @@ -116,7 +116,7 @@ public int CanonicalSingleTryFormat() var totalLength = 0; foreach (var value in _singleValues) { - CanonicalFloatingPointFormatter.TryFormat(value, destination, out var charsWritten); + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten); totalLength += charsWritten; } @@ -141,7 +141,7 @@ public void CanonicalDoubleFormat() { foreach (var value in _doubleValues) { - _consumer.Consume(CanonicalFloatingPointFormatter.Format(value)); + _consumer.Consume(CanonicalTextFormatter.Format(value)); } } @@ -163,7 +163,7 @@ public void CanonicalSingleFormat() { foreach (var value in _singleValues) { - _consumer.Consume(CanonicalFloatingPointFormatter.Format(value)); + _consumer.Consume(CanonicalTextFormatter.Format(value)); } } diff --git a/benchmarks/Benchmarks/CloudEventsWritingBenchmarks.cs b/benchmarks/Benchmarks/CloudEventsWritingBenchmarks.cs index 2d2d506..22b87eb 100644 --- a/benchmarks/Benchmarks/CloudEventsWritingBenchmarks.cs +++ b/benchmarks/Benchmarks/CloudEventsWritingBenchmarks.cs @@ -70,6 +70,34 @@ public void Setup() "traceid", MetadataValue.FromString("trace-456", MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes) ); + metadataBuilder.Add( + "correlationid", + MetadataValue.FromGuid( + new Guid("a1b2c3d4-e5f6-7890-abcd-ef1234567890"), + MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes + ) + ); + metadataBuilder.Add( + "recordedat", + MetadataValue.FromDateTime( + new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Utc).AddTicks(1_234_567), + MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes + ) + ); + metadataBuilder.Add( + "measurement", + MetadataValue.FromDouble( + 36_028_797_018_963_968.0, + MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes + ) + ); + metadataBuilder.Add( + "sequence", + MetadataValue.FromInt64( + (long) int.MaxValue + 1, + MetadataValueAnnotation.SerializeInCloudEventsExtensionAttributes + ) + ); var metadata = metadataBuilder.Build(); _genericSuccessWithMetadataResult = Result.Ok( @@ -459,4 +487,6 @@ public sealed class ContactDto } [JsonSerializable(typeof(CloudEventsWritingBenchmarks.ContactDto))] +[JsonSerializable(typeof(CloudEventsEnvelopeForWriting))] +[JsonSerializable(typeof(CloudEventsEnvelopeForWriting))] internal partial class CloudEventsWritingBenchmarksJsonContext : JsonSerializerContext; diff --git a/benchmarks/Benchmarks/HttpWriteSerializationBenchmarks.cs b/benchmarks/Benchmarks/HttpWriteSerializationBenchmarks.cs index fdef3f2..dd66df4 100644 --- a/benchmarks/Benchmarks/HttpWriteSerializationBenchmarks.cs +++ b/benchmarks/Benchmarks/HttpWriteSerializationBenchmarks.cs @@ -77,6 +77,26 @@ public void Setup() var metadata = MetadataObject.Create( ("correlationId", MetadataValue.FromString("corr-123", MetadataValueAnnotation.SerializeInHttpResponseBody)), + ("requestId", + MetadataValue.FromGuid( + new Guid("a1b2c3d4-e5f6-7890-abcd-ef1234567890"), + MetadataValueAnnotation.SerializeInHttpResponseBody + )), + ("recordedAt", + MetadataValue.FromDateTime( + new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Utc).AddTicks(1_234_567), + MetadataValueAnnotation.SerializeInHttpResponseBody + )), + ("measurement", + MetadataValue.FromDouble( + 36_028_797_018_963_968.0, + MetadataValueAnnotation.SerializeInHttpResponseBody + )), + ("sequence", + MetadataValue.FromInt64( + (long) int.MaxValue + 1, + MetadataValueAnnotation.SerializeInHttpResponseBody + )), ("traceId", MetadataValue.FromString("trace-456", MetadataValueAnnotation.SerializeInHttpHeader)) ); _genericSuccessWithMetadata = Result.Ok( diff --git a/src/Light.PortableResults/CloudEvents/Writing/Json/JsonCloudEventsExtensions.cs b/src/Light.PortableResults/CloudEvents/Writing/Json/JsonCloudEventsExtensions.cs index fa9e532..ff00bfe 100644 --- a/src/Light.PortableResults/CloudEvents/Writing/Json/JsonCloudEventsExtensions.cs +++ b/src/Light.PortableResults/CloudEvents/Writing/Json/JsonCloudEventsExtensions.cs @@ -11,10 +11,6 @@ namespace Light.PortableResults.CloudEvents.Writing.Json; /// public static class JsonCloudEventsExtensions { - // Guid's canonical D format is the longest bounded primitive encoding. Arbitrarily long String and Uri - // values reuse their existing text when this buffer is insufficient. - private const int CanonicalTextBufferLength = 36; - /// /// Serializes the contents of a into the provided /// using the supplied serializer options. @@ -375,7 +371,7 @@ private static void WriteStringAttribute( MetadataValue value ) { - Span canonicalText = stackalloc char[CanonicalTextBufferLength]; + Span canonicalText = stackalloc char[MetadataValue.MaximumPrimitiveCanonicalLength]; if (value.TryFormatCanonical(canonicalText, out var charsWritten)) { writer.WritePropertyName(attributeName); diff --git a/src/Light.PortableResults/Light.PortableResults.csproj b/src/Light.PortableResults/Light.PortableResults.csproj index 9c32266..abf731b 100644 --- a/src/Light.PortableResults/Light.PortableResults.csproj +++ b/src/Light.PortableResults/Light.PortableResults.csproj @@ -19,6 +19,10 @@ hosts and corrected on .NET Framework and legacy Mono hosts. - Adds allocation-free UTF-8 span formatting for canonical Double and Single values, plus public maximum-length constants shared by the UTF-16 and UTF-8 formatting APIs. + - Adds CanonicalTextFormatter with allocation-free UTF-16 and UTF-8 formatters for all primitive + metadata types, MetadataValue.TryFormatCanonicalUtf8, and public cross-encoding length bounds. + - Removes canonical-string allocations from metadata round-trip validation, JSON metadata writing, + and CloudEvents extension-attribute writing for bounded primitive values. - Emits primitive metadata arrays as ordered, separate HTTP header values. Custom header converters can reuse the new public HttpHeaderValueFormatter. @@ -28,6 +32,8 @@ - Values of the ten newly typed BCL metadata types no longer flatten to MetadataKind.String. Their JSON representations now use the canonical encoding of their dedicated kind. - Whole-number Double and Single metadata values serialize with a trailing .0. + - CanonicalFloatingPointFormatter has been consolidated into the partial CanonicalTextFormatter; + its floating-point constants and formatting overloads now reside on CanonicalTextFormatter. - Default HTTP header conversion no longer surrounds string metadata values with quotes. - Float metadata values now use MetadataKind.Single rather than MetadataKind.Double. - Decimal metadata values now have the dedicated kind MetadataKind.Decimal instead of MetadataKind.String, diff --git a/src/Light.PortableResults/Metadata/MetadataValue.cs b/src/Light.PortableResults/Metadata/MetadataValue.cs index d5bb530..6a5576f 100644 --- a/src/Light.PortableResults/Metadata/MetadataValue.cs +++ b/src/Light.PortableResults/Metadata/MetadataValue.cs @@ -1,7 +1,8 @@ using System; +using System.Diagnostics; using System.Globalization; using System.Xml; -using Light.PortableResults.Numbers; +using Light.PortableResults.Text; namespace Light.PortableResults.Metadata; @@ -13,16 +14,26 @@ namespace Light.PortableResults.Metadata; /// public readonly struct MetadataValue : IEquatable { - private const string DateTimeFormat = "yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK"; - private const string DateTimeOffsetFormat = "yyyy-MM-dd'T'HH:mm:ss.FFFFFFFzzz"; private const string DateOnlyFormat = "yyyy-MM-dd"; - private const string TimeOnlyFormat = "HH:mm:ss.FFFFFFF"; + private const string NullCanonicalText = "null"; + private const string TrueCanonicalText = "true"; + private const string FalseCanonicalText = "false"; /// /// Gets the default annotation for metadata values, which is . /// public const MetadataValueAnnotation DefaultAnnotation = MetadataValueAnnotation.SerializeInBodies; + /// + /// A destination size that holds the canonical text of every bounded primitive metadata value, + /// in UTF-16 characters or UTF-8 bytes. + /// + /// + /// and are unbounded and may + /// require a larger destination. Every other primitive kind fits this bound. + /// + public const int MaximumPrimitiveCanonicalLength = CanonicalTextFormatter.MaximumGuidLength; + private readonly MetadataPayload _payload; private MetadataValue( @@ -54,13 +65,13 @@ private MetadataValue( /// /// Gets a representing a null value. /// - public static MetadataValue Null => new (MetadataKind.Null, default); + public static MetadataValue Null => new(MetadataKind.Null, default); /// /// Creates a null metadata value. /// public static MetadataValue FromNull(MetadataValueAnnotation annotation = DefaultAnnotation) => - new (MetadataKind.Null, default, annotation); + new(MetadataKind.Null, default, annotation); /// /// Creates a metadata value from a Boolean. @@ -69,7 +80,7 @@ public static MetadataValue FromBoolean( bool value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Boolean, new MetadataPayload(value ? 1L : 0L), annotation); + new(MetadataKind.Boolean, new MetadataPayload(value ? 1L : 0L), annotation); /// /// Creates a metadata value from a signed 64-bit integer. @@ -78,7 +89,7 @@ public static MetadataValue FromInt64( long value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Int64, new MetadataPayload(value), annotation); + new(MetadataKind.Int64, new MetadataPayload(value), annotation); /// /// Creates a metadata value from a double-precision floating-point number. @@ -115,7 +126,7 @@ public static MetadataValue FromDecimal( decimal value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Decimal, new MetadataPayload(value), annotation); + new(MetadataKind.Decimal, new MetadataPayload(value), annotation); /// /// Creates a metadata value from an unsigned 64-bit integer. @@ -124,7 +135,7 @@ public static MetadataValue FromUInt64( ulong value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.UInt64, MetadataPayload.FromUInt64(value), annotation); + new(MetadataKind.UInt64, MetadataPayload.FromUInt64(value), annotation); /// /// Creates a metadata value from a single-precision floating-point number. @@ -146,7 +157,7 @@ public static MetadataValue FromChar( char value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Char, new MetadataPayload(value), annotation); + new(MetadataKind.Char, new MetadataPayload(value), annotation); /// /// @@ -182,7 +193,7 @@ public static MetadataValue FromDateTimeOffset( DateTimeOffset value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.DateTimeOffset, new MetadataPayload(value), annotation); + new(MetadataKind.DateTimeOffset, new MetadataPayload(value), annotation); #if NET10_0_OR_GREATER /// @@ -211,7 +222,7 @@ public static MetadataValue FromTimeSpan( TimeSpan value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.TimeSpan, new MetadataPayload(value.Ticks), annotation); + new(MetadataKind.TimeSpan, new MetadataPayload(value.Ticks), annotation); /// /// Creates a metadata value from a globally unique identifier. @@ -220,7 +231,7 @@ public static MetadataValue FromGuid( Guid value, MetadataValueAnnotation annotation = DefaultAnnotation ) => - new (MetadataKind.Guid, new MetadataPayload(value), annotation); + new(MetadataKind.Guid, new MetadataPayload(value), annotation); /// /// Creates a metadata value from a URI. A null URI becomes a null metadata value. @@ -392,7 +403,7 @@ public bool TryGetInt64(out long value) if (TryGetRawString(out var text) && long.TryParse(text, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out value) && - string.Equals(value.ToString(CultureInfo.InvariantCulture), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -490,7 +501,7 @@ public bool TryGetUInt64(out ulong value) if (TryGetRawString(out var text) && ulong.TryParse(text, NumberStyles.None, CultureInfo.InvariantCulture, out value) && - string.Equals(FormatUInt64(value), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -512,7 +523,7 @@ public bool TryGetSingle(out float value) float.TryParse(text, NumberStyles.Float, CultureInfo.InvariantCulture, out value) && !float.IsNaN(value) && !float.IsInfinity(value) && - string.Equals(FormatSingle(value), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -560,7 +571,7 @@ public bool TryGetDateTime(out DateTime value) out value ) && value.Kind != DateTimeKind.Local && - string.Equals(FormatDateTime(value), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -600,13 +611,13 @@ out value } private static bool IsCanonicalDateTimeOffsetText(DateTimeOffset value, string text) => - string.Equals(FormatDateTimeOffset(value), text, StringComparison.Ordinal) || + IsCanonical(value, text) || // A zero offset is also written as "Z" by RFC 3339 emitters everywhere, including this library's own // DateTime kind, so it is accepted although the DateTimeOffset writer never produces it. Text without // any designator is still rejected: it parses against the reading machine's local offset and matches // neither form. (value.Offset == TimeSpan.Zero && - string.Equals(FormatDateTime(value.UtcDateTime), text, StringComparison.Ordinal)); + IsCanonical(value.UtcDateTime, text)); #if NET10_0_OR_GREATER /// Attempts to get a date or parse its canonical RFC 3339 full-date encoding. @@ -634,7 +645,7 @@ public bool TryGetDateOnly(out DateOnly value) DateTimeStyles.None, out value ) && - string.Equals(FormatDateOnly(value.DayNumber), text, StringComparison.Ordinal)) + IsCanonicalDate(value.DayNumber, text)) { return true; } @@ -667,7 +678,7 @@ public bool TryGetTimeOnly(out TimeOnly value) DateTimeStyles.None, out value ) && - string.Equals(FormatTimeOnly(value.Ticks), text, StringComparison.Ordinal)) + IsCanonicalTime(value.Ticks, text)) { return true; } @@ -691,7 +702,7 @@ public bool TryGetTimeSpan(out TimeSpan value) try { value = XmlConvert.ToTimeSpan(text); - if (string.Equals(FormatTimeSpan(value), text, StringComparison.Ordinal)) + if (IsCanonical(value, text)) { return true; } @@ -719,7 +730,7 @@ public bool TryGetGuid(out Guid value) if (TryGetRawString(out var text) && Guid.TryParseExact(text, "D", out value) && - string.Equals(FormatGuid(value), text, StringComparison.Ordinal)) + IsCanonical(value, text)) { return true; } @@ -806,24 +817,13 @@ public MetadataObject AsObject() => public string ToCanonicalString() => Kind switch { - MetadataKind.Null => "null", - MetadataKind.Boolean => _payload.Int64 != 0 ? "true" : "false", - MetadataKind.Int64 => _payload.Int64.ToString(CultureInfo.InvariantCulture), - MetadataKind.Double => FormatDouble(_payload.Float64), + MetadataKind.Null => NullCanonicalText, + MetadataKind.Boolean => _payload.Int64 != 0 ? TrueCanonicalText : FalseCanonicalText, MetadataKind.String => GetRequiredReference(), - MetadataKind.Decimal => GetRequiredReference().ToString(CultureInfo.InvariantCulture), - MetadataKind.UInt64 => FormatUInt64(_payload.UInt64), - MetadataKind.Single => FormatSingle((float) _payload.Float64), - MetadataKind.Char => ((char) _payload.Int64).ToString(), - MetadataKind.DateTime => FormatStoredDateTime(_payload.Int64), - MetadataKind.DateTimeOffset => FormatDateTimeOffset(GetRequiredReference()), - MetadataKind.DateOnly => FormatDateOnly(_payload.Int64), - MetadataKind.TimeOnly => FormatTimeOnly(_payload.Int64), - MetadataKind.TimeSpan => FormatTimeSpan(new TimeSpan(_payload.Int64)), - MetadataKind.Guid => FormatGuid(GetRequiredReference()), MetadataKind.Uri => GetRequiredReference().OriginalString, MetadataKind.Array => ThrowComplexCanonicalText(MetadataKind.Array), - MetadataKind.Object => ThrowComplexCanonicalText(MetadataKind.Object) + MetadataKind.Object => ThrowComplexCanonicalText(MetadataKind.Object), + _ => FormatBoundedCanonical(this) }; /// @@ -831,40 +831,185 @@ public string ToCanonicalString() => /// /// The destination for the encoded characters. /// The number of characters written. - /// when the destination was large enough; otherwise, . + /// + /// when the destination was large enough; otherwise, . + /// On failure, is zero and the destination is unchanged. + /// + /// The value is complex or has a malformed payload. public bool TryFormatCanonical(Span destination, out int charsWritten) { - if (Kind == MetadataKind.Double) - { - return CanonicalFloatingPointFormatter.TryFormat( - _payload.Float64, - destination, - out charsWritten - ); - } - - if (Kind == MetadataKind.Single) + switch (Kind) { - return CanonicalFloatingPointFormatter.TryFormat( - (float) _payload.Float64, - destination, - out charsWritten - ); + case MetadataKind.Null: + return TryCopyAscii(NullCanonicalText, destination, out charsWritten); + case MetadataKind.Boolean: + return TryCopyAscii( + _payload.Int64 != 0 ? TrueCanonicalText : FalseCanonicalText, + destination, + out charsWritten + ); + case MetadataKind.Int64: + return CanonicalTextFormatter.TryFormat(_payload.Int64, destination, out charsWritten); + case MetadataKind.Double: + return CanonicalTextFormatter.TryFormat( + _payload.Float64, + destination, + out charsWritten + ); + case MetadataKind.String: + return TryCopyText(GetRequiredReference(), destination, out charsWritten); + case MetadataKind.Decimal: + return CanonicalTextFormatter.TryFormat( + GetRequiredReference(), + destination, + out charsWritten + ); + case MetadataKind.UInt64: + return CanonicalTextFormatter.TryFormat(_payload.UInt64, destination, out charsWritten); + case MetadataKind.Single: + return CanonicalTextFormatter.TryFormat( + (float) _payload.Float64, + destination, + out charsWritten + ); + case MetadataKind.Char: + return CanonicalTextFormatter.TryFormat( + (char) _payload.Int64, + destination, + out charsWritten + ); + case MetadataKind.DateTime: + return TryFormatStoredDateTime(_payload.Int64, destination, out charsWritten); + case MetadataKind.DateTimeOffset: + return CanonicalTextFormatter.TryFormat( + GetRequiredReference(), + destination, + out charsWritten + ); + case MetadataKind.DateOnly: + return TryFormatStoredDate(_payload.Int64, destination, out charsWritten); + case MetadataKind.TimeOnly: + return TryFormatStoredTime(_payload.Int64, destination, out charsWritten); + case MetadataKind.TimeSpan: + return CanonicalTextFormatter.TryFormat( + new TimeSpan(_payload.Int64), + destination, + out charsWritten + ); + case MetadataKind.Guid: + return CanonicalTextFormatter.TryFormat( + GetRequiredReference(), + destination, + out charsWritten + ); + case MetadataKind.Uri: + return TryCopyText( + GetRequiredReference().OriginalString, + destination, + out charsWritten + ); + case MetadataKind.Array: + case MetadataKind.Object: + default: + return ThrowComplexCanonicalFormat(Kind, out charsWritten); } + } - var canonicalText = ToCanonicalString(); - if (canonicalText.AsSpan().TryCopyTo(destination)) + /// + /// Attempts to write the replacement-fallback UTF-8 encoding of the canonical text to the supplied destination. + /// + /// The destination for the encoded bytes. + /// The number of bytes written. + /// + /// when the destination was large enough; otherwise, . + /// On failure, is zero and the destination is unchanged. + /// + /// The value is complex or has a malformed payload. + public bool TryFormatCanonicalUtf8(Span destination, out int bytesWritten) + { + switch (Kind) { - charsWritten = canonicalText.Length; - return true; + case MetadataKind.Null: + return TryCopyAscii(NullCanonicalText, destination, out bytesWritten); + case MetadataKind.Boolean: + return TryCopyAscii( + _payload.Int64 != 0 ? TrueCanonicalText : FalseCanonicalText, + destination, + out bytesWritten + ); + case MetadataKind.Int64: + return CanonicalTextFormatter.TryFormatUtf8(_payload.Int64, destination, out bytesWritten); + case MetadataKind.Double: + return CanonicalTextFormatter.TryFormatUtf8( + _payload.Float64, + destination, + out bytesWritten + ); + case MetadataKind.String: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference().AsSpan(), + destination, + out bytesWritten + ); + case MetadataKind.Decimal: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference(), + destination, + out bytesWritten + ); + case MetadataKind.UInt64: + return CanonicalTextFormatter.TryFormatUtf8(_payload.UInt64, destination, out bytesWritten); + case MetadataKind.Single: + return CanonicalTextFormatter.TryFormatUtf8( + (float) _payload.Float64, + destination, + out bytesWritten + ); + case MetadataKind.Char: + return CanonicalTextFormatter.TryFormatUtf8( + (char) _payload.Int64, + destination, + out bytesWritten + ); + case MetadataKind.DateTime: + return TryFormatStoredDateTimeUtf8(_payload.Int64, destination, out bytesWritten); + case MetadataKind.DateTimeOffset: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference(), + destination, + out bytesWritten + ); + case MetadataKind.DateOnly: + return TryFormatStoredDateUtf8(_payload.Int64, destination, out bytesWritten); + case MetadataKind.TimeOnly: + return TryFormatStoredTimeUtf8(_payload.Int64, destination, out bytesWritten); + case MetadataKind.TimeSpan: + return CanonicalTextFormatter.TryFormatUtf8( + new TimeSpan(_payload.Int64), + destination, + out bytesWritten + ); + case MetadataKind.Guid: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference(), + destination, + out bytesWritten + ); + case MetadataKind.Uri: + return CanonicalTextFormatter.TryFormatUtf8( + GetRequiredReference().OriginalString.AsSpan(), + destination, + out bytesWritten + ); + case MetadataKind.Array: + case MetadataKind.Object: + default: + return ThrowComplexCanonicalFormat(Kind, out bytesWritten); } - - charsWritten = 0; - return false; } internal MetadataValue WithAnnotation(MetadataValueAnnotation annotation) => - new (Kind, _payload, annotation); + new(Kind, _payload, annotation); /// public bool Equals(MetadataValue other) @@ -1017,11 +1162,114 @@ private static void ValidateFinite(double value, string parameterName) } } - private static string FormatUInt64(ulong value) => value.ToString(CultureInfo.InvariantCulture); + private static string FormatBoundedCanonical(MetadataValue value) + { + Span destination = stackalloc char[MaximumPrimitiveCanonicalLength]; + var formatted = value.TryFormatCanonical(destination, out var charsWritten); + Debug.Assert(formatted); + return destination.Slice(0, charsWritten).ToString(); + } + + private static bool TryCopyText( + string text, + Span destination, + out int charsWritten + ) + { + if (!text.AsSpan().TryCopyTo(destination)) + { + charsWritten = 0; + return false; + } + + charsWritten = text.Length; + return true; + } + + private static bool TryCopyAscii( + string text, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if (destination.Length < text.Length) + { + unitsWritten = 0; + return false; + } + + for (var index = 0; index < text.Length; index++) + { + destination[index] = CanonicalCodeUnit.FromAscii((byte) text[index]); + } + + unitsWritten = text.Length; + return true; + } + + private static bool IsCanonical(long value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumInt64Length]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(ulong value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumUInt64Length]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(float value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(DateTime value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(DateTimeOffset value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateTimeOffsetLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } - private static string FormatSingle(float value) => CanonicalFloatingPointFormatter.Format(value); + private static bool IsCanonicalDate(int dayNumber, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDateLength]; + return CanonicalTextFormatter.TryFormatDate(dayNumber, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } - private static string FormatDouble(double value) => CanonicalFloatingPointFormatter.Format(value); + private static bool IsCanonicalTime(long ticks, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumTimeLength]; + return CanonicalTextFormatter.TryFormatTime(ticks, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(TimeSpan value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumTimeSpanLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } + + private static bool IsCanonical(Guid value, string text) + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumGuidLength]; + return CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) && + destination.Slice(0, charsWritten).SequenceEqual(text.AsSpan()); + } private static bool TryReadStoredDateTime(long binaryValue, out DateTime value) { @@ -1047,47 +1295,88 @@ private static bool TryReadStoredDateTime(long binaryValue, out DateTime value) } } - private static string FormatStoredDateTime(long binaryValue) => + private static bool TryFormatStoredDateTime( + long binaryValue, + Span destination, + out int charsWritten + ) => TryReadStoredDateTime(binaryValue, out var value) ? - FormatDateTime(value) : + CanonicalTextFormatter.TryFormat(value, destination, out charsWritten) : throw new InvalidOperationException("The DateTime metadata payload is invalid."); - private static string FormatDateTime(DateTime value) => - value.ToString(DateTimeFormat, CultureInfo.InvariantCulture); - - private static string FormatDateTimeOffset(DateTimeOffset value) => - value.ToString(DateTimeOffsetFormat, CultureInfo.InvariantCulture); + private static bool TryFormatStoredDateTimeUtf8( + long binaryValue, + Span destination, + out int bytesWritten + ) => + TryReadStoredDateTime(binaryValue, out var value) ? + CanonicalTextFormatter.TryFormatUtf8(value, destination, out bytesWritten) : + throw new InvalidOperationException("The DateTime metadata payload is invalid."); - private static string FormatDateOnly(long dayNumber) + private static bool TryFormatStoredDate( + long dayNumber, + Span destination, + out int charsWritten + ) { - try + if ((ulong) dayNumber > CanonicalTextFormatter.MaximumDayNumber) { - var ticks = checked(dayNumber * TimeSpan.TicksPerDay); - return new DateTime(ticks, DateTimeKind.Unspecified).ToString( - DateOnlyFormat, - CultureInfo.InvariantCulture - ); + throw new InvalidOperationException("The DateOnly metadata payload is invalid."); } - catch (Exception exception) when (exception is ArgumentOutOfRangeException or OverflowException) + + return CanonicalTextFormatter.TryFormatDate((int) dayNumber, destination, out charsWritten); + } + + private static bool TryFormatStoredDateUtf8( + long dayNumber, + Span destination, + out int bytesWritten + ) + { + if ((ulong) dayNumber > CanonicalTextFormatter.MaximumDayNumber) { - throw new InvalidOperationException("The DateOnly metadata payload is invalid.", exception); + throw new InvalidOperationException("The DateOnly metadata payload is invalid."); } + + return CanonicalTextFormatter.TryFormatDateUtf8((int) dayNumber, destination, out bytesWritten); } - private static string FormatTimeOnly(long ticks) + private static bool TryFormatStoredTime( + long ticks, + Span destination, + out int charsWritten + ) { - if (ticks < 0 || ticks >= TimeSpan.TicksPerDay) + if ((ulong) ticks > (ulong) CanonicalTextFormatter.MaximumTimeOfDayTicks) { throw new InvalidOperationException("The TimeOnly metadata payload is invalid."); } - return new DateTime(ticks, DateTimeKind.Unspecified).ToString(TimeOnlyFormat, CultureInfo.InvariantCulture); + return CanonicalTextFormatter.TryFormatTime(ticks, destination, out charsWritten); } - private static string FormatTimeSpan(TimeSpan value) => XmlConvert.ToString(value); + private static bool TryFormatStoredTimeUtf8( + long ticks, + Span destination, + out int bytesWritten + ) + { + if ((ulong) ticks > (ulong) CanonicalTextFormatter.MaximumTimeOfDayTicks) + { + throw new InvalidOperationException("The TimeOnly metadata payload is invalid."); + } + + return CanonicalTextFormatter.TryFormatTimeUtf8(ticks, destination, out bytesWritten); + } - // The "D" format is specified to produce lowercase hexadecimal digits, so no additional lowering is needed. - private static string FormatGuid(Guid value) => value.ToString("D", CultureInfo.InvariantCulture); + private static bool ThrowComplexCanonicalFormat( + MetadataKind kind, + out int unitsWritten + ) + { + unitsWritten = 0; + throw new InvalidOperationException($"Kind '{kind}' does not have a primitive canonical text encoding."); + } private static string ThrowComplexCanonicalText(MetadataKind kind) => throw new InvalidOperationException($"Kind '{kind}' does not have a primitive canonical text encoding."); diff --git a/src/Light.PortableResults/Numbers/README.md b/src/Light.PortableResults/Numbers/README.md index 00d53c5..3ffd686 100644 --- a/src/Light.PortableResults/Numbers/README.md +++ b/src/Light.PortableResults/Numbers/README.md @@ -15,8 +15,9 @@ The following upstream files under - `Number.NumberBuffer.cs` - `Number.Formatting.cs` (IEEE bit extraction and the floating-point-to-number dispatch) -Files containing adapted runtime code retain the .NET Foundation MIT header. The public -`CanonicalFloatingPointFormatter` and its invariant renderer are Light.PortableResults code. +Files containing adapted runtime code retain the .NET Foundation MIT header. The floating-point +portion of the public `CanonicalTextFormatter` and its invariant renderer are +Light.PortableResults code. ## Adaptations @@ -47,8 +48,8 @@ Files containing adapted runtime code retain the .NET Foundation MIT header. The ## Retained code that shortest-unique mode cannot reach -Two small regions survive from upstream that no input can execute through -`CanonicalFloatingPointFormatter`. They are kept rather than deleted because they belong to the +Two small regions survive from upstream that no input can execute through the floating-point +overloads on `CanonicalTextFormatter`. They are kept rather than deleted because they belong to the published shape of these algorithms, and cutting into them would make the remaining source harder to compare against its pinned origin. Neither is excluded from code coverage. The reasoning below records why the test suite cannot cover them, so a later reader does not mistake them for a gap. diff --git a/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs b/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs index f7ee235..7cdd557 100644 --- a/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs +++ b/src/Light.PortableResults/SharedJsonSerialization/Writing/MetadataExtensions.cs @@ -1,4 +1,5 @@ using System; +using System.Diagnostics; using System.Text.Json; using Light.PortableResults.Metadata; @@ -66,7 +67,7 @@ MetadataValueAnnotation requiredAnnotation WriteNumberValue(writer, value); break; case MetadataJsonShape.String: - writer.WriteStringValue(value.ToCanonicalString()); + WriteStringValue(writer, value); break; case MetadataJsonShape.Array: value.TryGetArray(out var arrayMetadataValue); @@ -98,7 +99,10 @@ private static void WriteNumberValue(Utf8JsonWriter writer, MetadataValue value) // back as an Int64. The canonical text is therefore built and written raw. Validation is // skipped because the text comes from our own formatter and non-finite values are rejected at // construction, so it is always a well-formed JSON number. - writer.WriteRawValue(value.ToCanonicalString(), skipInputValidation: true); + Span canonicalNumber = stackalloc byte[MetadataValue.MaximumPrimitiveCanonicalLength]; + var formatted = value.TryFormatCanonicalUtf8(canonicalNumber, out var bytesWritten); + Debug.Assert(formatted); + writer.WriteRawValue(canonicalNumber.Slice(0, bytesWritten), skipInputValidation: true); return; case MetadataNumberEncoding.None: throw new InvalidOperationException( @@ -107,6 +111,32 @@ private static void WriteNumberValue(Utf8JsonWriter writer, MetadataValue value) } } + private static void WriteStringValue(Utf8JsonWriter writer, MetadataValue value) + { + switch (value.Kind) + { + case MetadataKind.String: + case MetadataKind.Uri: + // Keep text-bearing values on the UTF-16 writer route. Besides avoiding a redundant + // transcode, this preserves the configured encoder's handling of malformed UTF-16. + writer.WriteStringValue(value.ToCanonicalString()); + return; + case MetadataKind.Char: + value.TryGetChar(out var character); + Span characterText = stackalloc char[1]; + characterText[0] = character; + writer.WriteStringValue(characterText); + return; + default: + // Every remaining string-shaped kind has bounded, entirely ASCII canonical text. + Span canonicalText = stackalloc byte[MetadataValue.MaximumPrimitiveCanonicalLength]; + var formatted = value.TryFormatCanonicalUtf8(canonicalText, out var bytesWritten); + Debug.Assert(formatted); + writer.WriteStringValue(canonicalText.Slice(0, bytesWritten)); + return; + } + } + /// /// Writes the JSON representation for the specified metadata array. /// diff --git a/src/Light.PortableResults/Text/CanonicalCodeUnit.cs b/src/Light.PortableResults/Text/CanonicalCodeUnit.cs new file mode 100644 index 0000000..0f0252a --- /dev/null +++ b/src/Light.PortableResults/Text/CanonicalCodeUnit.cs @@ -0,0 +1,39 @@ +using System; +using System.Runtime.CompilerServices; + +namespace Light.PortableResults.Text; + +/// +/// Converts ASCII bytes to the UTF-8 or UTF-16 code units used by canonical text formatters. +/// +/// +/// This low-level helper lets one generic renderer target either or +/// destinations. Other code-unit types are not supported. +/// +public static class CanonicalCodeUnit +{ + /// Converts an ASCII byte to a UTF-8 byte or UTF-16 character code unit. + /// The destination code-unit type, either or . + /// The ASCII byte to convert. + /// The equivalent code unit. + /// + /// is neither nor . + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TCodeUnit FromAscii(byte value) + where TCodeUnit : unmanaged + { + if (typeof(TCodeUnit) == typeof(byte)) + { + return Unsafe.As(ref value); + } + + if (typeof(TCodeUnit) == typeof(char)) + { + var character = (char) value; + return Unsafe.As(ref character); + } + + throw new NotSupportedException("Only byte and char code units are supported."); + } +} diff --git a/src/Light.PortableResults/Numbers/CanonicalFloatingPointFormatter.cs b/src/Light.PortableResults/Text/CanonicalTextFormatter.FloatingPoint.cs similarity index 86% rename from src/Light.PortableResults/Numbers/CanonicalFloatingPointFormatter.cs rename to src/Light.PortableResults/Text/CanonicalTextFormatter.FloatingPoint.cs index 3e9db0e..eab5be4 100644 --- a/src/Light.PortableResults/Numbers/CanonicalFloatingPointFormatter.cs +++ b/src/Light.PortableResults/Text/CanonicalTextFormatter.FloatingPoint.cs @@ -1,7 +1,7 @@ using System; -using System.Runtime.CompilerServices; +using Light.PortableResults.Numbers; -namespace Light.PortableResults.Numbers; +namespace Light.PortableResults.Text; /// /// Formats finite IEEE 754 binary floating-point values with one runtime-independent invariant encoding. @@ -19,7 +19,7 @@ namespace Light.PortableResults.Numbers; /// including zero, end in .0 so their floating-point JSON shape is unambiguous. /// /// -public static class CanonicalFloatingPointFormatter +public static partial class CanonicalTextFormatter { /// /// A destination size that is always large enough to hold the canonical encoding of any finite @@ -226,30 +226,32 @@ out int unitsWritten var index = 0; if (isNegative) { - destination[index++] = ToCodeUnit((byte) '-'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); } if (useScientificNotation) { - destination[index++] = ToCodeUnit(digits[0]); + destination[index++] = CanonicalCodeUnit.FromAscii(digits[0]); if (digits.Length > 1) { - destination[index++] = ToCodeUnit((byte) '.'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); CopyDigits(digits[1..], destination, ref index); } - destination[index++] = ToCodeUnit((byte) 'E'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'E'); var exponent = scale - 1; - destination[index++] = ToCodeUnit(exponent < 0 ? (byte) '-' : (byte) '+'); + destination[index++] = CanonicalCodeUnit.FromAscii( + exponent < 0 ? (byte) '-' : (byte) '+' + ); WriteExponent(exponent < 0 ? -exponent : exponent, destination, ref index); } else if (scale <= 0) { - destination[index++] = ToCodeUnit((byte) '0'); - destination[index++] = ToCodeUnit((byte) '.'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); for (var zeroIndex = 0; zeroIndex < -scale; zeroIndex++) { - destination[index++] = ToCodeUnit((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); } CopyDigits(digits, destination, ref index); @@ -257,7 +259,7 @@ out int unitsWritten else if (scale < digits.Length) { CopyDigits(digits[..scale], destination, ref index); - destination[index++] = ToCodeUnit((byte) '.'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); CopyDigits(digits[scale..], destination, ref index); } else @@ -265,11 +267,11 @@ out int unitsWritten CopyDigits(digits, destination, ref index); for (var zeroIndex = digits.Length; zeroIndex < scale; zeroIndex++) { - destination[index++] = ToCodeUnit((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); } - destination[index++] = ToCodeUnit((byte) '.'); - destination[index++] = ToCodeUnit((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); } unitsWritten = index; @@ -285,7 +287,7 @@ ref int destinationIndex { for (var index = 0; index < digits.Length; index++) { - destination[destinationIndex++] = ToCodeUnit(digits[index]); + destination[destinationIndex++] = CanonicalCodeUnit.FromAscii(digits[index]); } } @@ -298,24 +300,17 @@ ref int index { if (exponent >= 100) { - destination[index++] = ToCodeUnit((byte) ('0' + exponent / 100)); + destination[index++] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + exponent / 100) + ); exponent %= 100; } - destination[index++] = ToCodeUnit((byte) ('0' + exponent / 10)); - destination[index++] = ToCodeUnit((byte) ('0' + exponent % 10)); - } - - [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static TCodeUnit ToCodeUnit(byte value) - where TCodeUnit : unmanaged - { - if (typeof(TCodeUnit) == typeof(byte)) - { - return Unsafe.As(ref value); - } - - var character = (char) value; - return Unsafe.As(ref character); + destination[index++] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + exponent / 10) + ); + destination[index++] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + exponent % 10) + ); } } diff --git a/src/Light.PortableResults/Text/CanonicalTextFormatter.cs b/src/Light.PortableResults/Text/CanonicalTextFormatter.cs new file mode 100644 index 0000000..c40ac76 --- /dev/null +++ b/src/Light.PortableResults/Text/CanonicalTextFormatter.cs @@ -0,0 +1,1100 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +using System; +using System.Runtime.CompilerServices; +#if !NET10_0_OR_GREATER +using System.Text; +#endif +#if NET10_0_OR_GREATER +using System.Text.Unicode; +#endif + +namespace Light.PortableResults.Text; + +/// +/// Formats primitive values with the culture-independent canonical encodings used by metadata. +/// +/// +/// Every maximum-length constant bounds both UTF-16 characters and UTF-8 bytes. Except for +/// , the corresponding canonical alphabet is ASCII, so the counts +/// in both encodings are equal. +/// +public static partial class CanonicalTextFormatter +{ + /// The maximum canonical length of a signed 64-bit integer. + public const int MaximumInt64Length = 20; + + /// The maximum canonical length of an unsigned 64-bit integer. + public const int MaximumUInt64Length = 20; + + /// The maximum canonical length of a decimal value. + public const int MaximumDecimalLength = 31; + + /// The maximum canonical length of one UTF-16 code unit in either encoding. + public const int MaximumCharLength = 3; + + /// The largest valid day number, equal to DateOnly.MaxValue.DayNumber. + public const int MaximumDayNumber = 3_652_058; + + /// The largest valid time-of-day tick count. + public const long MaximumTimeOfDayTicks = TimeSpan.TicksPerDay - 1; + + /// The maximum canonical length of a . + public const int MaximumDateTimeLength = 28; + + /// The maximum canonical length of a . + public const int MaximumDateTimeOffsetLength = 33; + + /// The canonical length of a date. + public const int MaximumDateLength = 10; + + /// The maximum canonical length of a time of day. + public const int MaximumTimeLength = 16; + + /// The maximum canonical length of a . + public const int MaximumTimeSpanLength = 27; + + /// The canonical length of a GUID in lowercase D format. + public const int MaximumGuidLength = 36; + + private const uint OneBillion = 1_000_000_000; + +#if !NET10_0_OR_GREATER + // The netstandard2.0 asset has no allocation-free decimal.GetBits overload. Its fallback relies + // on the historical little-endian decimal layout used by .NET Framework, .NET, and Mono: + // flags, high, low, middle. Keep the check non-throwing so unrelated formatters remain usable. + private static readonly bool IsLegacyDecimalLayoutSupported = ValidateLegacyDecimalLayout(); +#endif + + /// Attempts to format one UTF-16 code unit. + /// on success; otherwise . + public static bool TryFormat(char value, Span destination, out int charsWritten) + { + if (destination.IsEmpty) + { + charsWritten = 0; + return false; + } + + destination[0] = value; + charsWritten = 1; + return true; + } + + /// Attempts to format a signed 64-bit integer. + /// on success; otherwise . + public static bool TryFormat(long value, Span destination, out int charsWritten) => + TryFormatInt64(value, destination, out charsWritten); + + /// Attempts to format an unsigned 64-bit integer. + /// on success; otherwise . + public static bool TryFormat(ulong value, Span destination, out int charsWritten) => + TryFormatUInt64(value, destination, out charsWritten); + + /// Attempts to format a decimal while preserving its scale. + /// on success; otherwise . + /// + /// The netstandard2.0 asset is running on a platform whose decimal layout is unsupported. + /// + public static bool TryFormat(decimal value, Span destination, out int charsWritten) => + TryFormatDecimal(value, destination, out charsWritten); + + /// + /// Attempts to format a date and time. Local values are converted to UTC; UTC and unspecified + /// values are unchanged. + /// + /// Local conversion depends on the machine's local time zone. + /// on success; otherwise . + public static bool TryFormat(DateTime value, Span destination, out int charsWritten) => + TryFormatDateTime(value, destination, out charsWritten); + + /// Attempts to format a date and time with an offset. + /// on success; otherwise . + public static bool TryFormat( + DateTimeOffset value, + Span destination, + out int charsWritten + ) => + TryFormatDateTimeOffset(value, destination, out charsWritten); + + /// Attempts to format an XML Schema duration. + /// on success; otherwise . + public static bool TryFormat(TimeSpan value, Span destination, out int charsWritten) => + TryFormatTimeSpan(value, destination, out charsWritten); + + /// Attempts to format a GUID in lowercase D format. + /// on success; otherwise . + public static bool TryFormat(Guid value, Span destination, out int charsWritten) => + TryFormatGuid(value, destination, out charsWritten); + + /// Attempts to format a zero-based Gregorian day number as a date. + /// on success; otherwise . + /// is invalid. + public static bool TryFormatDate( + int dayNumber, + Span destination, + out int charsWritten + ) => + TryFormatDateNumber(dayNumber, destination, out charsWritten); + + /// Attempts to format a time-of-day tick count. + /// on success; otherwise . + /// is invalid. + public static bool TryFormatTime( + long ticks, + Span destination, + out int charsWritten + ) => + TryFormatTimeOfDay(ticks, destination, out charsWritten); + + /// Attempts to format one UTF-16 code unit as replacement-fallback UTF-8. + /// on success; otherwise . + public static unsafe bool TryFormatUtf8( + char value, + Span destination, + out int bytesWritten + ) + { + var pointer = &value; + return TryFormatUtf8(new ReadOnlySpan(pointer, 1), destination, out bytesWritten); + } + + /// Attempts to format a signed 64-bit integer as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8(long value, Span destination, out int bytesWritten) => + TryFormatInt64(value, destination, out bytesWritten); + + /// Attempts to format an unsigned 64-bit integer as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8(ulong value, Span destination, out int bytesWritten) => + TryFormatUInt64(value, destination, out bytesWritten); + + /// Attempts to format a decimal as UTF-8 while preserving its scale. + /// on success; otherwise . + /// + /// The netstandard2.0 asset is running on a platform whose decimal layout is unsupported. + /// + public static bool TryFormatUtf8(decimal value, Span destination, out int bytesWritten) => + TryFormatDecimal(value, destination, out bytesWritten); + + /// + /// Attempts to format a date and time as UTF-8. Local values are converted to UTC; UTC and + /// unspecified values are unchanged. + /// + /// Local conversion depends on the machine's local time zone. + /// on success; otherwise . + public static bool TryFormatUtf8( + DateTime value, + Span destination, + out int bytesWritten + ) => + TryFormatDateTime(value, destination, out bytesWritten); + + /// Attempts to format a date and time with an offset as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8( + DateTimeOffset value, + Span destination, + out int bytesWritten + ) => + TryFormatDateTimeOffset(value, destination, out bytesWritten); + + /// Attempts to format an XML Schema duration as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8( + TimeSpan value, + Span destination, + out int bytesWritten + ) => + TryFormatTimeSpan(value, destination, out bytesWritten); + + /// Attempts to format a GUID in lowercase D format as UTF-8. + /// on success; otherwise . + public static bool TryFormatUtf8(Guid value, Span destination, out int bytesWritten) => + TryFormatGuid(value, destination, out bytesWritten); + + /// Attempts to format a zero-based Gregorian day number as a UTF-8 date. + /// on success; otherwise . + /// is invalid. + public static bool TryFormatDateUtf8( + int dayNumber, + Span destination, + out int bytesWritten + ) => + TryFormatDateNumber(dayNumber, destination, out bytesWritten); + + /// Attempts to format a time-of-day tick count as UTF-8. + /// on success; otherwise . + /// is invalid. + public static bool TryFormatTimeUtf8( + long ticks, + Span destination, + out int bytesWritten + ) => + TryFormatTimeOfDay(ticks, destination, out bytesWritten); + + /// + /// Attempts to transcode UTF-16 text to UTF-8 with replacement fallback for malformed sequences. + /// + /// + /// on success; otherwise . On failure, + /// is zero and the destination is unchanged. + /// + public static unsafe bool TryFormatUtf8( + ReadOnlySpan text, + Span destination, + out int bytesWritten + ) + { + if (text.IsEmpty) + { + bytesWritten = 0; + return true; + } + + var requiredLength = GetUtf8ByteCountWithReplacement(text); + if (destination.Length < requiredLength) + { + bytesWritten = 0; + return false; + } + +#if NET10_0_OR_GREATER + Utf8.FromUtf16( + text, + destination, + out _, + out bytesWritten, + replaceInvalidSequences: true, + isFinalBlock: true + ); + return true; +#else + fixed (char* textPointer = text) + { + fixed (byte* destinationPointer = destination) + { + bytesWritten = TranscodeUtf16WithReplacement( + textPointer, + text.Length, + destinationPointer, + destination.Length + ); + return true; + } + } +#endif + } + + private static long GetUtf8ByteCountWithReplacement(ReadOnlySpan text) + { + long count = 0; + for (var index = 0; index < text.Length; index++) + { + var character = text[index]; + if (character <= 0x7F) + { + count++; + } + else if (character <= 0x7FF) + { + count += 2; + } + else if (character is >= '\uD800' and <= '\uDBFF' && + index + 1 < text.Length && + text[index + 1] is >= '\uDC00' and <= '\uDFFF') + { + count += 4; + index++; + } + else + { + count += 3; + } + } + + return count; + } + +#if !NET10_0_OR_GREATER + private static unsafe int TranscodeUtf16WithReplacement( + char* text, + int textLength, + byte* destination, + int destinationLength + ) + { + var segmentStart = 0; + var destinationIndex = 0; + for (var index = 0; index < textLength; index++) + { + var scalar = (int) text[index]; + if (scalar is >= '\uD800' and <= '\uDBFF' && + index + 1 < textLength && + text[index + 1] is >= '\uDC00' and <= '\uDFFF') + { + index++; + continue; + } + + if (scalar is not (>= '\uD800' and <= '\uDFFF')) + { + continue; + } + + destinationIndex += Encoding.UTF8.GetBytes( + text + segmentStart, + index - segmentStart, + destination + destinationIndex, + destinationLength - destinationIndex + ); + destination[destinationIndex++] = 0xEF; + destination[destinationIndex++] = 0xBF; + destination[destinationIndex++] = 0xBD; + segmentStart = index + 1; + } + + destinationIndex += Encoding.UTF8.GetBytes( + text + segmentStart, + textLength - segmentStart, + destination + destinationIndex, + destinationLength - destinationIndex + ); + return destinationIndex; + } +#endif + + private static bool TryFormatInt64( + long value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + var isNegative = value < 0; + var magnitude = isNegative ? unchecked((ulong) -value) : (ulong) value; + var requiredLength = CountDigits(magnitude) + (isNegative ? 1 : 0); + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + WriteUInt64Digits(magnitude, destination, requiredLength, isNegative ? 1 : 0); + if (isNegative) + { + destination[0] = CanonicalCodeUnit.FromAscii((byte) '-'); + } + + unitsWritten = requiredLength; + return true; + } + + private static bool TryFormatUInt64( + ulong value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + var requiredLength = CountDigits(value); + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + WriteUInt64Digits(value, destination, requiredLength, 0); + unitsWritten = requiredLength; + return true; + } + + private static bool TryFormatDecimal( + decimal value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + GetDecimalFields(value, out var low, out var middle, out var high, out var flags); + var scale = (int) ((flags >> 16) & 0xFF); + var isZero = (low | middle | high) == 0; + var isNegative = !isZero && (flags & 0x80000000U) != 0; + var digitCount = isZero ? 1 : CountDecimalDigits(low, middle, high); + + int requiredLength; + if (scale == 0) + { + requiredLength = digitCount + (isNegative ? 1 : 0); + } + else if (scale >= digitCount) + { + requiredLength = scale + 2 + (isNegative ? 1 : 0); + } + else + { + requiredLength = digitCount + 1 + (isNegative ? 1 : 0); + } + + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + var prefixLength = isNegative ? 1 : 0; + if (isNegative) + { + destination[0] = CanonicalCodeUnit.FromAscii((byte) '-'); + } + + int digitStart; + var decimalPointIndex = -1; + if (scale == 0) + { + digitStart = prefixLength; + } + else if (scale >= digitCount) + { + destination[prefixLength] = CanonicalCodeUnit.FromAscii((byte) '0'); + destination[prefixLength + 1] = CanonicalCodeUnit.FromAscii((byte) '.'); + digitStart = prefixLength + 2 + scale - digitCount; + for (var index = prefixLength + 2; index < digitStart; index++) + { + destination[index] = CanonicalCodeUnit.FromAscii((byte) '0'); + } + } + else + { + decimalPointIndex = prefixLength + digitCount - scale; + destination[decimalPointIndex] = CanonicalCodeUnit.FromAscii((byte) '.'); + digitStart = prefixLength; + } + + WriteDecimalDigits( + low, + middle, + high, + destination, + digitStart, + digitCount, + decimalPointIndex + ); + unitsWritten = requiredLength; + return true; + } + + private static bool TryFormatDateTime( + DateTime value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if (value.Kind == DateTimeKind.Local) + { + value = value.ToUniversalTime(); + } + + return TryFormatDateAndTime( + value, + value.Kind == DateTimeKind.Utc, + default, + hasOffset: false, + destination, + out unitsWritten + ); + } + + private static bool TryFormatDateTimeOffset( + DateTimeOffset value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged => + TryFormatDateAndTime( + value.DateTime, + appendUtcDesignator: false, + value.Offset, + hasOffset: true, + destination, + out unitsWritten + ); + + private static bool TryFormatDateAndTime( + DateTime value, + bool appendUtcDesignator, + TimeSpan offset, + bool hasOffset, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + var fraction = value.Ticks % TimeSpan.TicksPerSecond; + var fractionDigits = CountFractionDigits(fraction); + var requiredLength = 19 + + (fractionDigits == 0 ? 0 : fractionDigits + 1) + + (appendUtcDesignator ? 1 : 0) + + (hasOffset ? 6 : 0); + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + WriteFourDigits((uint) value.Year, destination, 0); + destination[4] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteTwoDigits((uint) value.Month, destination, 5); + destination[7] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteTwoDigits((uint) value.Day, destination, 8); + destination[10] = CanonicalCodeUnit.FromAscii((byte) 'T'); + WriteTwoDigits((uint) value.Hour, destination, 11); + destination[13] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) value.Minute, destination, 14); + destination[16] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) value.Second, destination, 17); + + var index = 19; + if (fractionDigits != 0) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); + WriteFraction(fraction, fractionDigits, destination, index); + index += fractionDigits; + } + + if (appendUtcDesignator) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'Z'); + } + else if (hasOffset) + { + var totalMinutes = (int) (offset.Ticks / TimeSpan.TicksPerMinute); + destination[index++] = CanonicalCodeUnit.FromAscii( + totalMinutes < 0 ? (byte) '-' : (byte) '+' + ); + if (totalMinutes < 0) + { + totalMinutes = -totalMinutes; + } + + WriteTwoDigits((uint) (totalMinutes / 60), destination, index); + index += 2; + destination[index++] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) (totalMinutes % 60), destination, index); + index += 2; + } + + unitsWritten = index; + return true; + } + + private static bool TryFormatDateNumber( + int dayNumber, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if ((uint) dayNumber > MaximumDayNumber) + { + throw new ArgumentOutOfRangeException(nameof(dayNumber)); + } + + if (destination.Length < MaximumDateLength) + { + unitsWritten = 0; + return false; + } + + var value = new DateTime((long) dayNumber * TimeSpan.TicksPerDay, DateTimeKind.Unspecified); + WriteFourDigits((uint) value.Year, destination, 0); + destination[4] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteTwoDigits((uint) value.Month, destination, 5); + destination[7] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteTwoDigits((uint) value.Day, destination, 8); + unitsWritten = MaximumDateLength; + return true; + } + + private static bool TryFormatTimeOfDay( + long ticks, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if ((ulong) ticks > (ulong) MaximumTimeOfDayTicks) + { + throw new ArgumentOutOfRangeException(nameof(ticks)); + } + + var fraction = ticks % TimeSpan.TicksPerSecond; + var fractionDigits = CountFractionDigits(fraction); + var requiredLength = 8 + (fractionDigits == 0 ? 0 : fractionDigits + 1); + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + var hours = ticks / TimeSpan.TicksPerHour; + var minutes = ticks / TimeSpan.TicksPerMinute % 60; + var seconds = ticks / TimeSpan.TicksPerSecond % 60; + WriteTwoDigits((uint) hours, destination, 0); + destination[2] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) minutes, destination, 3); + destination[5] = CanonicalCodeUnit.FromAscii((byte) ':'); + WriteTwoDigits((uint) seconds, destination, 6); + + var index = 8; + if (fractionDigits != 0) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); + WriteFraction(fraction, fractionDigits, destination, index); + index += fractionDigits; + } + + unitsWritten = index; + return true; + } + + private static bool TryFormatTimeSpan( + TimeSpan value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + var ticks = value.Ticks; + var isNegative = ticks < 0; + var magnitude = isNegative ? unchecked((ulong) -ticks) : (ulong) ticks; + var days = magnitude / (ulong) TimeSpan.TicksPerDay; + var hours = magnitude / (ulong) TimeSpan.TicksPerHour % 24; + var minutes = magnitude / (ulong) TimeSpan.TicksPerMinute % 60; + var seconds = magnitude / (ulong) TimeSpan.TicksPerSecond % 60; + var fraction = (long) (magnitude % (ulong) TimeSpan.TicksPerSecond); + var fractionDigits = CountFractionDigits(fraction); + var hasTime = hours != 0 || minutes != 0 || seconds != 0 || fraction != 0; + + var requiredLength = (isNegative ? 1 : 0) + 1; + if (days != 0) + { + requiredLength += CountDigits(days) + 1; + } + + if (hasTime) + { + requiredLength++; + if (hours != 0) + { + requiredLength += CountDigits(hours) + 1; + } + + if (minutes != 0) + { + requiredLength += CountDigits(minutes) + 1; + } + + if (seconds != 0 || fraction != 0) + { + requiredLength += CountDigits(seconds) + 1; + if (fractionDigits != 0) + { + requiredLength += fractionDigits + 1; + } + } + } + else if (days == 0) + { + requiredLength += 3; + } + + if (destination.Length < requiredLength) + { + unitsWritten = 0; + return false; + } + + var index = 0; + if (isNegative) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + } + + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'P'); + if (days != 0) + { + WriteUnsignedComponent(days, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'D'); + } + + if (hasTime) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'T'); + if (hours != 0) + { + WriteUnsignedComponent(hours, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'H'); + } + + if (minutes != 0) + { + WriteUnsignedComponent(minutes, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'M'); + } + + if (seconds != 0 || fraction != 0) + { + WriteUnsignedComponent(seconds, destination, ref index); + if (fractionDigits != 0) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '.'); + WriteFraction(fraction, fractionDigits, destination, index); + index += fractionDigits; + } + + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'S'); + } + } + else if (days == 0) + { + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'T'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '0'); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) 'S'); + } + + unitsWritten = index; + return true; + } + + private static bool TryFormatGuid( + Guid value, + Span destination, + out int unitsWritten + ) + where TCodeUnit : unmanaged + { + if (destination.Length < MaximumGuidLength) + { + unitsWritten = 0; + return false; + } + + ref var fields = ref Unsafe.As(ref value); + var index = 0; + WriteHexByte((byte) ((uint) fields.A >> 24), destination, ref index); + WriteHexByte((byte) ((uint) fields.A >> 16), destination, ref index); + WriteHexByte((byte) ((uint) fields.A >> 8), destination, ref index); + WriteHexByte((byte) fields.A, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteHexByte((byte) ((ushort) fields.B >> 8), destination, ref index); + WriteHexByte((byte) fields.B, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteHexByte((byte) ((ushort) fields.C >> 8), destination, ref index); + WriteHexByte((byte) fields.C, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteHexByte(fields.D, destination, ref index); + WriteHexByte(fields.E, destination, ref index); + destination[index++] = CanonicalCodeUnit.FromAscii((byte) '-'); + WriteHexByte(fields.F, destination, ref index); + WriteHexByte(fields.G, destination, ref index); + WriteHexByte(fields.H, destination, ref index); + WriteHexByte(fields.I, destination, ref index); + WriteHexByte(fields.J, destination, ref index); + WriteHexByte(fields.K, destination, ref index); + unitsWritten = index; + return true; + } + + private static void WriteUInt64Digits( + ulong value, + Span destination, + int end, + int start + ) + where TCodeUnit : unmanaged + { + var index = end; + while (value >= OneBillion) + { + var remainder = (uint) (value % OneBillion); + value /= OneBillion; + WriteUInt32Digits(remainder, destination, ref index, 9); + } + + WriteUInt32Digits((uint) value, destination, ref index, index - start); + } + + private static void WriteUInt32Digits( + uint value, + Span destination, + ref int end, + int digits + ) + where TCodeUnit : unmanaged + { + while (digits-- > 0) + { + var quotient = value / 10; + destination[--end] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + value - quotient * 10) + ); + value = quotient; + } + } + + private static void WriteDecimalDigits( + uint low, + uint middle, + uint high, + Span destination, + int digitStart, + int digitCount, + int decimalPointIndex + ) + where TCodeUnit : unmanaged + { + var index = digitStart + digitCount + (decimalPointIndex >= 0 ? 1 : 0); + var remainingDigits = digitCount; + do + { + var group = DivideDecimalByOneBillion(ref low, ref middle, ref high); + var groupDigits = remainingDigits > 9 ? 9 : remainingDigits; + for (var digit = 0; digit < groupDigits; digit++) + { + if (index - 1 == decimalPointIndex) + { + index--; + } + + var quotient = group / 10; + destination[--index] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + group - quotient * 10) + ); + group = quotient; + } + + remainingDigits -= groupDigits; + } + while (remainingDigits != 0); + } + + private static int CountDecimalDigits(uint low, uint middle, uint high) + { + var groupCount = 0; + uint mostSignificantGroup; + do + { + mostSignificantGroup = DivideDecimalByOneBillion(ref low, ref middle, ref high); + groupCount++; + } + while ((low | middle | high) != 0); + + return (groupCount - 1) * 9 + CountDigits(mostSignificantGroup); + } + + private static uint DivideDecimalByOneBillion(ref uint low, ref uint middle, ref uint high) + { + var high64 = ((ulong) high << 32) + middle; + var quotient64 = high64 / OneBillion; + high = (uint) (quotient64 >> 32); + middle = (uint) quotient64; + + var value = ((high64 - (uint) quotient64 * OneBillion) << 32) + low; + var quotient = (uint) (value / OneBillion); + low = quotient; + return (uint) value - quotient * OneBillion; + } + + private static void GetDecimalFields( + decimal value, + out uint low, + out uint middle, + out uint high, + out uint flags + ) + { +#if NET10_0_OR_GREATER + Span bits = stackalloc int[4]; + decimal.GetBits(value, bits); + low = (uint) bits[0]; + middle = (uint) bits[1]; + high = (uint) bits[2]; + flags = (uint) bits[3]; +#else + if (!IsLegacyDecimalLayoutSupported) + { + throw new PlatformNotSupportedException( + "Allocation-free decimal formatting requires a little-endian runtime with the historical decimal field layout." + ); + } + + ref var layout = ref Unsafe.As(ref value); + low = layout.Low; + middle = layout.Middle; + high = layout.High; + flags = layout.Flags; +#endif + } + +#if !NET10_0_OR_GREATER + private static bool ValidateLegacyDecimalLayout() + { + var probe = new decimal( + lo: 0x11111111, + mid: 0x22222222, + hi: 0x33333333, + isNegative: true, + scale: 5 + ); + ref var layout = ref Unsafe.As(ref probe); + return BitConverter.IsLittleEndian && + layout.Flags == 0x80050000U && + layout.High == 0x33333333U && + layout.Low == 0x11111111U && + layout.Middle == 0x22222222U; + } + +#pragma warning disable CS0649 // Fields are populated by reinterpreting decimal storage via Unsafe.As. + private struct DecimalLayout + { + public uint Flags; + public uint High; + public uint Low; + public uint Middle; + } +#pragma warning restore CS0649 +#endif + + private static int CountDigits(ulong value) + { + var count = 1; + while (value >= 10) + { + value /= 10; + count++; + } + + return count; + } + + private static int CountFractionDigits(long fraction) + { + if (fraction == 0) + { + return 0; + } + + var digits = 7; + while (fraction % 10 == 0) + { + fraction /= 10; + digits--; + } + + return digits; + } + + private static void WriteFraction( + long fraction, + int digits, + Span destination, + int start + ) + where TCodeUnit : unmanaged + { + for (var index = 7; index > digits; index--) + { + fraction /= 10; + } + + var end = start + digits; + while (end > start) + { + var quotient = fraction / 10; + destination[--end] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + fraction - quotient * 10) + ); + fraction = quotient; + } + } + + private static void WriteUnsignedComponent( + ulong value, + Span destination, + ref int index + ) + where TCodeUnit : unmanaged + { + var digits = CountDigits(value); + var end = index + digits; + WriteUInt64Digits(value, destination, end, index); + index = end; + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void WriteTwoDigits( + uint value, + Span destination, + int index + ) + where TCodeUnit : unmanaged + { + var quotient = value / 10; + destination[index] = CanonicalCodeUnit.FromAscii((byte) ('0' + quotient)); + destination[index + 1] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + value - quotient * 10) + ); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void WriteFourDigits( + uint value, + Span destination, + int index + ) + where TCodeUnit : unmanaged + { + for (var offset = 3; offset >= 0; offset--) + { + var quotient = value / 10; + destination[index + offset] = CanonicalCodeUnit.FromAscii( + (byte) ('0' + value - quotient * 10) + ); + value = quotient; + } + } + + private static void WriteHexByte( + byte value, + Span destination, + ref int index + ) + where TCodeUnit : unmanaged + { + destination[index++] = CanonicalCodeUnit.FromAscii(ToLowerHex(value >> 4)); + destination[index++] = CanonicalCodeUnit.FromAscii(ToLowerHex(value & 0xF)); + } + + private static byte ToLowerHex(int value) => + (byte) (value < 10 ? '0' + value : 'a' + value - 10); + +#pragma warning disable CS0649 // Fields are populated by reinterpreting Guid storage via Unsafe.As. + private struct GuidFields + { + public int A; + public short B; + public short C; + public byte D; + public byte E; + public byte F; + public byte G; + public byte H; + public byte I; + public byte J; + public byte K; + } +#pragma warning restore CS0649 +} diff --git a/src/Light.PortableResults/Text/README.md b/src/Light.PortableResults/Text/README.md new file mode 100644 index 0000000..c0d1d6d --- /dev/null +++ b/src/Light.PortableResults/Text/README.md @@ -0,0 +1,41 @@ +# Runtime canonical-text provenance + +The scalar formatting routines in this folder are adapted from the +[dotnet/runtime](https://github.com/dotnet/runtime) repository's `release/6.0` servicing line, +using immutable tag `v6.0.36` at commit +`f1dd57165bfd91875761329ac3a8b17f6606ad18`. + +The following upstream files supplied the implementation: + +- `src/libraries/System.Private.CoreLib/src/System/Number.Formatting.cs` +- `src/libraries/System.Private.CoreLib/src/System/Decimal.DecCalc.cs` +- `src/libraries/System.Private.CoreLib/src/System/Globalization/DateTimeFormat.cs` +- `src/libraries/System.Private.CoreLib/src/System/Guid.cs` +- `src/libraries/System.Private.Xml/src/System/Xml/Schema/XsdDuration.cs` + +## Adaptations + +- Retained only invariant signed and unsigned 64-bit decimal formatting and made the renderer generic + over UTF-16 characters and UTF-8 bytes. +- Retained decimal's 96-bit division by one billion, added a scale-preserving direct renderer, and + split field extraction by target. `net10.0` uses `decimal.GetBits(decimal, Span)`; + `netstandard2.0` validates and uses the historical little-endian decimal layout without allocating. +- Reduced round-trip date/time formatting to the metadata canonical forms, omitted zero fractions, + trimmed trailing fractional zeros, normalized local `DateTime` values to UTC, and retained explicit + offsets for `DateTimeOffset`. +- Reduced `XsdDuration` to the `TimeSpan` constructor and duration renderer, including the unchecked + `TimeSpan.MinValue` magnitude conversion and XML Schema component-omission rules. +- Retained GUID's lowercase `D` branch and hexadecimal conversion, reading the sequential numeric + fields without an intermediate byte or string allocation. +- Added all-or-nothing capacity checks and direct generic output so both encodings share each renderer. + Only UTF-16-to-UTF-8 transcoding differs by target. + +The complete upstream license is reproduced in the repository-root `THIRD-PARTY-NOTICES.md`, which +is packed at the root of the `Light.PortableResults` NuGet package. + +## Floating-point formatting + +`CanonicalTextFormatter.FloatingPoint.cs` supplies the `double` and `float` portions of the partial +formatter. Its invariant renderer is Light.PortableResults code, while its shortest-digit generation +uses the internal Grisu3 and Dragon4 implementation under `Numbers/`. See `Numbers/README.md` for that +implementation's provenance and adaptations. diff --git a/tests/Light.PortableResults.Tests/CloudEvents/Writing/JsonCloudEventsExtensionsTests.cs b/tests/Light.PortableResults.Tests/CloudEvents/Writing/JsonCloudEventsExtensionsTests.cs index c59f4d1..b457dde 100644 --- a/tests/Light.PortableResults.Tests/CloudEvents/Writing/JsonCloudEventsExtensionsTests.cs +++ b/tests/Light.PortableResults.Tests/CloudEvents/Writing/JsonCloudEventsExtensionsTests.cs @@ -159,31 +159,15 @@ public void WriteCloudEventsExtensionAttributeShouldOmitNullAtomically() [Fact] public void FloatingPointExtensionAttributeWritingShouldAllocateNothingAfterWarmup() { - var output = new ArrayBufferWriter(1024 * 1024); - using var writer = new Utf8JsonWriter(output); var doubleValue = MetadataValue.FromDouble(36_028_797_018_963_968.0); var singleValue = MetadataValue.FromSingle(123_456_789f); - writer.WriteStartObject(); - - for (var index = 0; index < 100; index++) - { - writer.WriteCloudEventsExtensionAttribute("doublevalue", doubleValue); - writer.WriteCloudEventsExtensionAttribute("singlevalue", singleValue); - } - var before = GC.GetAllocatedBytesForCurrentThread(); - for (var index = 0; index < 1_000; index++) - { - writer.WriteCloudEventsExtensionAttribute("doublevalue", doubleValue); - writer.WriteCloudEventsExtensionAttribute("singlevalue", singleValue); - } - - var after = GC.GetAllocatedBytesForCurrentThread(); - after.Should().Be(before); + MeasureMinimumWriterAllocations(doubleValue).Should().Be(0); + MeasureMinimumWriterAllocations(singleValue).Should().Be(0); } [Fact] - public void OtherStringMappedKindsShouldNotAllocateMoreThanCanonicalFormatting() + public void OtherStringMappedKindsShouldNotAllocateCanonicalText() { var values = new[] { @@ -212,11 +196,8 @@ public void OtherStringMappedKindsShouldNotAllocateMoreThanCanonicalFormatting() var canonicalAllocations = MeasureMinimumCanonicalFormattingAllocations(value); var writerAllocations = MeasureMinimumWriterAllocations(value); - writerAllocations.Should().BeLessThanOrEqualTo( - canonicalAllocations, - "writing {0} should materialize at most its existing canonical string", - value.Kind - ); + canonicalAllocations.Should().Be(0, "formatting {0} is span-based", value.Kind); + writerAllocations.Should().Be(0, "writing {0} should not materialize canonical text", value.Kind); } } diff --git a/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs b/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs new file mode 100644 index 0000000..1083161 --- /dev/null +++ b/tests/Light.PortableResults.Tests/Metadata/CanonicalTextFormatterTests.cs @@ -0,0 +1,523 @@ +using System; +using System.Collections.Generic; +using System.Globalization; +using System.Runtime.CompilerServices; +using System.Text; +using System.Xml; +using FluentAssertions; +using Light.PortableResults.Metadata; +using Light.PortableResults.Text; +using Xunit; + +namespace Light.PortableResults.Tests.Metadata; + +public sealed class CanonicalTextFormatterTests +{ + private const int AllocationIterations = 1_000; + + public static TheoryData CanonicalValues + { + get + { + var data = new TheoryData + { + { "null", MetadataValue.Null, "null" }, + { "boolean-false", MetadataValue.FromBoolean(false), "false" }, + { "boolean-true", MetadataValue.FromBoolean(true), "true" }, + { "int64-minimum", MetadataValue.FromInt64(long.MinValue), Invariant(long.MinValue) }, + { "int64-maximum", MetadataValue.FromInt64(long.MaxValue), Invariant(long.MaxValue) }, + { "double", MetadataValue.FromDouble(-0.0), "-0.0" }, + { "string-empty", MetadataValue.FromString(string.Empty), string.Empty }, + { "string-ascii", MetadataValue.FromString("canonical text"), "canonical text" }, + { "string-non-ascii", MetadataValue.FromString("Grüße 日本語 😀"), "Grüße 日本語 😀" }, + { "string-unpaired-high", MetadataValue.FromString("a\uD800b"), "a\uD800b" }, + { "string-unpaired-low", MetadataValue.FromString("a\uDC00b"), "a\uDC00b" }, + { "decimal-scale", MetadataValue.FromDecimal(19.50m), Invariant(19.50m) }, + { "decimal-negative-scale", MetadataValue.FromDecimal(-19.50m), Invariant(-19.50m) }, + { "decimal-negative-fraction", MetadataValue.FromDecimal(-0.5m), Invariant(-0.5m) }, + { "decimal-negative-zero", MetadataValue.FromDecimal(-0.000m), Invariant(-0.000m) }, + { "decimal-minimum", MetadataValue.FromDecimal(decimal.MinValue), Invariant(decimal.MinValue) }, + { "decimal-maximum", MetadataValue.FromDecimal(decimal.MaxValue), Invariant(decimal.MaxValue) }, + { + "decimal-smallest-scaled", + MetadataValue.FromDecimal(0.0000000000000000000000000001m), + Invariant(0.0000000000000000000000000001m) + }, + { "uint64-maximum", MetadataValue.FromUInt64(ulong.MaxValue), Invariant(ulong.MaxValue) }, + { "single", MetadataValue.FromSingle(float.Epsilon), "1E-45" }, + { "char-ascii", MetadataValue.FromChar('x'), "x" }, + { "char-non-ascii", MetadataValue.FromChar('ß'), "ß" }, + { "char-unpaired", MetadataValue.FromChar('\uD800'), "\uD800" } + }; + + var utcMinimum = DateTime.SpecifyKind(DateTime.MinValue, DateTimeKind.Utc); + var utcMaximum = DateTime.SpecifyKind(DateTime.MaxValue, DateTimeKind.Utc); + var unspecified = new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Unspecified).AddTicks(1_230_000); + data.Add("datetime-utc-minimum", MetadataValue.FromDateTime(utcMinimum), Format(utcMinimum)); + data.Add("datetime-utc-maximum", MetadataValue.FromDateTime(utcMaximum), Format(utcMaximum)); + data.Add("datetime-unspecified-fraction", MetadataValue.FromDateTime(unspecified), Format(unspecified)); + + var offsetMinimum = DateTimeOffset.MinValue; + var offsetMaximum = DateTimeOffset.MaxValue; + var offsetFraction = new DateTimeOffset( + 2026, + 7, + 26, + 13, + 45, + 30, + TimeSpan.FromHours(-3.5) + ).AddTicks(1_234_500); + data.Add("datetimeoffset-minimum", MetadataValue.FromDateTimeOffset(offsetMinimum), Format(offsetMinimum)); + data.Add("datetimeoffset-maximum", MetadataValue.FromDateTimeOffset(offsetMaximum), Format(offsetMaximum)); + data.Add( + "datetimeoffset-fraction", + MetadataValue.FromDateTimeOffset(offsetFraction), + Format(offsetFraction) + ); + +#if !TESTING_NETSTANDARD_ASSET + data.Add("date-minimum", MetadataValue.FromDateOnly(DateOnly.MinValue), Format(DateOnly.MinValue)); + data.Add("date-maximum", MetadataValue.FromDateOnly(DateOnly.MaxValue), Format(DateOnly.MaxValue)); + data.Add("time-minimum", MetadataValue.FromTimeOnly(TimeOnly.MinValue), Format(TimeOnly.MinValue)); + data.Add("time-maximum", MetadataValue.FromTimeOnly(TimeOnly.MaxValue), Format(TimeOnly.MaxValue)); +#endif + + var duration = TimeSpan.FromDays(2) + + TimeSpan.FromHours(3) + + TimeSpan.FromMinutes(4) + + TimeSpan.FromSeconds(5) + + TimeSpan.FromTicks(600_000); + data.Add("timespan-zero", MetadataValue.FromTimeSpan(TimeSpan.Zero), XmlConvert.ToString(TimeSpan.Zero)); + data.Add( + "timespan-days-only", + MetadataValue.FromTimeSpan(TimeSpan.FromDays(2)), + XmlConvert.ToString(TimeSpan.FromDays(2)) + ); + data.Add("timespan-components", MetadataValue.FromTimeSpan(duration), XmlConvert.ToString(duration)); + data.Add( + "timespan-minimum", + MetadataValue.FromTimeSpan(TimeSpan.MinValue), + XmlConvert.ToString(TimeSpan.MinValue) + ); + data.Add( + "timespan-maximum", + MetadataValue.FromTimeSpan(TimeSpan.MaxValue), + XmlConvert.ToString(TimeSpan.MaxValue) + ); + + var guid = new Guid("a1b2c3d4-e5f6-7890-abcd-ef1234567890"); + data.Add("guid-empty", MetadataValue.FromGuid(Guid.Empty), Guid.Empty.ToString("D")); + data.Add("guid", MetadataValue.FromGuid(guid), guid.ToString("D")); + data.Add( + "uri", + MetadataValue.FromUri(new Uri("https://example.com/Grüße?q=日本語#😀")), + "https://example.com/Grüße?q=日本語#😀" + ); + + return data; + } + } + + public static TheoryData AllocationValues + { + get + { + var data = new TheoryData + { + MetadataValue.Null, + MetadataValue.FromBoolean(true), + MetadataValue.FromInt64(long.MinValue), + MetadataValue.FromDouble(double.MaxValue), + MetadataValue.FromString("Grüße 日本語 😀"), + MetadataValue.FromString("a\uD800b"), + MetadataValue.FromDecimal(0.0000000000000000000000000001m), + MetadataValue.FromUInt64(ulong.MaxValue), + MetadataValue.FromSingle(float.Epsilon), + MetadataValue.FromChar('\uD800'), + MetadataValue.FromDateTime(DateTime.SpecifyKind(DateTime.MaxValue, DateTimeKind.Utc)), + MetadataValue.FromDateTimeOffset(DateTimeOffset.MaxValue) + }; +#if !TESTING_NETSTANDARD_ASSET + data.Add(MetadataValue.FromDateOnly(DateOnly.MaxValue)); + data.Add(MetadataValue.FromTimeOnly(TimeOnly.MaxValue)); +#endif + data.Add(MetadataValue.FromTimeSpan(TimeSpan.MinValue)); + data.Add(MetadataValue.FromGuid(Guid.Empty)); + data.Add(MetadataValue.FromUri(new Uri("https://example.com/Grüße?q=日本語"))); + + return data; + } + } + + public static TheoryData ValidatorValues + { + get + { + var data = new TheoryData + { + { 0, Invariant(long.MinValue) }, + { 1, Invariant(ulong.MaxValue) }, + { 2, "0.1" }, + { 3, "2026-07-26T13:45:30.123Z" }, + { 4, "2026-07-26T13:45:30.123+02:00" } + }; +#if !TESTING_NETSTANDARD_ASSET + data.Add(5, "2026-07-26"); + data.Add(6, "13:45:30.123"); +#endif + data.Add(7, "P2DT3H4M5.06S"); + data.Add(8, "a1b2c3d4-e5f6-7890-abcd-ef1234567890"); + + return data; + } + } + + [Theory] + [MemberData(nameof(CanonicalValues))] + public void MetadataCanonicalFormattingShouldMatchTheExistingTextInBothEncodings( + string caseName, + MetadataValue value, + string expected + ) + { + var expectedUtf8 = Encoding.UTF8.GetBytes(expected); + var chars = new char[expected.Length]; + var bytes = new byte[expectedUtf8.Length]; + + value.TryFormatCanonical(chars, out var charsWritten).Should().BeTrue(caseName); + value.TryFormatCanonicalUtf8(bytes, out var bytesWritten).Should().BeTrue(caseName); + + charsWritten.Should().Be(expected.Length, caseName); + bytesWritten.Should().Be(expectedUtf8.Length, caseName); + chars.AsSpan().SequenceEqual(expected.AsSpan()).Should().BeTrue(caseName); + bytes.Should().Equal(expectedUtf8, caseName); + value.ToCanonicalString().Should().Be(expected, caseName); + if (IsAscii(expected)) + { + bytesWritten.Should().Be(charsWritten, caseName); + } + } + + [Theory] + [MemberData(nameof(CanonicalValues))] + public void MetadataCanonicalFormattingShouldBeAtomicWhenCapacityIsOneShort( + string caseName, + MetadataValue value, + string expected + ) + { + var expectedUtf8Length = Encoding.UTF8.GetByteCount(expected); + var chars = CreateFilledArray(Math.Max(expected.Length, 1), '\uA55A'); + var bytes = CreateFilledArray(Math.Max(expectedUtf8Length, 1), (byte) 0xA5); + + if (expected.Length == 0) + { + value.TryFormatCanonical(chars.AsSpan(0, 0), out var emptyCharsWritten) + .Should() + .BeTrue(caseName); + value.TryFormatCanonicalUtf8(bytes.AsSpan(0, 0), out var emptyBytesWritten) + .Should() + .BeTrue(caseName); + + emptyCharsWritten.Should().Be(0, caseName); + emptyBytesWritten.Should().Be(0, caseName); + chars.Should().OnlyContain(character => character == '\uA55A', caseName); + bytes.Should().OnlyContain(element => element == 0xA5, caseName); + return; + } + + value.TryFormatCanonical(chars.AsSpan(0, expected.Length - 1), out var charsWritten) + .Should() + .BeFalse(caseName); + value.TryFormatCanonicalUtf8(bytes.AsSpan(0, expectedUtf8Length - 1), out var bytesWritten) + .Should() + .BeFalse(caseName); + + charsWritten.Should().Be(0, caseName); + bytesWritten.Should().Be(0, caseName); + chars.Should().OnlyContain(character => character == '\uA55A', caseName); + bytes.Should().OnlyContain(value => value == 0xA5, caseName); + } + + [Theory] + [MemberData(nameof(AllocationValues))] + public void MetadataCanonicalSpanFormattingShouldNotAllocate(MetadataValue value) + { + Span chars = stackalloc char[128]; + Span bytes = stackalloc byte[256]; + for (var index = 0; index < 100; index++) + { + value.TryFormatCanonical(chars, out _); + value.TryFormatCanonicalUtf8(bytes, out _); + } + + var minimumAllocations = long.MaxValue; + for (var sample = 0; sample < 5; sample++) + { + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var index = 0; index < AllocationIterations; index++) + { + value.TryFormatCanonical(chars, out _); + value.TryFormatCanonicalUtf8(bytes, out _); + } + + minimumAllocations = Math.Min( + minimumAllocations, + GC.GetAllocatedBytesForCurrentThread() - before + ); + } + + minimumAllocations.Should().Be(0, value.Kind.ToString()); + } + + [Theory] + [MemberData(nameof(ValidatorValues))] + public void CanonicalRoundTripValidatorsShouldNotAllocateCandidateText(int validator, string text) + { + var value = MetadataValue.FromString(text); + for (var index = 0; index < 100; index++) + { + Validate(value, validator).Should().BeTrue(); + } + + var minimumAllocations = long.MaxValue; + var allValid = true; + for (var sample = 0; sample < 5; sample++) + { + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var index = 0; index < AllocationIterations; index++) + { + allValid &= Validate(value, validator); + } + + minimumAllocations = Math.Min( + minimumAllocations, + GC.GetAllocatedBytesForCurrentThread() - before + ); + } + + allValid.Should().BeTrue(); + minimumAllocations.Should().Be(0); + } + + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(2)] + [InlineData(3)] + public void ExistingAllocationFreeCanonicalStringsShouldReuseTheirText(int caseIndex) + { + var value = caseIndex switch + { + 0 => MetadataValue.Null, + 1 => MetadataValue.FromBoolean(true), + 2 => MetadataValue.FromString("stored string"), + _ => MetadataValue.FromUri(new Uri("https://example.com/stored")) + }; + var expected = value.ToCanonicalString(); + + for (var index = 0; index < 100; index++) + { + _ = value.ToCanonicalString(); + } + + var before = GC.GetAllocatedBytesForCurrentThread(); + var allReferencesMatch = true; + for (var index = 0; index < AllocationIterations; index++) + { + allReferencesMatch &= ReferenceEquals(value.ToCanonicalString(), expected); + } + + var after = GC.GetAllocatedBytesForCurrentThread(); + allReferencesMatch.Should().BeTrue(); + after.Should().Be(before); + } + + [Fact] + public void CanonicalTextFormatterShouldResolveACharLiteralToTheCharOverload() + { + Span destination = stackalloc char[CanonicalTextFormatter.MaximumCharLength]; + + CanonicalTextFormatter.TryFormat('x', destination, out var charsWritten).Should().BeTrue(); + + destination.Slice(0, charsWritten).ToString().Should().Be("x"); + + CanonicalTextFormatter.TryFormat(42, destination, out charsWritten).Should().BeTrue(); + destination.Slice(0, charsWritten).ToString().Should().Be("42"); + } + + [Fact] + public void CanonicalCodeUnitShouldConvertAsciiToBothSupportedCodeUnits() + { + const byte value = (byte) 'x'; + + CanonicalCodeUnit.FromAscii(value).Should().Be(value); + CanonicalCodeUnit.FromAscii(value).Should().Be('x'); + } + + [Fact] + public void CanonicalCodeUnitShouldRejectUnsupportedCodeUnitTypes() + { + var act = () => CanonicalCodeUnit.FromAscii((byte) 'x'); + + act.Should().Throw(); + } + + [Fact] + public void CanonicalTextFormatterShouldNormalizeOnlyLocalDateTimes() + { + var local = new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Local).AddTicks(1_230_000); + var unspecified = DateTime.SpecifyKind(local, DateTimeKind.Unspecified); + Span localText = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + Span unspecifiedText = stackalloc char[CanonicalTextFormatter.MaximumDateTimeLength]; + + CanonicalTextFormatter.TryFormat(local, localText, out var localLength).Should().BeTrue(); + CanonicalTextFormatter.TryFormat(unspecified, unspecifiedText, out var unspecifiedLength).Should().BeTrue(); + + localText.Slice(0, localLength).ToString().Should().Be(Format(local.ToUniversalTime())); + localLength.Should().BeLessThanOrEqualTo(CanonicalTextFormatter.MaximumDateTimeLength); + unspecifiedText.Slice(0, unspecifiedLength).ToString().Should().Be(Format(unspecified)); + } + + [Theory] + [InlineData(-1)] + [InlineData(CanonicalTextFormatter.MaximumDayNumber + 1)] + public void CanonicalDateFormatterShouldRejectInvalidDayNumbers(int dayNumber) + { + var chars = CreateFilledArray(CanonicalTextFormatter.MaximumDateLength, '\uA55A'); + var bytes = CreateFilledArray(CanonicalTextFormatter.MaximumDateLength, (byte) 0xA5); + + var charAct = () => CanonicalTextFormatter.TryFormatDate(dayNumber, chars, out _); + var byteAct = () => CanonicalTextFormatter.TryFormatDateUtf8(dayNumber, bytes, out _); + + charAct.Should().Throw(); + byteAct.Should().Throw(); + chars.Should().OnlyContain(value => value == '\uA55A'); + bytes.Should().OnlyContain(value => value == 0xA5); + } + + [Theory] + [InlineData(-1L)] + [InlineData(CanonicalTextFormatter.MaximumTimeOfDayTicks + 1)] + public void CanonicalTimeFormatterShouldRejectInvalidTicks(long ticks) + { + var chars = CreateFilledArray(CanonicalTextFormatter.MaximumTimeLength, '\uA55A'); + var bytes = CreateFilledArray(CanonicalTextFormatter.MaximumTimeLength, (byte) 0xA5); + + var charAct = () => CanonicalTextFormatter.TryFormatTime(ticks, chars, out _); + var byteAct = () => CanonicalTextFormatter.TryFormatTimeUtf8(ticks, bytes, out _); + + charAct.Should().Throw(); + byteAct.Should().Throw(); + chars.Should().OnlyContain(value => value == '\uA55A'); + bytes.Should().OnlyContain(value => value == 0xA5); + } + + [Fact] + public void ComplexAndMalformedMetadataShouldThrowInBothEncodings() + { + var values = new List + { + MetadataValue.FromArray(MetadataArray.Empty), + MetadataValue.FromObject(MetadataObject.Empty), + MetadataValueTestFactory.CreateWithInt64Payload(MetadataKind.DateTime, long.MaxValue), + MetadataValueTestFactory.CreateWithInt64Payload(MetadataKind.DateTime, -1L), + MetadataValueTestFactory.CreateWithInt64Payload(MetadataKind.DateOnly, long.MaxValue), + MetadataValueTestFactory.CreateWithInt64Payload(MetadataKind.TimeOnly, long.MaxValue) + }; + + foreach (var value in values) + { + var chars = new char[MetadataValue.MaximumPrimitiveCanonicalLength]; + var bytes = new byte[MetadataValue.MaximumPrimitiveCanonicalLength]; + var charAct = () => value.TryFormatCanonical(chars, out _); + var byteAct = () => value.TryFormatCanonicalUtf8(bytes, out _); + + charAct.Should().Throw(value.Kind.ToString()); + byteAct.Should().Throw(value.Kind.ToString()); + } + } + + [Fact] + public void LegacyDecimalFieldOrderShouldMatchTheContractualBits() + { + var value = new decimal( + lo: 0x11111111, + mid: 0x22222222, + hi: 0x33333333, + isNegative: true, + scale: 5 + ); + var expected = decimal.GetBits(value); + ref var layout = ref Unsafe.As(ref value); + + new[] { layout.Low, layout.Middle, layout.High, layout.Flags } + .Should() + .Equal((uint) expected[0], (uint) expected[1], (uint) expected[2], (uint) expected[3]); + } + + private static bool IsAscii(string text) + { + foreach (var character in text) + { + if (character > 0x7F) + { + return false; + } + } + + return true; + } + + private static bool Validate(MetadataValue value, int validator) => + validator switch + { + 0 => value.TryGetInt64(out _), + 1 => value.TryGetUInt64(out _), + 2 => value.TryGetSingle(out _), + 3 => value.TryGetDateTime(out _), + 4 => value.TryGetDateTimeOffset(out _), +#if !TESTING_NETSTANDARD_ASSET + 5 => value.TryGetDateOnly(out _), + 6 => value.TryGetTimeOnly(out _), +#endif + 7 => value.TryGetTimeSpan(out _), + _ => value.TryGetGuid(out _) + }; + + private static string Invariant(T value) + where T : IFormattable => + value.ToString(null, CultureInfo.InvariantCulture); + + private static string Format(DateTime value) => + value.ToString("yyyy-MM-dd'T'HH:mm:ss.FFFFFFFK", CultureInfo.InvariantCulture); + + private static string Format(DateTimeOffset value) => + value.ToString("yyyy-MM-dd'T'HH:mm:ss.FFFFFFFzzz", CultureInfo.InvariantCulture); + + private static T[] CreateFilledArray(int length, T value) + { + var array = new T[length]; + Array.Fill(array, value); + return array; + } + +#pragma warning disable CS0649 // Fields are populated by reinterpreting decimal storage via Unsafe.As. + private struct DecimalLayout + { + public uint Flags; + public uint High; + public uint Low; + public uint Middle; + } +#pragma warning restore CS0649 + +#if !TESTING_NETSTANDARD_ASSET + private static string Format(DateOnly value) => + value.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture); + + private static string Format(TimeOnly value) => + value.ToString("HH:mm:ss.FFFFFFF", CultureInfo.InvariantCulture); +#endif +} diff --git a/tests/Light.PortableResults.Tests/Numbers/CanonicalFloatingPointFormatterTests.cs b/tests/Light.PortableResults.Tests/Numbers/CanonicalTextFormatterFloatingPointTests.cs similarity index 80% rename from tests/Light.PortableResults.Tests/Numbers/CanonicalFloatingPointFormatterTests.cs rename to tests/Light.PortableResults.Tests/Numbers/CanonicalTextFormatterFloatingPointTests.cs index 3551928..c055e46 100644 --- a/tests/Light.PortableResults.Tests/Numbers/CanonicalFloatingPointFormatterTests.cs +++ b/tests/Light.PortableResults.Tests/Numbers/CanonicalTextFormatterFloatingPointTests.cs @@ -6,12 +6,12 @@ using System.Text; using FluentAssertions; using Light.PortableResults.Metadata; -using Light.PortableResults.Numbers; +using Light.PortableResults.Text; using Xunit; namespace Light.PortableResults.Tests.Numbers; -public sealed class CanonicalFloatingPointFormatterTests +public sealed class CanonicalTextFormatterFloatingPointTests { private const int CorpusSize = 50_000; private const int CorpusSeed = 0x5EED_0058; @@ -31,7 +31,7 @@ public sealed class CanonicalFloatingPointFormatterTests CreateSingleDragon4Formatter(); public static TheoryData DoubleScenarios => - new () + new() { { 0x3F1A36E2EB1C432DUL, "0.0001" }, { 0x3EE4F8B588E368F1UL, "1E-05" }, @@ -63,7 +63,7 @@ public sealed class CanonicalFloatingPointFormatterTests }; public static TheoryData SingleScenarios => - new () + new() { { 0x38D1B717U, "0.0001" }, { 0x3727C5ACU, "1E-05" }, @@ -113,7 +113,7 @@ public void DoubleNamedScenariosShouldUseTheCanonicalEncoding(ulong bits, string { var value = BitConverter.Int64BitsToDouble((long) bits); - CanonicalFloatingPointFormatter.Format(value).Should().Be(expected); + CanonicalTextFormatter.Format(value).Should().Be(expected); Format(value).Should().Be(expected); FormatWithDragon4(value).Should().Be(expected); AssertUtf8Matches(value, expected, forceDragon4: false); @@ -127,7 +127,7 @@ public void SingleNamedScenariosShouldUseTheCanonicalEncoding(uint bits, string { var value = BitConverter.Int32BitsToSingle((int) bits); - CanonicalFloatingPointFormatter.Format(value).Should().Be(expected); + CanonicalTextFormatter.Format(value).Should().Be(expected); Format(value).Should().Be(expected); FormatWithDragon4(value).Should().Be(expected); AssertUtf8Matches(value, expected, forceDragon4: false); @@ -140,17 +140,17 @@ public void NonFiniteDoubleValuesShouldBeRejectedByEveryOverload() { foreach (var value in new[] { double.NaN, double.PositiveInfinity, double.NegativeInfinity }) { - var format = () => CanonicalFloatingPointFormatter.Format(value); + var format = () => CanonicalTextFormatter.Format(value); var tryFormat = () => - CanonicalFloatingPointFormatter.TryFormat( + CanonicalTextFormatter.TryFormat( value, - new char[CanonicalFloatingPointFormatter.MaximumDoubleLength], + new char[CanonicalTextFormatter.MaximumDoubleLength], out _ ); var tryFormatUtf8 = () => - CanonicalFloatingPointFormatter.TryFormatUtf8( + CanonicalTextFormatter.TryFormatUtf8( value, - new byte[CanonicalFloatingPointFormatter.MaximumDoubleLength], + new byte[CanonicalTextFormatter.MaximumDoubleLength], out _ ); @@ -165,17 +165,17 @@ public void NonFiniteSingleValuesShouldBeRejectedByEveryOverload() { foreach (var value in new[] { float.NaN, float.PositiveInfinity, float.NegativeInfinity }) { - var format = () => CanonicalFloatingPointFormatter.Format(value); + var format = () => CanonicalTextFormatter.Format(value); var tryFormat = () => - CanonicalFloatingPointFormatter.TryFormat( + CanonicalTextFormatter.TryFormat( value, - new char[CanonicalFloatingPointFormatter.MaximumSingleLength], + new char[CanonicalTextFormatter.MaximumSingleLength], out _ ); var tryFormatUtf8 = () => - CanonicalFloatingPointFormatter.TryFormatUtf8( + CanonicalTextFormatter.TryFormatUtf8( value, - new byte[CanonicalFloatingPointFormatter.MaximumSingleLength], + new byte[CanonicalTextFormatter.MaximumSingleLength], out _ ); @@ -197,28 +197,28 @@ public void InsufficientDestinationsShouldRemainUnmodified() doubleUtf8Destination.Fill(0xAA); singleUtf8Destination.Fill(0xBB); - CanonicalFloatingPointFormatter.TryFormat( + CanonicalTextFormatter.TryFormat( double.MaxValue, doubleDestination, out var doubleCharsWritten ) .Should() .BeFalse(); - CanonicalFloatingPointFormatter.TryFormat( + CanonicalTextFormatter.TryFormat( float.MaxValue, singleDestination, out var singleCharsWritten ) .Should() .BeFalse(); - CanonicalFloatingPointFormatter.TryFormatUtf8( + CanonicalTextFormatter.TryFormatUtf8( double.MaxValue, doubleUtf8Destination, out var doubleBytesWritten ) .Should() .BeFalse(); - CanonicalFloatingPointFormatter.TryFormatUtf8( + CanonicalTextFormatter.TryFormatUtf8( float.MaxValue, singleUtf8Destination, out var singleBytesWritten @@ -255,7 +255,7 @@ public void RandomFiniteBitPatternsShouldMatchTheRuntimeOracle() var value = BitConverter.Int64BitsToDouble((long) bits); var expected = CanonicalizeRuntimeText(value.ToString("R", CultureInfo.InvariantCulture)); - CanonicalFloatingPointFormatter.Format(value) + CanonicalTextFormatter.Format(value) .Should() .Be(expected, "binary64 bits 0x{0:X16} must be deterministic", bits); FormatWithDragon4(value) @@ -277,7 +277,7 @@ public void RandomFiniteBitPatternsShouldMatchTheRuntimeOracle() var value = BitConverter.Int32BitsToSingle((int) bits); var expected = CanonicalizeRuntimeText(value.ToString("R", CultureInfo.InvariantCulture)); - CanonicalFloatingPointFormatter.Format(value) + CanonicalTextFormatter.Format(value) .Should() .Be(expected, "binary32 bits 0x{0:X8} must be deterministic", bits); FormatWithDragon4(value) @@ -316,15 +316,15 @@ public void FloatingPointSpanFormattingShouldAllocateNothingAfterWarmup() const float singleValue = 123_456_789f; var doubleMetadata = MetadataValue.FromDouble(doubleValue); var singleMetadata = MetadataValue.FromSingle(singleValue); - Span charDestination = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; - Span byteDestination = stackalloc byte[CanonicalFloatingPointFormatter.MaximumDoubleLength]; + Span charDestination = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; + Span byteDestination = stackalloc byte[CanonicalTextFormatter.MaximumDoubleLength]; for (var index = 0; index < 100; index++) { - CanonicalFloatingPointFormatter.TryFormat(doubleValue, charDestination, out _); - CanonicalFloatingPointFormatter.TryFormat(singleValue, charDestination, out _); - CanonicalFloatingPointFormatter.TryFormatUtf8(doubleValue, byteDestination, out _); - CanonicalFloatingPointFormatter.TryFormatUtf8(singleValue, byteDestination, out _); + CanonicalTextFormatter.TryFormat(doubleValue, charDestination, out _); + CanonicalTextFormatter.TryFormat(singleValue, charDestination, out _); + CanonicalTextFormatter.TryFormatUtf8(doubleValue, byteDestination, out _); + CanonicalTextFormatter.TryFormatUtf8(singleValue, byteDestination, out _); ForceDragon4DoubleChars(doubleValue, charDestination, out _, true); ForceDragon4SingleChars(singleValue, charDestination, out _, true); ForceDragon4DoubleBytes(doubleValue, byteDestination, out _, true); @@ -336,10 +336,10 @@ public void FloatingPointSpanFormattingShouldAllocateNothingAfterWarmup() var before = GC.GetAllocatedBytesForCurrentThread(); for (var index = 0; index < 1_000; index++) { - CanonicalFloatingPointFormatter.TryFormat(doubleValue, charDestination, out _); - CanonicalFloatingPointFormatter.TryFormat(singleValue, charDestination, out _); - CanonicalFloatingPointFormatter.TryFormatUtf8(doubleValue, byteDestination, out _); - CanonicalFloatingPointFormatter.TryFormatUtf8(singleValue, byteDestination, out _); + CanonicalTextFormatter.TryFormat(doubleValue, charDestination, out _); + CanonicalTextFormatter.TryFormat(singleValue, charDestination, out _); + CanonicalTextFormatter.TryFormatUtf8(doubleValue, byteDestination, out _); + CanonicalTextFormatter.TryFormatUtf8(singleValue, byteDestination, out _); ForceDragon4DoubleChars(doubleValue, charDestination, out _, true); ForceDragon4SingleChars(singleValue, charDestination, out _, true); ForceDragon4DoubleBytes(doubleValue, byteDestination, out _, true); @@ -356,15 +356,15 @@ public void StringFormattingShouldAllocateOnlyTheReturnedStrings() { const double doubleValue = 36_028_797_018_963_968.0; const float singleValue = 123_456_789f; - var doubleText = CanonicalFloatingPointFormatter.Format(doubleValue); - var singleText = CanonicalFloatingPointFormatter.Format(singleValue); + var doubleText = CanonicalTextFormatter.Format(doubleValue); + var singleText = CanonicalTextFormatter.Format(singleValue); var doubleMetadata = MetadataValue.FromDouble(doubleValue); var singleMetadata = MetadataValue.FromSingle(singleValue); for (var index = 0; index < 100; index++) { - _ = CanonicalFloatingPointFormatter.Format(doubleValue); - _ = CanonicalFloatingPointFormatter.Format(singleValue); + _ = CanonicalTextFormatter.Format(doubleValue); + _ = CanonicalTextFormatter.Format(singleValue); _ = doubleMetadata.ToCanonicalString(); _ = singleMetadata.ToCanonicalString(); } @@ -372,13 +372,13 @@ public void StringFormattingShouldAllocateOnlyTheReturnedStrings() var doubleBaseline = MeasureStringAllocations(() => new string(doubleText.AsSpan())); var singleBaseline = MeasureStringAllocations(() => new string(singleText.AsSpan())); - MeasureStringAllocations(() => CanonicalFloatingPointFormatter.Format(doubleValue)) + MeasureStringAllocations(() => CanonicalTextFormatter.Format(doubleValue)) .Should() .Be(doubleBaseline); MeasureStringAllocations(() => doubleMetadata.ToCanonicalString()) .Should() .Be(doubleBaseline); - MeasureStringAllocations(() => CanonicalFloatingPointFormatter.Format(singleValue)) + MeasureStringAllocations(() => CanonicalTextFormatter.Format(singleValue)) .Should() .Be(singleBaseline); MeasureStringAllocations(() => singleMetadata.ToCanonicalString()) @@ -390,13 +390,13 @@ private static void AssertFitsIntoMaximumLength(double value) { // The destinations are sized exactly at the constant, so a successful call is itself the // proof that the constant bounds this value in this encoding. - Span chars = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; - Span bytes = stackalloc byte[CanonicalFloatingPointFormatter.MaximumDoubleLength]; + Span chars = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; + Span bytes = stackalloc byte[CanonicalTextFormatter.MaximumDoubleLength]; - CanonicalFloatingPointFormatter.TryFormat(value, chars, out var charsWritten) + CanonicalTextFormatter.TryFormat(value, chars, out var charsWritten) .Should() .BeTrue(); - CanonicalFloatingPointFormatter.TryFormatUtf8(value, bytes, out var bytesWritten) + CanonicalTextFormatter.TryFormatUtf8(value, bytes, out var bytesWritten) .Should() .BeTrue(); ForceDragon4DoubleChars(value, chars, out var dragon4CharsWritten, true).Should().BeTrue(); @@ -409,13 +409,13 @@ private static void AssertFitsIntoMaximumLength(double value) private static void AssertFitsIntoMaximumLength(float value) { - Span chars = stackalloc char[CanonicalFloatingPointFormatter.MaximumSingleLength]; - Span bytes = stackalloc byte[CanonicalFloatingPointFormatter.MaximumSingleLength]; + Span chars = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; + Span bytes = stackalloc byte[CanonicalTextFormatter.MaximumSingleLength]; - CanonicalFloatingPointFormatter.TryFormat(value, chars, out var charsWritten) + CanonicalTextFormatter.TryFormat(value, chars, out var charsWritten) .Should() .BeTrue(); - CanonicalFloatingPointFormatter.TryFormatUtf8(value, bytes, out var bytesWritten) + CanonicalTextFormatter.TryFormatUtf8(value, bytes, out var bytesWritten) .Should() .BeTrue(); ForceDragon4SingleChars(value, chars, out var dragon4CharsWritten, true).Should().BeTrue(); @@ -429,7 +429,7 @@ private static void AssertFitsIntoMaximumLength(float value) private static void AssertMatchesOracle(double value) { var expected = CanonicalizeRuntimeText(value.ToString("R", CultureInfo.InvariantCulture)); - CanonicalFloatingPointFormatter.Format(value).Should().Be(expected); + CanonicalTextFormatter.Format(value).Should().Be(expected); FormatWithDragon4(value).Should().Be(expected); AssertUtf8Matches(value, expected, forceDragon4: false); AssertUtf8Matches(value, expected, forceDragon4: true); @@ -438,7 +438,7 @@ private static void AssertMatchesOracle(double value) private static void AssertMatchesOracle(float value) { var expected = CanonicalizeRuntimeText(value.ToString("R", CultureInfo.InvariantCulture)); - CanonicalFloatingPointFormatter.Format(value).Should().Be(expected); + CanonicalTextFormatter.Format(value).Should().Be(expected); FormatWithDragon4(value).Should().Be(expected); AssertUtf8Matches(value, expected, forceDragon4: false); AssertUtf8Matches(value, expected, forceDragon4: true); @@ -449,8 +449,8 @@ private static string CanonicalizeRuntimeText(string value) => private static string Format(double value) { - Span destination = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; - CanonicalFloatingPointFormatter.TryFormat(value, destination, out var charsWritten) + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) .Should() .BeTrue(); return new string(destination[..charsWritten]); @@ -458,8 +458,8 @@ private static string Format(double value) private static string Format(float value) { - Span destination = stackalloc char[CanonicalFloatingPointFormatter.MaximumSingleLength]; - CanonicalFloatingPointFormatter.TryFormat(value, destination, out var charsWritten) + Span destination = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; + CanonicalTextFormatter.TryFormat(value, destination, out var charsWritten) .Should() .BeTrue(); return new string(destination[..charsWritten]); @@ -467,30 +467,30 @@ private static string Format(float value) private static string FormatWithDragon4(double value) { - Span destination = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; + Span destination = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; ForceDragon4DoubleChars(value, destination, out var charsWritten, true).Should().BeTrue(); return new string(destination[..charsWritten]); } private static string FormatWithDragon4(float value) { - Span destination = stackalloc char[CanonicalFloatingPointFormatter.MaximumSingleLength]; + Span destination = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; ForceDragon4SingleChars(value, destination, out var charsWritten, true).Should().BeTrue(); return new string(destination[..charsWritten]); } private static void AssertUtf8Matches(double value, string expected, bool forceDragon4) { - Span chars = stackalloc char[CanonicalFloatingPointFormatter.MaximumDoubleLength]; - Span bytes = stackalloc byte[CanonicalFloatingPointFormatter.MaximumDoubleLength + 1]; + Span chars = stackalloc char[CanonicalTextFormatter.MaximumDoubleLength]; + Span bytes = stackalloc byte[CanonicalTextFormatter.MaximumDoubleLength + 1]; bytes.Fill(UnwrittenByte); var charsSucceeded = forceDragon4 ? ForceDragon4DoubleChars(value, chars, out var charsWritten, true) : - CanonicalFloatingPointFormatter.TryFormat(value, chars, out charsWritten); + CanonicalTextFormatter.TryFormat(value, chars, out charsWritten); var bytesSucceeded = forceDragon4 ? ForceDragon4DoubleBytes(value, bytes, out var bytesWritten, true) : - CanonicalFloatingPointFormatter.TryFormatUtf8(value, bytes, out bytesWritten); + CanonicalTextFormatter.TryFormatUtf8(value, bytes, out bytesWritten); charsSucceeded.Should().BeTrue(); bytesSucceeded.Should().BeTrue(); @@ -502,16 +502,16 @@ private static void AssertUtf8Matches(double value, string expected, bool forceD private static void AssertUtf8Matches(float value, string expected, bool forceDragon4) { - Span chars = stackalloc char[CanonicalFloatingPointFormatter.MaximumSingleLength]; - Span bytes = stackalloc byte[CanonicalFloatingPointFormatter.MaximumSingleLength + 1]; + Span chars = stackalloc char[CanonicalTextFormatter.MaximumSingleLength]; + Span bytes = stackalloc byte[CanonicalTextFormatter.MaximumSingleLength + 1]; bytes.Fill(UnwrittenByte); var charsSucceeded = forceDragon4 ? ForceDragon4SingleChars(value, chars, out var charsWritten, true) : - CanonicalFloatingPointFormatter.TryFormat(value, chars, out charsWritten); + CanonicalTextFormatter.TryFormat(value, chars, out charsWritten); var bytesSucceeded = forceDragon4 ? ForceDragon4SingleBytes(value, bytes, out var bytesWritten, true) : - CanonicalFloatingPointFormatter.TryFormatUtf8(value, bytes, out bytesWritten); + CanonicalTextFormatter.TryFormatUtf8(value, bytes, out bytesWritten); charsSucceeded.Should().BeTrue(); bytesSucceeded.Should().BeTrue(); @@ -536,7 +536,7 @@ private static TFormatter CreateSingleDragon4Formatter() .CreateDelegate(typeof(TFormatter)); private static MethodInfo GetTryFormatCore(Type numberType) => - typeof(CanonicalFloatingPointFormatter) + typeof(CanonicalTextFormatter) .GetMethods(BindingFlags.NonPublic | BindingFlags.Static) .Single( method => diff --git a/tests/Light.PortableResults.Tests/SharedJsonSerialization/Writing/SharedWritingExtensionsTests.cs b/tests/Light.PortableResults.Tests/SharedJsonSerialization/Writing/SharedWritingExtensionsTests.cs index 2184864..a63bdc7 100644 --- a/tests/Light.PortableResults.Tests/SharedJsonSerialization/Writing/SharedWritingExtensionsTests.cs +++ b/tests/Light.PortableResults.Tests/SharedJsonSerialization/Writing/SharedWritingExtensionsTests.cs @@ -1,7 +1,9 @@ using System; +using System.Buffers; using System.Globalization; using System.IO; using System.Text; +using System.Text.Encodings.Web; using System.Text.Json; using System.Text.Json.Serialization.Metadata; using FluentAssertions; @@ -13,6 +15,8 @@ namespace Light.PortableResults.Tests.SharedJsonSerialization.Writing; public sealed class SharedWritingExtensionsTests { + private const int AllocationIterations = 1_000; + [Fact] public void WriteMetadataValue_ShouldThrow_WhenWriterIsNull() { @@ -162,6 +166,63 @@ public void WriteMetadataArray_ShouldWriteDecimalElementsAsUnquotedNumbers() json.Should().Be("[9.99,99.90]"); } + [Theory] + [InlineData(false)] + [InlineData(true)] + public void TextBearingMetadataShouldPreserveTheWriterUtf16RouteBytes(bool useUnsafeEncoder) + { + var values = new (MetadataValue Value, string DefaultJson, string UnsafeJson)[] + { + (MetadataValue.FromString("Grüße"), "\"Gr\\u00FC\\u00DFe\"", "\"Grüße\""), + (MetadataValue.FromString("a\uD800b"), "\"a\\uFFFDb\"", "\"a\\uFFFDb\""), + (MetadataValue.FromChar('\uD800'), "\"\\uFFFD\"", "\"\\uFFFD\"") + }; + var options = new JsonWriterOptions + { + Encoder = useUnsafeEncoder ? JavaScriptEncoder.UnsafeRelaxedJsonEscaping : null + }; + + foreach (var (value, defaultJson, unsafeJson) in values) + { + var actual = SerializeToUtf8( + writer => writer.WriteMetadataValue( + value, + MetadataValueAnnotation.SerializeInHttpResponseBody + ), + options + ); + + actual.Should().Equal( + Encoding.UTF8.GetBytes(useUnsafeEncoder ? unsafeJson : defaultJson), + value.Kind.ToString() + ); + } + } + + [Fact] + public void CanonicallyFormattedJsonValuesShouldNotAllocateStrings() + { + var values = new[] + { + MetadataValue.FromDouble(36_028_797_018_963_968.0), + MetadataValue.FromSingle(123_456_789f), + MetadataValue.FromUInt64(ulong.MaxValue), + MetadataValue.FromDateTime( + new DateTime(2026, 7, 26, 13, 45, 30, DateTimeKind.Utc).AddTicks(1_234_567) + ), + MetadataValue.FromTimeSpan(TimeSpan.MaxValue), + MetadataValue.FromGuid(new Guid("a1b2c3d4-e5f6-7890-abcd-ef1234567890")), + MetadataValue.FromString("Grüße 日本語 😀"), + MetadataValue.FromChar('ß'), + MetadataValue.FromUri(new Uri("https://example.com/Grüße?q=日本語")) + }; + + foreach (var value in values) + { + MeasureWriterAllocations(value).Should().Be(0, value.Kind.ToString()); + } + } + [Fact] public void WriteRichErrors_ShouldThrow_WhenWriterIsNull() { @@ -229,4 +290,44 @@ private static string Serialize(Action writeAction) return Encoding.UTF8.GetString(stream.ToArray()); } + + private static byte[] SerializeToUtf8( + Action writeAction, + JsonWriterOptions options + ) + { + var output = new ArrayBufferWriter(); + using var writer = new Utf8JsonWriter(output, options); + writeAction(writer); + writer.Flush(); + return output.WrittenSpan.ToArray(); + } + + private static long MeasureWriterAllocations(MetadataValue value) + { + var minimumAllocations = long.MaxValue; + for (var sample = 0; sample < 5; sample++) + { + var output = new ArrayBufferWriter(1024 * 1024); + using var writer = new Utf8JsonWriter(output); + writer.WriteStartArray(); + for (var index = 0; index < 100; index++) + { + writer.WriteMetadataValue(value, MetadataValueAnnotation.SerializeInHttpResponseBody); + } + + var before = GC.GetAllocatedBytesForCurrentThread(); + for (var index = 0; index < AllocationIterations; index++) + { + writer.WriteMetadataValue(value, MetadataValueAnnotation.SerializeInHttpResponseBody); + } + + minimumAllocations = Math.Min( + minimumAllocations, + GC.GetAllocatedBytesForCurrentThread() - before + ); + } + + return minimumAllocations; + } }