From 8fd262da5ae8ba6f0deb97319ddfd8b59a0bbc89 Mon Sep 17 00:00:00 2001 From: Maximum Code Date: Sun, 26 Apr 2026 18:37:53 -0700 Subject: [PATCH 1/4] Prepare collection binary payload release --- docs/roadmap.md | 6 + .../CollectionDocumentCodec.cs | 20 + .../CollectionFieldAccessor.cs | 3 + src/CSharpDB.Engine/CollectionIndexBinding.cs | 125 +- .../CollectionIndexedFieldReader.cs | 225 +- src/CSharpDB.Engine/CollectionModels.cs | 15 +- src/CSharpDB.Engine/README.md | 3 + .../CollectionModelGenerator.cs | 3259 ++++++++++++++++- src/CSharpDB.Generators/README.md | 6 + .../Indexing/OrderedTextIndexKeyCodec.cs | 10 +- .../CollectionBinaryDocumentCodec.cs | 150 + .../Serialization/CollectionPayloadCodec.cs | 320 +- .../CSharpDB.Benchmarks.csproj | 1 + tests/CSharpDB.Benchmarks/HISTORY.md | 132 +- .../GeneratedCollectionCodecBenchmarks.cs | 378 ++ tests/CSharpDB.Benchmarks/README.md | 104 +- .../release-core-manifest.json | 26 +- .../CollectionBinaryDocumentCodecTests.cs | 6 + .../GeneratedCollectionModelTests.cs | 146 + .../OrderedTextIndexKeyCodecTests.cs | 1 + 20 files changed, 4821 insertions(+), 115 deletions(-) create mode 100644 tests/CSharpDB.Benchmarks/Micro/GeneratedCollectionCodecBenchmarks.cs diff --git a/docs/roadmap.md b/docs/roadmap.md index bb217aa7..084b48a2 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -56,6 +56,8 @@ SQL feature parity, provider/tooling compatibility, and ecosystem expansion. | **NuGet package** | Publish and maintain `CSharpDB.Engine`, `CSharpDB.Data`, `CSharpDB.Client`, and `CSharpDB.Primitives` as the primary NuGet packages | Done | | **Connection pooling** | Pool underlying direct embedded sessions behind `CSharpDbConnection` to amortize open/close cost | Done | | **Admin dashboard improvements** | Richer SQL editor UX, query history, deeper diagnostics, and integrated Forms/Reports tooling beyond the core schema/procedure/storage surface | Done | +| **Admin Forms Access parity** | Close the highest-impact Access-style form gaps: runtime responsive layouts, full inferred validation enforcement, richer record-source/filter/sort models, Layout View, form modes, command/action events, and broader control coverage | Planned | +| **Admin Reports Access parity** | Close the highest-impact Access-style report gaps: bounded saved-query previews, full report rendering/export, parameter/filter prompts, richer grouping/totals options, Layout View, conditional formatting, subreports, and broader report controls | Planned | | **Visual query designer** | Classic Admin query builder with source canvas, join editing, design grid, SQL preview, and saved designer layouts | Done | | **ETL pipelines** | Built-in package-driven pipeline runtime with validation, dry-run, execute/resume flows, API/CLI/client coverage, run history, and Admin visual designer support | Done | | **VS Code extension** | Schema explorer, SQL editor with IntelliSense, data browser, table designer, storage diagnostics | Done | @@ -104,6 +106,8 @@ These are known simplifications in the current implementation: | **Collections** | `FindByIndexAsync` supports declared field-equality lookups; `FindByPathAsync` and `FindByPathRangeAsync` support path-based queries on indexed paths; `FindAsync` remains a full scan for unindexed predicates | | **Networking** | `CSharpDB.Daemon` now hosts both REST and gRPC from one process; named pipes remain reserved but are not implemented end to end today | | **Security** | Remote HTTP and gRPC deployment still rely on external network controls or front-end TLS termination; built-in authentication, authorization, and TLS/mTLS support are still planned | +| **Admin Forms** | The Forms designer/runtime supports the core generated-form and data-entry path, but still needs Access-parity work for responsive runtime rendering, complete inferred validation, richer form modes, command/action events, advanced filtering/sorting, and broader controls | +| **Admin Reports** | The Reports designer/runtime supports the core banded preview path, but still needs Access-parity work for bounded saved-query previews, full report output/export, parameters, richer grouping and totals semantics, conditional formatting, subreports, and broader controls | | **Text / Multilingual** | Text is stored as UTF-8 and supports all Unicode languages; default semantics remain ordinal, but opt-in `BINARY`, `NOCASE`, `NOCASE_AI`, and `ICU:` collation are implemented for SQL and collection indexes. Dedicated ordered SQL text index optimization remains planned | | **Concurrency** | The physical WAL commit path is still serialized at the storage boundary. Initial multi-writer support is shipped, but observed gains still depend on conflict shape and whether shared auto-commit `INSERT` is left on the default serialized path | | **Storage** | No page-level compression | @@ -185,4 +189,6 @@ Major features already implemented: - [Native FFI Tutorials](tutorials/native-ffi/README.md) — Python and Node.js examples using the NativeAOT shared library - [User-Defined Functions Plan](user-defined-functions/README.md) — C# library functions callable by the database, native plugin extensions, and WASM sandboxing - [Pub/Sub Change Events Plan](pub-sub-events/README.md) — Engine-level change events with channel-based delivery for real-time data subscriptions +- [Admin Forms Access Parity Plan](admin-forms-access-parity/README.md) — Microsoft Access parity review findings and forms roadmap +- [Admin Reports Access Parity Plan](admin-reports-access-parity/README.md) — Microsoft Access parity review findings and reports roadmap - [Benchmark Suite](../tests/CSharpDB.Benchmarks/README.md) — Performance data informing optimization priorities diff --git a/src/CSharpDB.Engine/CollectionDocumentCodec.cs b/src/CSharpDB.Engine/CollectionDocumentCodec.cs index 518d396f..65c8795b 100644 --- a/src/CSharpDB.Engine/CollectionDocumentCodec.cs +++ b/src/CSharpDB.Engine/CollectionDocumentCodec.cs @@ -78,6 +78,26 @@ public byte[] Encode(string key, T document) if (_generatedCodec is not null) return _generatedCodec.Decode(payload); + if (UsesDirectPayloadFormat && + CollectionPayloadCodec.TryReadFastHeader(payload, out var header)) + { + string key = Encoding.UTF8.GetString(CollectionPayloadCodec.GetKeyUtf8(payload, header)); + ReadOnlySpan documentPayload = CollectionPayloadCodec.GetDocumentPayload(payload, header); + + try + { + T document = header.Format == CollectionPayloadCodec.CollectionPayloadFormat.Binary + ? CollectionBinaryDocumentCodec.Decode(documentPayload) + : JsonSerializer.Deserialize(documentPayload, s_jsonOptions)!; + + return (key, document); + } + catch (Exception ex) when (IsFastHeaderFallbackCandidate(ex)) + { + // Fall through to the existing slower path for marker collisions or corrupt direct payloads. + } + } + return (DecodeKey(payload), DecodeDocument(payload)); } diff --git a/src/CSharpDB.Engine/CollectionFieldAccessor.cs b/src/CSharpDB.Engine/CollectionFieldAccessor.cs index 837ac559..9b2ffaf3 100644 --- a/src/CSharpDB.Engine/CollectionFieldAccessor.cs +++ b/src/CSharpDB.Engine/CollectionFieldAccessor.cs @@ -112,6 +112,9 @@ internal bool TryReadInt64(ReadOnlySpan payload, out long value) internal bool TryReadString(ReadOnlySpan payload, out string? value) => CollectionIndexedFieldReader.TryReadString(payload, this, out value); + internal bool TryReadStringUtf8(ReadOnlySpan payload, out ReadOnlySpan value) + => CollectionIndexedFieldReader.TryReadStringUtf8(payload, this, out value); + internal bool TryReadBoolean(ReadOnlySpan payload, out bool value) => CollectionIndexedFieldReader.TryReadBoolean(payload, this, out value); diff --git a/src/CSharpDB.Engine/CollectionIndexBinding.cs b/src/CSharpDB.Engine/CollectionIndexBinding.cs index d82cfc11..be236336 100644 --- a/src/CSharpDB.Engine/CollectionIndexBinding.cs +++ b/src/CSharpDB.Engine/CollectionIndexBinding.cs @@ -1,3 +1,4 @@ +using System.Buffers; using System.Diagnostics.CodeAnalysis; using System.Globalization; using System.Linq.Expressions; @@ -16,6 +17,7 @@ internal sealed class CollectionIndexBinding< { private readonly Func _fieldAccessor; private readonly CollectionFieldAccessor _payloadAccessor; + private readonly CollectionField? _fieldDescriptor; private readonly CollectionIndexDataKind _valueKind; private readonly string? _collation; @@ -26,13 +28,15 @@ private CollectionIndexBinding( Func fieldAccessor, CollectionFieldAccessor payloadAccessor, CollectionIndexDataKind valueKind, - string? collation) + string? collation, + CollectionField? fieldDescriptor = null) { FieldPath = fieldPath; IndexName = indexName; IndexStore = indexStore; _fieldAccessor = fieldAccessor; _payloadAccessor = payloadAccessor; + _fieldDescriptor = fieldDescriptor; _valueKind = valueKind; _collation = ValidateCollationForValueKind(valueKind, collation, fieldPath); } @@ -181,7 +185,8 @@ internal static CollectionIndexBinding Create( field.ReadValue, field.PayloadAccessor, field.DataKind, - collation); + collation, + field); } internal static Func CreateFieldAccessor(string fieldPath) @@ -250,14 +255,21 @@ internal bool TryBuildKeyFromDirectPayload(ReadOnlySpan payload, out long indexKey = 0; if (_valueKind == CollectionIndexDataKind.Integer) { - if (!_payloadAccessor.TryReadInt64(payload, out long integerValue)) + if (!TryReadPayloadInt64(payload, out long integerValue)) return false; indexKey = integerValue; return true; } - if (!_payloadAccessor.TryReadString(payload, out string? textValue) || textValue == null) + if (CollationSupport.IsBinaryOrDefault(_collation) && + TryReadPayloadStringUtf8(payload, out ReadOnlySpan textUtf8)) + { + indexKey = OrderedTextIndexKeyCodec.ComputeKey(textUtf8); + return true; + } + + if (!TryReadPayloadString(payload, out string? textValue) || textValue == null) return false; return TryBuildKey(DbValue.FromText(textValue), out indexKey); @@ -271,7 +283,7 @@ internal bool TryGetSingleTextValueFromDirectPayload( if (!UsesTextKey || IsMultiValueArray) return false; - if (!_payloadAccessor.TryReadString(payload, out string? rawText) || rawText == null) + if (!TryReadPayloadString(payload, out string? rawText) || rawText == null) return false; textValue = NormalizeTextForIndex(rawText); @@ -317,7 +329,7 @@ internal bool TryCollectTextValuesFromDirectPayload(ReadOnlySpan payload, if (!IsMultiValueArray) { - if (_payloadAccessor.TryReadString(payload, out string? textValue) && textValue != null) + if (TryReadPayloadString(payload, out string? textValue) && textValue != null) textValues.Add(NormalizeTextForIndex(textValue)); return textValues.Count != startCount; @@ -340,6 +352,14 @@ internal bool TryDirectPayloadValueEquals(ReadOnlySpan payload, DbValue va { if (!IsMultiValueArray) { + if (UsesTextKey && + value.Type == DbType.Text && + CollationSupport.IsBinaryOrDefault(_collation) && + TryDirectPayloadTextEqualsBinary(payload, value.AsText, out bool textEquals)) + { + return textEquals; + } + if (!TryReadComparableValue(payload, out var actualValue)) return false; @@ -475,7 +495,7 @@ private bool TryReadComparableValue(ReadOnlySpan payload, out DbValue valu value = DbValue.Null; if (!IsMultiValueArray) - return _payloadAccessor.TryReadValue(payload, out value); + return TryReadPayloadValue(payload, out value); var values = new List(); if (!_payloadAccessor.TryReadIndexValues(payload, values)) @@ -493,6 +513,50 @@ private bool TryReadComparableValue(ReadOnlySpan payload, out DbValue valu return false; } + private bool TryReadPayloadValue(ReadOnlySpan payload, out DbValue value) + { + if (_fieldDescriptor is not null) + return _fieldDescriptor.TryReadPayloadValue(payload, out value); + + return _payloadAccessor.TryReadValue(payload, out value); + } + + private bool TryReadPayloadInt64(ReadOnlySpan payload, out long value) + { + if (_fieldDescriptor is not null) + return _fieldDescriptor.TryReadPayloadInt64(payload, out value); + + return _payloadAccessor.TryReadInt64(payload, out value); + } + + private bool TryReadPayloadString(ReadOnlySpan payload, out string? value) + { + if (_fieldDescriptor is not null) + return _fieldDescriptor.TryReadPayloadString(payload, out value); + + return _payloadAccessor.TryReadString(payload, out value); + } + + private bool TryReadPayloadStringUtf8(ReadOnlySpan payload, out ReadOnlySpan value) + { + if (_fieldDescriptor is not null) + return _fieldDescriptor.TryReadPayloadStringUtf8(payload, out value); + + return _payloadAccessor.TryReadStringUtf8(payload, out value); + } + + private bool TryDirectPayloadTextEqualsBinary(ReadOnlySpan payload, string expectedText, out bool equals) + { + if (TryReadPayloadStringUtf8(payload, out ReadOnlySpan textUtf8)) + { + equals = Utf8EqualsString(textUtf8, expectedText); + return true; + } + + equals = false; + return false; + } + private bool ValueMatches(object? actual, DbValue expected) { if (!IsMultiValueArray) @@ -521,6 +585,53 @@ private bool ValueMatches(object? actual, DbValue expected) private string NormalizeTextForIndex(string text) => CollationSupport.NormalizeText(text, _collation); + private static bool Utf8EqualsString(ReadOnlySpan utf8, string text) + { + if (utf8.Length == text.Length) + { + bool ascii = true; + for (int i = 0; i < text.Length; i++) + { + char ch = text[i]; + if (ch > 0x7F) + { + ascii = false; + break; + } + + if (utf8[i] != (byte)ch) + return false; + } + + if (ascii) + return true; + } + + int byteCount = Encoding.UTF8.GetByteCount(text); + if (byteCount != utf8.Length) + return false; + + const int StackallocTextThreshold = 256; + byte[]? rented = null; + Span expected = byteCount <= StackallocTextThreshold + ? stackalloc byte[StackallocTextThreshold] + : (rented = ArrayPool.Shared.Rent(byteCount)); + + try + { + int written = Encoding.UTF8.GetBytes(text.AsSpan(), expected); + return utf8.SequenceEqual(expected[..written]); + } + finally + { + if (rented is not null) + { + expected[..byteCount].Clear(); + ArrayPool.Shared.Return(rented); + } + } + } + private static MemberInfo[] ResolveMemberPath(string[] segments, bool[] arraySegments, string fieldPath) { var memberPath = new MemberInfo[segments.Length]; diff --git a/src/CSharpDB.Engine/CollectionIndexedFieldReader.cs b/src/CSharpDB.Engine/CollectionIndexedFieldReader.cs index f95be8a1..125c823d 100644 --- a/src/CSharpDB.Engine/CollectionIndexedFieldReader.cs +++ b/src/CSharpDB.Engine/CollectionIndexedFieldReader.cs @@ -1,3 +1,4 @@ +using System.Buffers; using System.Text.Json; using CSharpDB.Primitives; using CSharpDB.Storage.Serialization; @@ -6,6 +7,8 @@ namespace CSharpDB.Engine; internal static class CollectionIndexedFieldReader { + private const int StackallocPropertyNameThreshold = 256; + public static bool TryReadInt64(ReadOnlySpan payload, CollectionFieldAccessor accessor, out long value) { ArgumentNullException.ThrowIfNull(accessor); @@ -15,8 +18,70 @@ public static bool TryReadInt64(ReadOnlySpan payload, CollectionFieldAcces public static bool TryReadInt64(ReadOnlySpan payload, string jsonPropertyName, out long value) { ArgumentNullException.ThrowIfNull(jsonPropertyName); - byte[][] pathSegments = [System.Text.Encoding.UTF8.GetBytes(jsonPropertyName)]; - return TryReadInt64(payload, pathSegments, out value); + if (!CollectionPayloadCodec.TryReadFastHeader(payload, out var header)) + { + if (!CollectionPayloadCodec.TryReadValidatedHeader(payload, out header)) + { + value = default; + return false; + } + } + + try + { + ReadOnlySpan documentPayload = CollectionPayloadCodec.GetDocumentPayload(payload, header); + if (header.Format == CollectionPayloadCodec.CollectionPayloadFormat.Binary) + { + int byteCount = System.Text.Encoding.UTF8.GetByteCount(jsonPropertyName); + byte[]? rented = null; + Span propertyNameUtf8 = byteCount <= StackallocPropertyNameThreshold + ? stackalloc byte[StackallocPropertyNameThreshold] + : (rented = ArrayPool.Shared.Rent(byteCount)); + + try + { + int written = System.Text.Encoding.UTF8.GetBytes(jsonPropertyName.AsSpan(), propertyNameUtf8); + return CollectionBinaryDocumentCodec.TryReadInt64( + documentPayload, + propertyNameUtf8[..written], + out value); + } + finally + { + if (rented is not null) + { + propertyNameUtf8[..byteCount].Clear(); + ArrayPool.Shared.Return(rented); + } + } + } + + var reader = new Utf8JsonReader(documentPayload, isFinalBlock: true, state: default); + while (reader.Read()) + { + if (reader.TokenType != JsonTokenType.PropertyName || reader.CurrentDepth != 1) + continue; + + if (!reader.ValueTextEquals(jsonPropertyName.AsSpan())) + continue; + + if (!reader.Read()) + break; + + if (reader.TokenType == JsonTokenType.Number && reader.TryGetInt64(out value)) + return true; + + break; + } + + value = default; + return false; + } + catch (Exception ex) when (IsFastHeaderFallbackCandidate(ex)) + { + value = default; + return false; + } } public static bool TryReadString(ReadOnlySpan payload, CollectionFieldAccessor accessor, out string? value) @@ -25,11 +90,82 @@ public static bool TryReadString(ReadOnlySpan payload, CollectionFieldAcce return TryReadString(payload, accessor.JsonPathSegmentsUtf8, out value); } + public static bool TryReadStringUtf8(ReadOnlySpan payload, CollectionFieldAccessor accessor, out ReadOnlySpan value) + { + ArgumentNullException.ThrowIfNull(accessor); + return TryReadStringUtf8(payload, accessor.JsonPathSegmentsUtf8, out value); + } + public static bool TryReadString(ReadOnlySpan payload, string jsonPropertyName, out string? value) { ArgumentNullException.ThrowIfNull(jsonPropertyName); - byte[][] pathSegments = [System.Text.Encoding.UTF8.GetBytes(jsonPropertyName)]; - return TryReadString(payload, pathSegments, out value); + if (!CollectionPayloadCodec.TryReadFastHeader(payload, out var header)) + { + if (!CollectionPayloadCodec.TryReadValidatedHeader(payload, out header)) + { + value = null; + return false; + } + } + + try + { + ReadOnlySpan documentPayload = CollectionPayloadCodec.GetDocumentPayload(payload, header); + if (header.Format == CollectionPayloadCodec.CollectionPayloadFormat.Binary) + { + int byteCount = System.Text.Encoding.UTF8.GetByteCount(jsonPropertyName); + byte[]? rented = null; + Span propertyNameUtf8 = byteCount <= StackallocPropertyNameThreshold + ? stackalloc byte[StackallocPropertyNameThreshold] + : (rented = ArrayPool.Shared.Rent(byteCount)); + + try + { + int written = System.Text.Encoding.UTF8.GetBytes(jsonPropertyName.AsSpan(), propertyNameUtf8); + return CollectionBinaryDocumentCodec.TryReadString( + documentPayload, + propertyNameUtf8[..written], + out value); + } + finally + { + if (rented is not null) + { + propertyNameUtf8[..byteCount].Clear(); + ArrayPool.Shared.Return(rented); + } + } + } + + var reader = new Utf8JsonReader(documentPayload, isFinalBlock: true, state: default); + while (reader.Read()) + { + if (reader.TokenType != JsonTokenType.PropertyName || reader.CurrentDepth != 1) + continue; + + if (!reader.ValueTextEquals(jsonPropertyName.AsSpan())) + continue; + + if (!reader.Read()) + break; + + if (reader.TokenType == JsonTokenType.String) + { + value = reader.GetString(); + return true; + } + + break; + } + + value = null; + return false; + } + catch (Exception ex) when (IsFastHeaderFallbackCandidate(ex)) + { + value = null; + return false; + } } public static bool TryReadBoolean(ReadOnlySpan payload, CollectionFieldAccessor accessor, out bool value) @@ -77,11 +213,28 @@ public static bool TryReadValue(ReadOnlySpan payload, string jsonPropertyN ReadOnlySpan documentPayload = CollectionPayloadCodec.GetDocumentPayload(payload, header); if (header.Format == CollectionPayloadCodec.CollectionPayloadFormat.Binary) { - byte[][] pathSegments = [System.Text.Encoding.UTF8.GetBytes(jsonPropertyName)]; - return CollectionBinaryDocumentCodec.TryReadValue( - documentPayload, - pathSegments, - out value); + int byteCount = System.Text.Encoding.UTF8.GetByteCount(jsonPropertyName); + byte[]? rented = null; + Span propertyNameUtf8 = byteCount <= StackallocPropertyNameThreshold + ? stackalloc byte[StackallocPropertyNameThreshold] + : (rented = ArrayPool.Shared.Rent(byteCount)); + + try + { + int written = System.Text.Encoding.UTF8.GetBytes(jsonPropertyName.AsSpan(), propertyNameUtf8); + return CollectionBinaryDocumentCodec.TryReadValue( + documentPayload, + propertyNameUtf8[..written], + out value); + } + finally + { + if (rented is not null) + { + propertyNameUtf8[..byteCount].Clear(); + ArrayPool.Shared.Return(rented); + } + } } var reader = new Utf8JsonReader(documentPayload, isFinalBlock: true, state: default); @@ -138,11 +291,28 @@ public static bool TryTextEquals(ReadOnlySpan payload, string jsonProperty ReadOnlySpan documentPayload = CollectionPayloadCodec.GetDocumentPayload(payload, header); if (header.Format == CollectionPayloadCodec.CollectionPayloadFormat.Binary) { - byte[][] pathSegments = [System.Text.Encoding.UTF8.GetBytes(jsonPropertyName)]; - return CollectionBinaryDocumentCodec.TryTextEquals( - documentPayload, - pathSegments, - expectedValue); + int byteCount = System.Text.Encoding.UTF8.GetByteCount(jsonPropertyName); + byte[]? rented = null; + Span propertyNameUtf8 = byteCount <= StackallocPropertyNameThreshold + ? stackalloc byte[StackallocPropertyNameThreshold] + : (rented = ArrayPool.Shared.Rent(byteCount)); + + try + { + int written = System.Text.Encoding.UTF8.GetBytes(jsonPropertyName.AsSpan(), propertyNameUtf8); + return CollectionBinaryDocumentCodec.TryTextEquals( + documentPayload, + propertyNameUtf8[..written], + expectedValue); + } + finally + { + if (rented is not null) + { + propertyNameUtf8[..byteCount].Clear(); + ArrayPool.Shared.Return(rented); + } + } } var reader = new Utf8JsonReader(documentPayload, isFinalBlock: true, state: default); @@ -245,6 +415,33 @@ private static bool TryReadString(ReadOnlySpan payload, byte[][] jsonPathS } } + private static bool TryReadStringUtf8(ReadOnlySpan payload, byte[][] jsonPathSegmentsUtf8, out ReadOnlySpan value) + { + if (!CollectionPayloadCodec.TryReadFastHeader(payload, out var header)) + { + if (!CollectionPayloadCodec.TryReadValidatedHeader(payload, out header)) + { + value = default; + return false; + } + } + + try + { + ReadOnlySpan documentPayload = CollectionPayloadCodec.GetDocumentPayload(payload, header); + if (header.Format == CollectionPayloadCodec.CollectionPayloadFormat.Binary) + return CollectionBinaryDocumentCodec.TryReadStringUtf8(documentPayload, jsonPathSegmentsUtf8, out value); + + value = default; + return false; + } + catch (Exception ex) when (IsFastHeaderFallbackCandidate(ex)) + { + value = default; + return false; + } + } + private static bool TryReadBoolean(ReadOnlySpan payload, byte[][] jsonPathSegmentsUtf8, out bool value) { if (!CollectionPayloadCodec.TryReadFastHeader(payload, out var header)) diff --git a/src/CSharpDB.Engine/CollectionModels.cs b/src/CSharpDB.Engine/CollectionModels.cs index c5ff60c5..78312962 100644 --- a/src/CSharpDB.Engine/CollectionModels.cs +++ b/src/CSharpDB.Engine/CollectionModels.cs @@ -1,4 +1,5 @@ using System.Diagnostics.CodeAnalysis; +using CSharpDB.Primitives; using CSharpDB.Storage.Serialization; namespace CSharpDB.Engine; @@ -152,6 +153,18 @@ protected CollectionField( internal object? ReadValue(TDocument document) => ReadValueCore(document); + public virtual bool TryReadPayloadValue(ReadOnlySpan payload, out DbValue value) + => _payloadAccessor.TryReadValue(payload, out value); + + public virtual bool TryReadPayloadInt64(ReadOnlySpan payload, out long value) + => _payloadAccessor.TryReadInt64(payload, out value); + + public virtual bool TryReadPayloadString(ReadOnlySpan payload, out string? value) + => _payloadAccessor.TryReadString(payload, out value); + + public virtual bool TryReadPayloadStringUtf8(ReadOnlySpan payload, out ReadOnlySpan value) + => _payloadAccessor.TryReadStringUtf8(payload, out value); + protected abstract object? ReadValueCore(TDocument document); private static string NormalizeFieldPath(string fieldPath, out bool targetsArrayElements) @@ -208,7 +221,7 @@ private static string NormalizeFieldPath(string fieldPath, out bool targetsArray /// /// Strongly typed collection field descriptor for generated collection APIs. /// -public sealed class CollectionField : CollectionField +public class CollectionField : CollectionField { private readonly Func _accessor; diff --git a/src/CSharpDB.Engine/README.md b/src/CSharpDB.Engine/README.md index 7947b985..2a0ce3f3 100644 --- a/src/CSharpDB.Engine/README.md +++ b/src/CSharpDB.Engine/README.md @@ -309,6 +309,9 @@ through `JsonPropertyName` without changing the public descriptor names. `GetGeneratedCollectionAsync(...)` requires a generated or manually registered collection model and exposes only the descriptor-based collection surface. That keeps the call path off the reflection-based collection APIs. +For supported document graphs, generated codecs write the binary direct-payload +format; generated models with unsupported binary member shapes continue to use +the source-generated JSON payload path. Generated collections also require existing collection indexes on that collection to resolve through registered generated descriptors. If a collection diff --git a/src/CSharpDB.Generators/CollectionModelGenerator.cs b/src/CSharpDB.Generators/CollectionModelGenerator.cs index 589fbf37..ecb280bc 100644 --- a/src/CSharpDB.Generators/CollectionModelGenerator.cs +++ b/src/CSharpDB.Generators/CollectionModelGenerator.cs @@ -181,7 +181,8 @@ private static CollectionGenerationResult InspectTarget(GeneratorAttributeSyntax jsonContextType.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat), GetPartialTypeKeyword(typeSymbol), MakeSafeIdentifier(typeSymbol.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat)), - fields.ToImmutable()), + fields.ToImmutable(), + TryCreateBinaryTypeSpec(typeSymbol, ImmutableArray.Empty)), diagnostics.ToImmutable()); } @@ -501,6 +502,271 @@ private static bool TryGetCollectionElementType(ITypeSymbol type, out ITypeSymbo return false; } + private static BinaryTypeSpec? TryCreateBinaryTypeSpec( + INamedTypeSymbol type, + ImmutableArray recursionStack) + { + if (ContainsType(recursionStack, type)) + return null; + + var memberCandidates = ImmutableArray.CreateBuilder(); + ImmutableArray nextStack = recursionStack.Add(type); + foreach (ISymbol member in type.GetMembers().OrderBy(static member => member.Name, StringComparer.Ordinal)) + { + if (!TryGetCollectionMember(member, out ITypeSymbol? memberType) || memberType is null) + continue; + + if (!TryCreateBinaryValueSpec(memberType, nextStack, out BinaryValueSpec valueSpec)) + return null; + + memberCandidates.Add(new BinaryMemberCandidate( + member, + memberType, + valueSpec)); + } + + ImmutableArray candidateMembers = memberCandidates.ToImmutable(); + if (!TryCreateBinaryConstructorSpec(type, candidateMembers, out BinaryConstructorSpec constructor)) + return null; + + candidateMembers = OrderBinaryMembersByConstructor(candidateMembers, constructor, out constructor); + + var members = ImmutableArray.CreateBuilder(candidateMembers.Length); + for (int i = 0; i < candidateMembers.Length; i++) + { + BinaryMemberCandidate candidate = candidateMembers[i]; + ISymbol member = candidate.Member; + members.Add(new BinaryMemberSpec( + member.Name, + EscapeIdentifier(member.Name), + GetJsonPropertyName(member) ?? JsonNamingPolicy.CamelCase.ConvertName(member.Name), + candidate.Value)); + } + + return new BinaryTypeSpec( + type.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat), + MakeSafeIdentifier(type.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat)), + members.ToImmutable(), + constructor); + } + + private static bool TryCreateBinaryConstructorSpec( + INamedTypeSymbol type, + ImmutableArray members, + out BinaryConstructorSpec constructor) + { + foreach (IMethodSymbol candidate in type.InstanceConstructors) + { + if (!CanGeneratedCodeCall(candidate) || + candidate.Parameters.Length != members.Length) + { + continue; + } + + var memberIndexes = ImmutableArray.CreateBuilder(candidate.Parameters.Length); + var usedMemberIndexes = new HashSet(); + bool matches = true; + foreach (IParameterSymbol parameter in candidate.Parameters) + { + int memberIndex = FindConstructorMemberIndex(parameter, members, usedMemberIndexes); + if (memberIndex < 0) + { + matches = false; + break; + } + + usedMemberIndexes.Add(memberIndex); + memberIndexes.Add(memberIndex); + } + + if (!matches) + continue; + + constructor = new BinaryConstructorSpec(memberIndexes.ToImmutable()); + return true; + } + + constructor = default; + return false; + } + + private static ImmutableArray OrderBinaryMembersByConstructor( + ImmutableArray members, + BinaryConstructorSpec constructor, + out BinaryConstructorSpec remappedConstructor) + { + var orderedMembers = ImmutableArray.CreateBuilder(members.Length); + var remappedParameterMemberIndexes = ImmutableArray.CreateBuilder(constructor.ParameterMemberIndexes.Length); + var usedMemberIndexes = new HashSet(); + + for (int i = 0; i < constructor.ParameterMemberIndexes.Length; i++) + { + int memberIndex = constructor.ParameterMemberIndexes[i]; + if ((uint)memberIndex >= (uint)members.Length || !usedMemberIndexes.Add(memberIndex)) + throw new InvalidOperationException("Generated binary constructor member indexes are invalid."); + + remappedParameterMemberIndexes.Add(orderedMembers.Count); + orderedMembers.Add(members[memberIndex]); + } + + for (int i = 0; i < members.Length; i++) + { + if (usedMemberIndexes.Add(i)) + orderedMembers.Add(members[i]); + } + + remappedConstructor = new BinaryConstructorSpec(remappedParameterMemberIndexes.ToImmutable()); + return orderedMembers.ToImmutable(); + } + + private static bool CanGeneratedCodeCall(IMethodSymbol constructor) + => constructor.DeclaredAccessibility is + Accessibility.Public or + Accessibility.Internal or + Accessibility.ProtectedOrInternal; + + private static int FindConstructorMemberIndex( + IParameterSymbol parameter, + ImmutableArray members, + HashSet usedMemberIndexes) + { + for (int i = 0; i < members.Length; i++) + { + if (usedMemberIndexes.Contains(i)) + continue; + + BinaryMemberCandidate member = members[i]; + if (!string.Equals(parameter.Name, member.Member.Name, StringComparison.OrdinalIgnoreCase)) + continue; + + if (!SymbolEqualityComparer.Default.Equals(parameter.Type, member.Type)) + continue; + + return i; + } + + return -1; + } + + private static bool TryCreateBinaryValueSpec( + ITypeSymbol type, + ImmutableArray recursionStack, + out BinaryValueSpec valueSpec) + { + if (TryGetBinaryCollectionElementType(type, out ITypeSymbol? elementType) && elementType is not null) + { + if (TryGetCollectionElementType(elementType, out _)) + { + valueSpec = null!; + return false; + } + + if (!TryCreateBinaryElementValueSpec(elementType, recursionStack, out BinaryValueSpec elementSpec)) + { + valueSpec = null!; + return false; + } + + valueSpec = BinaryValueSpec.ForArray( + type.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat), + type is IArrayTypeSymbol, + CanBeNull(type), + elementSpec); + return true; + } + + return TryCreateBinaryElementValueSpec(type, recursionStack, out valueSpec); + } + + private static bool TryCreateBinaryElementValueSpec( + ITypeSymbol type, + ImmutableArray recursionStack, + out BinaryValueSpec valueSpec) + { + ITypeSymbol effectiveType = UnwrapNullable(type); + if (TryGetBinaryScalarKind(effectiveType, out string valueKindName)) + { + valueSpec = BinaryValueSpec.ForScalar( + type.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat), + effectiveType.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat), + valueKindName, + CanBeNull(type), + IsNullableValueType(type)); + return true; + } + + if (TryGetNavigableComplexType(type, out INamedTypeSymbol? nestedType) && + nestedType is not null && + !ContainsType(recursionStack, nestedType)) + { + BinaryTypeSpec? nestedSpec = TryCreateBinaryTypeSpec(nestedType, recursionStack); + if (nestedSpec is not null) + { + valueSpec = BinaryValueSpec.ForObject( + type.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat), + effectiveType.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat), + CanBeNull(type), + IsNullableValueType(type), + nestedSpec); + return true; + } + } + + valueSpec = null!; + return false; + } + + private static bool TryGetBinaryScalarKind(ITypeSymbol type, out string valueKindName) + { + if (type.SpecialType == SpecialType.System_String) + { + valueKindName = "String"; + return true; + } + + if (IsWellKnownType(type, "System.Guid")) + { + valueKindName = "Guid"; + return true; + } + + if (IsWellKnownType(type, "System.DateOnly")) + { + valueKindName = "DateOnly"; + return true; + } + + if (IsWellKnownType(type, "System.TimeOnly")) + { + valueKindName = "TimeOnly"; + return true; + } + + if (type.TypeKind == TypeKind.Enum) + { + valueKindName = "Enum"; + return true; + } + + valueKindName = type.SpecialType switch + { + SpecialType.System_Boolean => "Boolean", + SpecialType.System_Byte => "Byte", + SpecialType.System_SByte => "SByte", + SpecialType.System_Int16 => "Int16", + SpecialType.System_UInt16 => "UInt16", + SpecialType.System_Int32 => "Int32", + SpecialType.System_UInt32 => "UInt32", + SpecialType.System_Int64 => "Int64", + SpecialType.System_Single => "Single", + SpecialType.System_Double => "Double", + SpecialType.System_Decimal => "Decimal", + _ => string.Empty, + }; + + return valueKindName.Length > 0; + } + private static bool IsEnumerableLike(INamedTypeSymbol constructedType) { string displayName = constructedType.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat); @@ -513,6 +779,54 @@ private static bool IsEnumerableLike(INamedTypeSymbol constructedType) "global::System.Collections.Generic.List"; } + private static bool TryGetBinaryCollectionElementType(ITypeSymbol type, out ITypeSymbol? elementType) + { + elementType = null; + if (type.SpecialType == SpecialType.System_String) + return false; + + if (type is IArrayTypeSymbol arrayType) + { + elementType = arrayType.ElementType; + return true; + } + + if (type is not INamedTypeSymbol namedType) + return false; + + if (namedType.IsGenericType && + namedType.TypeArguments.Length == 1 && + IsCountedEnumerableLike(namedType.ConstructedFrom)) + { + elementType = namedType.TypeArguments[0]; + return true; + } + + foreach (INamedTypeSymbol interfaceType in namedType.AllInterfaces) + { + if (interfaceType.IsGenericType && + interfaceType.TypeArguments.Length == 1 && + IsCountedEnumerableLike(interfaceType.ConstructedFrom)) + { + elementType = interfaceType.TypeArguments[0]; + return true; + } + } + + return false; + } + + private static bool IsCountedEnumerableLike(INamedTypeSymbol constructedType) + { + string displayName = constructedType.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat); + return displayName is + "global::System.Collections.Generic.ICollection" or + "global::System.Collections.Generic.IList" or + "global::System.Collections.Generic.IReadOnlyCollection" or + "global::System.Collections.Generic.IReadOnlyList" or + "global::System.Collections.Generic.List"; + } + private static ITypeSymbol UnwrapNullable(ITypeSymbol type) => type is INamedTypeSymbol named && named.OriginalDefinition.SpecialType == SpecialType.System_Nullable_T && @@ -777,6 +1091,111 @@ private static string MakeSafeIdentifier(string input) return builder.ToString(); } + private static string CreateBinaryPayloadFieldMethodName(CollectionFieldSpec field) + => "TryReadBinaryPayloadField_" + MakeSafeIdentifier(field.GeneratedMemberName); + + private static string CreateBinaryPayloadValueReaderName(CollectionFieldSpec field) + => "TryReadPayloadValue_" + MakeSafeIdentifier(field.GeneratedMemberName); + + private static string CreateBinaryPayloadIntegerReaderName(CollectionFieldSpec field) + => "TryReadPayloadInteger_" + MakeSafeIdentifier(field.GeneratedMemberName); + + private static string CreateBinaryPayloadTextReaderName(CollectionFieldSpec field) + => "TryReadPayloadText_" + MakeSafeIdentifier(field.GeneratedMemberName); + + private static string CreateBinaryPayloadTextUtf8ReaderName(CollectionFieldSpec field) + => "TryReadPayloadTextUtf8_" + MakeSafeIdentifier(field.GeneratedMemberName); + + private static bool TryCreateBinaryFieldReaderSpec( + BinaryTypeSpec root, + CollectionFieldSpec field, + out BinaryFieldReaderSpec reader) + { + reader = default; + if (field.PayloadFieldPath.IndexOf('[') >= 0 || + field.PayloadFieldPath.IndexOf(']') >= 0) + { + return false; + } + + string[] pathSegments = field.PayloadFieldPath.Split('.'); + if (pathSegments.Length == 0) + return false; + + var memberIndexes = ImmutableArray.CreateBuilder(pathSegments.Length); + BinaryTypeSpec currentType = root; + BinaryValueSpec? value = null; + for (int i = 0; i < pathSegments.Length; i++) + { + if (!TryFindBinaryMember(currentType, pathSegments[i], out int memberIndex)) + return false; + + memberIndexes.Add(memberIndex); + value = currentType.Members[memberIndex].Value; + if (i == pathSegments.Length - 1) + break; + + if (value.ObjectType is null) + return false; + + currentType = value.ObjectType; + } + + if (value is null || value.IsArray || value.ObjectType is not null) + return false; + + if (!IsSupportedBinaryPayloadFieldReader(value, field.DataKindName)) + return false; + + reader = new BinaryFieldReaderSpec(memberIndexes.ToImmutable(), value, field.DataKindName); + return true; + } + + private static bool TryFindBinaryMember(BinaryTypeSpec type, string jsonName, out int memberIndex) + { + for (int i = 0; i < type.Members.Length; i++) + { + if (string.Equals(type.Members[i].JsonName, jsonName, StringComparison.Ordinal)) + { + memberIndex = i; + return true; + } + } + + memberIndex = -1; + return false; + } + + private static bool IsSupportedBinaryPayloadFieldReader(BinaryValueSpec value, string dataKindName) + => dataKindName switch + { + "Integer" => value.ValueKindName is "Byte" or "SByte" or "Int16" or "UInt16" or "Int32" or "UInt32" or "Int64" or "Enum", + "Text" => value.ValueKindName is "String" or "Guid" or "DateOnly" or "TimeOnly", + _ => false, + }; + + private static bool CanUseBinaryRecordFormat(BinaryTypeSpec type) + { + for (int i = 0; i < type.Members.Length; i++) + { + if (ContainsBinaryArray(type.Members[i].Value)) + return false; + } + + return true; + } + + private static bool ContainsBinaryArray(BinaryValueSpec value) + { + if (value.IsArray) + return true; + + if (value.ObjectType is not null && !CanUseBinaryRecordFormat(value.ObjectType)) + return true; + + return value.Element is not null && ContainsBinaryArray(value.Element); + } + private static void EmitModel(SourceProductionContext context, CollectionModelTarget target) { var source = new StringBuilder(); @@ -796,6 +1215,17 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa foreach (CollectionFieldSpec field in target.Fields) { + BinaryFieldReaderSpec binaryFieldReader = default; + bool hasBinaryPayloadReader = target.BinaryModel is not null && + TryCreateBinaryFieldReaderSpec(target.BinaryModel, field, out binaryFieldReader); + string codecTypeName = "__CSharpDB_CollectionCodec_" + target.SafeIdentifier; + string? fieldTypeName = hasBinaryPayloadReader + ? "__CSharpDB_CollectionField_" + MakeSafeIdentifier(field.GeneratedMemberName) + : null; + + if (fieldTypeName is not null) + EmitGeneratedCollectionFieldType(source, target, field, binaryFieldReader, codecTypeName, fieldTypeName); + source.Append(" public static global::CSharpDB.Engine.CollectionField<") .Append(target.FullyQualifiedTypeName) .Append(", ") @@ -803,6 +1233,15 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa .Append("> ") .Append(field.EscapedMemberName) .AppendLine(" { get; } ="); + if (fieldTypeName is not null) + { + source.Append(" new ") + .Append(fieldTypeName) + .AppendLine("();"); + source.AppendLine(); + continue; + } + source.Append(" new(") .Append(SymbolDisplay.FormatLiteral(field.FieldPath, quote: true)) .Append(", ") @@ -811,6 +1250,37 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa .Append(field.DataKindName) .Append(", ") .Append(SymbolDisplay.FormatLiteral(field.PayloadFieldPath, quote: true)); + if (hasBinaryPayloadReader) + { + source.Append(", ") + .Append(codecTypeName) + .Append('.') + .Append(CreateBinaryPayloadValueReaderName(field)) + .Append(", "); + if (binaryFieldReader.DataKindName == "Integer") + { + source.Append(codecTypeName) + .Append('.') + .Append(CreateBinaryPayloadIntegerReaderName(field)); + } + else + { + source.Append("null"); + } + + source.Append(", "); + if (binaryFieldReader.DataKindName == "Text") + { + source.Append(codecTypeName) + .Append('.') + .Append(CreateBinaryPayloadTextReaderName(field)); + } + else + { + source.Append("null"); + } + } + source.AppendLine(");"); source.AppendLine(); } @@ -879,6 +1349,22 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa .AppendLine(">"); source.AppendLine("{"); source.AppendLine(" private const int StackallocKeyThreshold = 256;"); + if (target.BinaryModel is not null) + { + source.AppendLine(" private const byte RecordFormatMarker = 0xD0;"); + source.AppendLine(" private const byte RecordFormatMagic = 0xF0;"); + source.AppendLine(" private const byte RecordFormatVersion = 0x01;"); + source.AppendLine(" private const byte NullTag = 0;"); + source.AppendLine(" private const byte StringTag = 1;"); + source.AppendLine(" private const byte IntegerTag = 2;"); + source.AppendLine(" private const byte FalseTag = 3;"); + source.AppendLine(" private const byte TrueTag = 4;"); + source.AppendLine(" private const byte DoubleTag = 5;"); + source.AppendLine(" private const byte DecimalTag = 6;"); + source.AppendLine(" private const byte ObjectTag = 7;"); + source.AppendLine(" private const byte ArrayTag = 8;"); + } + source.AppendLine(" private readonly global::CSharpDB.Storage.Serialization.IRecordSerializer _recordSerializer;"); source.AppendLine(" private readonly bool _usesDirectPayloadFormat;"); source.Append(" private static readonly global::System.Text.Json.Serialization.Metadata.JsonTypeInfo<") @@ -908,8 +1394,19 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa source.AppendLine(); source.AppendLine(" if (_usesDirectPayloadFormat)"); source.AppendLine(" {"); - source.AppendLine(" byte[] jsonUtf8 = global::System.Text.Json.JsonSerializer.SerializeToUtf8Bytes(document, s_typeInfo);"); - source.AppendLine(" return global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.Encode(key, jsonUtf8);"); + if (target.BinaryModel is not null) + { + source.AppendLine(" int documentPayloadLength = GetBinaryDocumentSize(document);"); + source.AppendLine(" byte[] payload = global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.EncodeBinary(key, documentPayloadLength, out int documentPayloadStart);"); + source.AppendLine(" WriteBinaryDocument(payload.AsSpan(documentPayloadStart, documentPayloadLength), document);"); + source.AppendLine(" return payload;"); + } + else + { + source.AppendLine(" byte[] jsonUtf8 = global::System.Text.Json.JsonSerializer.SerializeToUtf8Bytes(document, s_typeInfo);"); + source.AppendLine(" return global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.Encode(key, jsonUtf8);"); + } + source.AppendLine(" }"); source.AppendLine(); source.AppendLine(" string json = global::System.Text.Json.JsonSerializer.Serialize(document, s_typeInfo);"); @@ -929,6 +1426,13 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa .Append(target.FullyQualifiedTypeName) .AppendLine(" DecodeDocument(global::System.ReadOnlySpan payload)"); source.AppendLine(" {"); + if (target.BinaryModel is not null) + { + source.AppendLine(" if (_usesDirectPayloadFormat && global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.TryGetBinaryDocumentPayload(payload, out global::System.ReadOnlySpan binaryDocument))"); + source.AppendLine(" return DecodeBinaryDocument(binaryDocument);"); + source.AppendLine(); + } + source.AppendLine(" if (_usesDirectPayloadFormat && global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.IsDirectPayload(payload))"); source.AppendLine(" {"); source.AppendLine(" if (!global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.IsBinaryPayload(payload))"); @@ -949,8 +1453,8 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa source.AppendLine(); source.AppendLine(" public string DecodeKey(global::System.ReadOnlySpan payload)"); source.AppendLine(" {"); - source.AppendLine(" if (_usesDirectPayloadFormat && global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.IsDirectPayload(payload))"); - source.AppendLine(" return global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.DecodeKey(payload);"); + source.AppendLine(" if (_usesDirectPayloadFormat && global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.TryDecodeDirectPayloadKey(payload, out string key))"); + source.AppendLine(" return key;"); source.AppendLine(); source.AppendLine(" global::CSharpDB.Primitives.DbValue[] values = _recordSerializer.DecodeUpTo(payload, 0);"); source.AppendLine(" return values[0].AsText;"); @@ -974,6 +1478,9 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa source.AppendLine(" {"); source.AppendLine(" global::System.ArgumentNullException.ThrowIfNull(expectedKey);"); source.AppendLine(); + source.AppendLine(" if (_usesDirectPayloadFormat && global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.TryDirectPayloadKeyEquals(payload, expectedKey, out bool directEquals))"); + source.AppendLine(" return directEquals;"); + source.AppendLine(); source.AppendLine(" int byteCount = global::System.Text.Encoding.UTF8.GetByteCount(expectedKey);"); source.AppendLine(" byte[]? rented = null;"); source.AppendLine(" global::System.Span utf8 = byteCount <= StackallocKeyThreshold"); @@ -985,9 +1492,6 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa source.AppendLine(" int written = global::System.Text.Encoding.UTF8.GetBytes(expectedKey.AsSpan(), utf8);"); source.AppendLine(" global::System.ReadOnlySpan expectedKeyUtf8 = utf8[..written];"); source.AppendLine(); - source.AppendLine(" if (_usesDirectPayloadFormat && global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.IsDirectPayload(payload))"); - source.AppendLine(" return global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.KeyEquals(payload, expectedKeyUtf8);"); - source.AppendLine(); source.AppendLine(" if (_recordSerializer.TryColumnTextEquals(payload, 0, expectedKeyUtf8, out bool equals))"); source.AppendLine(" return equals;"); source.AppendLine(); @@ -1002,6 +1506,12 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa source.AppendLine(" }"); source.AppendLine(" }"); source.AppendLine(" }"); + if (target.BinaryModel is not null) + { + source.AppendLine(); + EmitBinaryCodecMembers(source, target.BinaryModel, target.Fields); + } + source.AppendLine("}"); source.AppendLine(); @@ -1026,6 +1536,2546 @@ private static void EmitModel(SourceProductionContext context, CollectionModelTa sourceText: SourceText.From(source.ToString(), Encoding.UTF8)); } + private static void EmitGeneratedCollectionFieldType( + StringBuilder source, + CollectionModelTarget target, + CollectionFieldSpec field, + BinaryFieldReaderSpec binaryFieldReader, + string codecTypeName, + string fieldTypeName) + { + source.Append(" private sealed class ") + .Append(fieldTypeName) + .Append(" : global::CSharpDB.Engine.CollectionField<") + .Append(target.FullyQualifiedTypeName) + .Append(", ") + .Append(field.MemberTypeName) + .AppendLine(">"); + source.AppendLine(" {"); + source.Append(" public ") + .Append(fieldTypeName) + .AppendLine("()"); + source.Append(" : base(") + .Append(SymbolDisplay.FormatLiteral(field.FieldPath, quote: true)) + .Append(", ") + .Append(field.AccessorExpression) + .Append(", global::CSharpDB.Engine.CollectionIndexDataKind.") + .Append(field.DataKindName) + .Append(", ") + .Append(SymbolDisplay.FormatLiteral(field.PayloadFieldPath, quote: true)) + .AppendLine(")"); + source.AppendLine(" {"); + source.AppendLine(" }"); + source.AppendLine(); + + source.AppendLine(" public override bool TryReadPayloadValue(global::System.ReadOnlySpan payload, out global::CSharpDB.Primitives.DbValue value)"); + source.AppendLine(" {"); + source.Append(" if (") + .Append(codecTypeName) + .Append('.') + .Append(CreateBinaryPayloadValueReaderName(field)) + .AppendLine("(payload, out value))"); + source.AppendLine(" return true;"); + source.AppendLine(); + source.AppendLine(" return base.TryReadPayloadValue(payload, out value);"); + source.AppendLine(" }"); + + if (binaryFieldReader.DataKindName == "Integer") + { + source.AppendLine(); + source.AppendLine(" public override bool TryReadPayloadInt64(global::System.ReadOnlySpan payload, out long value)"); + source.AppendLine(" {"); + source.Append(" if (") + .Append(codecTypeName) + .Append('.') + .Append(CreateBinaryPayloadIntegerReaderName(field)) + .AppendLine("(payload, out value))"); + source.AppendLine(" return true;"); + source.AppendLine(); + source.AppendLine(" return base.TryReadPayloadInt64(payload, out value);"); + source.AppendLine(" }"); + } + else if (binaryFieldReader.DataKindName == "Text") + { + source.AppendLine(); + source.AppendLine(" public override bool TryReadPayloadString(global::System.ReadOnlySpan payload, out string? value)"); + source.AppendLine(" {"); + source.Append(" if (") + .Append(codecTypeName) + .Append('.') + .Append(CreateBinaryPayloadTextReaderName(field)) + .AppendLine("(payload, out value))"); + source.AppendLine(" return true;"); + source.AppendLine(); + source.AppendLine(" return base.TryReadPayloadString(payload, out value);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" public override bool TryReadPayloadStringUtf8(global::System.ReadOnlySpan payload, out global::System.ReadOnlySpan value)"); + source.AppendLine(" {"); + source.Append(" if (") + .Append(codecTypeName) + .Append('.') + .Append(CreateBinaryPayloadTextUtf8ReaderName(field)) + .AppendLine("(payload, out value))"); + source.AppendLine(" return true;"); + source.AppendLine(); + source.AppendLine(" return base.TryReadPayloadStringUtf8(payload, out value);"); + source.AppendLine(" }"); + } + + source.AppendLine(" }"); + source.AppendLine(); + } + + private static void EmitBinaryCodecMembers( + StringBuilder source, + BinaryTypeSpec root, + ImmutableArray fields) + { + bool useRecordFormat = CanUseBinaryRecordFormat(root); + + source.Append(" private static int GetBinaryDocumentSize(") + .Append(root.TypeName) + .AppendLine(" document)"); + source.Append(" => "); + if (useRecordFormat) + source.Append("3 + GetBinaryRecordSize_"); + else + source.Append("GetBinarySize_"); + + source.Append(root.SafeIdentifier).AppendLine("(document);"); + source.AppendLine(); + + source.Append(" private static void WriteBinaryDocument(global::System.Span destination, ") + .Append(root.TypeName) + .AppendLine(" document)"); + source.AppendLine(" {"); + source.AppendLine(" int position = 0;"); + if (useRecordFormat) + { + source.AppendLine(" WriteByte(destination, ref position, RecordFormatMarker);"); + source.AppendLine(" WriteByte(destination, ref position, RecordFormatMagic);"); + source.AppendLine(" WriteByte(destination, ref position, RecordFormatVersion);"); + source.Append(" WriteBinaryRecord_").Append(root.SafeIdentifier).AppendLine("(destination, ref position, document);"); + } + else + { + source.Append(" WriteBinary_").Append(root.SafeIdentifier).AppendLine("(destination, ref position, document);"); + } + + source.AppendLine(" if (position != destination.Length)"); + source.AppendLine(" throw new global::System.InvalidOperationException(\"Generated binary collection payload size mismatch.\");"); + source.AppendLine(" }"); + source.AppendLine(); + + source.Append(" private static ") + .Append(root.TypeName) + .AppendLine(" DecodeBinaryDocument(global::System.ReadOnlySpan payload)"); + source.AppendLine(" {"); + source.AppendLine(" int position = 0;"); + if (useRecordFormat) + { + source.AppendLine(" if (IsBinaryRecordPayload(payload))"); + source.AppendLine(" {"); + source.AppendLine(" position = 3;"); + source.Append(" ") + .Append(root.TypeName) + .Append(" recordDocument = ReadBinaryRecord_") + .Append(root.SafeIdentifier) + .AppendLine("(payload, ref position);"); + source.AppendLine(" if (position != payload.Length)"); + source.AppendLine(" throw new global::CSharpDB.Primitives.CSharpDbException(global::CSharpDB.Primitives.ErrorCode.CorruptDatabase, \"Invalid generated binary collection payload length.\");"); + source.AppendLine(); + source.AppendLine(" return recordDocument;"); + source.AppendLine(" }"); + source.AppendLine(); + } + + source.Append(" ") + .Append(root.TypeName) + .Append(" document = ReadBinary_") + .Append(root.SafeIdentifier) + .AppendLine("(payload, ref position);"); + source.AppendLine(" if (position != payload.Length)"); + source.AppendLine(" throw new global::CSharpDB.Primitives.CSharpDbException(global::CSharpDB.Primitives.ErrorCode.CorruptDatabase, \"Invalid generated binary collection payload length.\");"); + source.AppendLine(); + source.AppendLine(" return document;"); + source.AppendLine(" }"); + source.AppendLine(); + + var emittedSizeTypes = new HashSet(StringComparer.Ordinal); + EmitBinaryTypeSizer(source, root, emittedSizeTypes); + var emittedWriterTypes = new HashSet(StringComparer.Ordinal); + EmitBinaryTypeWriter(source, root, emittedWriterTypes); + var emittedReaderTypes = new HashSet(StringComparer.Ordinal); + EmitBinaryTypeReader(source, root, emittedReaderTypes); + if (useRecordFormat) + { + var emittedRecordSizeTypes = new HashSet(StringComparer.Ordinal); + EmitBinaryRecordTypeSizer(source, root, emittedRecordSizeTypes); + var emittedRecordWriterTypes = new HashSet(StringComparer.Ordinal); + EmitBinaryRecordTypeWriter(source, root, emittedRecordWriterTypes); + var emittedRecordReaderTypes = new HashSet(StringComparer.Ordinal); + EmitBinaryRecordTypeReader(source, root, emittedRecordReaderTypes); + var emittedRecordSkipperTypes = new HashSet(StringComparer.Ordinal); + EmitBinaryRecordTypeSkipper(source, root, emittedRecordSkipperTypes); + } + + EmitBinaryPayloadFieldReaders(source, root, fields); + EmitBinaryPrimitiveHelpers(source); + } + + private static void EmitBinaryTypeSizer( + StringBuilder source, + BinaryTypeSpec type, + HashSet emittedTypes) + { + if (!emittedTypes.Add(type.SafeIdentifier)) + return; + + source.Append(" private static int GetBinarySize_") + .Append(type.SafeIdentifier) + .Append('(') + .Append(type.TypeName) + .AppendLine(" value)"); + source.AppendLine(" {"); + source.Append(" int size = GetVarintSize((ulong)") + .Append(type.Members.Length.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(");"); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + string valueVariable = "value" + i.ToString(System.Globalization.CultureInfo.InvariantCulture); + source.Append(" size += GetLengthPrefixedUtf8Size(") + .Append(SymbolDisplay.FormatLiteral(member.JsonName, quote: true)) + .AppendLine("u8);"); + source.Append(" var ") + .Append(valueVariable) + .Append(" = value.") + .Append(member.EscapedClrName) + .AppendLine(";"); + EmitGetBinaryValueSize(source, member.Value, valueVariable, " ", i.ToString(System.Globalization.CultureInfo.InvariantCulture)); + } + + source.AppendLine(" return size;"); + source.AppendLine(" }"); + source.AppendLine(); + + for (int i = 0; i < type.Members.Length; i++) + EmitNestedBinaryTypeSizers(source, type.Members[i].Value, emittedTypes); + } + + private static void EmitNestedBinaryTypeSizers( + StringBuilder source, + BinaryValueSpec value, + HashSet emittedTypes) + { + if (value.ObjectType is not null) + EmitBinaryTypeSizer(source, value.ObjectType, emittedTypes); + + if (value.Element is not null) + EmitNestedBinaryTypeSizers(source, value.Element, emittedTypes); + } + + private static void EmitBinaryTypeWriter( + StringBuilder source, + BinaryTypeSpec type, + HashSet emittedTypes) + { + if (!emittedTypes.Add(type.SafeIdentifier)) + return; + + source.Append(" private static void WriteBinary_") + .Append(type.SafeIdentifier) + .Append("(global::System.Span destination, ref int position, ") + .Append(type.TypeName) + .AppendLine(" value)"); + source.AppendLine(" {"); + source.Append(" WriteVarint(destination, ref position, (ulong)") + .Append(type.Members.Length.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(");"); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + string valueVariable = "value" + i.ToString(System.Globalization.CultureInfo.InvariantCulture); + source.Append(" WriteLengthPrefixedUtf8(destination, ref position, ") + .Append(SymbolDisplay.FormatLiteral(member.JsonName, quote: true)) + .AppendLine("u8);"); + source.Append(" var ") + .Append(valueVariable) + .Append(" = value.") + .Append(member.EscapedClrName) + .AppendLine(";"); + EmitWriteBinaryValue(source, member.Value, valueVariable, " ", i.ToString(System.Globalization.CultureInfo.InvariantCulture)); + } + + source.AppendLine(" }"); + source.AppendLine(); + + for (int i = 0; i < type.Members.Length; i++) + EmitNestedBinaryTypeWriters(source, type.Members[i].Value, emittedTypes); + } + + private static void EmitNestedBinaryTypeWriters( + StringBuilder source, + BinaryValueSpec value, + HashSet emittedTypes) + { + if (value.ObjectType is not null) + EmitBinaryTypeWriter(source, value.ObjectType, emittedTypes); + + if (value.Element is not null) + EmitNestedBinaryTypeWriters(source, value.Element, emittedTypes); + } + + private static void EmitBinaryTypeReader( + StringBuilder source, + BinaryTypeSpec type, + HashSet emittedTypes) + { + if (!emittedTypes.Add(type.SafeIdentifier)) + return; + + source.Append(" private static ") + .Append(type.TypeName) + .Append(" ReadBinaryObject_") + .Append(type.SafeIdentifier) + .AppendLine("(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag == NullTag)"); + source.AppendLine(" return default!;"); + source.AppendLine(); + source.AppendLine(" EnsureTag(tag, ObjectTag);"); + source.Append(" return ReadBinary_") + .Append(type.SafeIdentifier) + .AppendLine("(payload, ref position);"); + source.AppendLine(" }"); + source.AppendLine(); + + source.Append(" private static ") + .Append(type.TypeName) + .Append(" ReadBinary_") + .Append(type.SafeIdentifier) + .AppendLine("(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" ulong fieldCount = ReadVarint(payload, ref position);"); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + source.Append(" ") + .Append(member.Value.TypeName) + .Append(" value") + .Append(i.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(" = default!;"); + } + + if (type.Members.Length > 0) + source.AppendLine(); + + source.AppendLine(" for (ulong i = 0; i < fieldCount; i++)"); + source.AppendLine(" {"); + source.AppendLine(" global::System.ReadOnlySpan fieldName = ReadLengthPrefixedBytes(payload, ref position);"); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + source.Append(" ") + .Append(i == 0 ? "if" : "else if") + .Append(" (fieldName.SequenceEqual(") + .Append(SymbolDisplay.FormatLiteral(member.JsonName, quote: true)) + .AppendLine("u8))"); + source.AppendLine(" {"); + source.Append(" value") + .Append(i.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .Append(" = "); + AppendReadBinaryValueExpression( + source, + member.Value, + "payload", + "position", + CreateBinaryReaderMethodName(type, i)); + source.AppendLine(";"); + source.AppendLine(" continue;"); + source.AppendLine(" }"); + } + + source.AppendLine(); + source.AppendLine(" SkipBinaryValue(payload, ref position);"); + source.AppendLine(" }"); + source.AppendLine(); + source.Append(" return new ") + .Append(type.TypeName) + .Append('('); + for (int i = 0; i < type.Constructor.ParameterMemberIndexes.Length; i++) + { + if (i > 0) + source.Append(", "); + + source.Append("value") + .Append(type.Constructor.ParameterMemberIndexes[i].ToString(System.Globalization.CultureInfo.InvariantCulture)); + } + + source.AppendLine(");"); + source.AppendLine(" }"); + source.AppendLine(); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + if (member.Value.IsArray) + EmitBinaryArrayReader(source, member.Value, CreateBinaryReaderMethodName(type, i)); + } + + for (int i = 0; i < type.Members.Length; i++) + EmitNestedBinaryTypeReaders(source, type.Members[i].Value, emittedTypes); + } + + private static void EmitNestedBinaryTypeReaders( + StringBuilder source, + BinaryValueSpec value, + HashSet emittedTypes) + { + if (value.ObjectType is not null) + EmitBinaryTypeReader(source, value.ObjectType, emittedTypes); + + if (value.Element is not null) + EmitNestedBinaryTypeReaders(source, value.Element, emittedTypes); + } + + private static void EmitBinaryRecordTypeSizer( + StringBuilder source, + BinaryTypeSpec type, + HashSet emittedTypes) + { + if (!emittedTypes.Add(type.SafeIdentifier)) + return; + + source.Append(" private static int GetBinaryRecordSize_") + .Append(type.SafeIdentifier) + .Append('(') + .Append(type.TypeName) + .AppendLine(" value)"); + source.AppendLine(" {"); + source.Append(" int size = ") + .Append(GetNullBitmapSize(type).ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(";"); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + string valueVariable = "value" + i.ToString(System.Globalization.CultureInfo.InvariantCulture); + source.Append(" var ") + .Append(valueVariable) + .Append(" = value.") + .Append(member.EscapedClrName) + .AppendLine(";"); + EmitGetBinaryRecordValueSize(source, member.Value, valueVariable, " "); + } + + source.AppendLine(" return size;"); + source.AppendLine(" }"); + source.AppendLine(); + + for (int i = 0; i < type.Members.Length; i++) + EmitNestedBinaryRecordTypeSizers(source, type.Members[i].Value, emittedTypes); + } + + private static void EmitNestedBinaryRecordTypeSizers( + StringBuilder source, + BinaryValueSpec value, + HashSet emittedTypes) + { + if (value.ObjectType is not null) + EmitBinaryRecordTypeSizer(source, value.ObjectType, emittedTypes); + } + + private static void EmitBinaryRecordTypeWriter( + StringBuilder source, + BinaryTypeSpec type, + HashSet emittedTypes) + { + if (!emittedTypes.Add(type.SafeIdentifier)) + return; + + int nullBitmapSize = GetNullBitmapSize(type); + source.Append(" private static void WriteBinaryRecord_") + .Append(type.SafeIdentifier) + .Append("(global::System.Span destination, ref int position, ") + .Append(type.TypeName) + .AppendLine(" value)"); + source.AppendLine(" {"); + source.AppendLine(" int nullBitmapStart = position;"); + if (nullBitmapSize > 0) + { + source.Append(" destination.Slice(position, ") + .Append(nullBitmapSize.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(").Clear();"); + source.Append(" position += ") + .Append(nullBitmapSize.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(";"); + } + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + string valueVariable = "value" + i.ToString(System.Globalization.CultureInfo.InvariantCulture); + source.Append(" var ") + .Append(valueVariable) + .Append(" = value.") + .Append(member.EscapedClrName) + .AppendLine(";"); + EmitWriteBinaryRecordValue(source, member.Value, valueVariable, i, " "); + } + + source.AppendLine(" }"); + source.AppendLine(); + + for (int i = 0; i < type.Members.Length; i++) + EmitNestedBinaryRecordTypeWriters(source, type.Members[i].Value, emittedTypes); + } + + private static void EmitNestedBinaryRecordTypeWriters( + StringBuilder source, + BinaryValueSpec value, + HashSet emittedTypes) + { + if (value.ObjectType is not null) + EmitBinaryRecordTypeWriter(source, value.ObjectType, emittedTypes); + } + + private static void EmitBinaryRecordTypeReader( + StringBuilder source, + BinaryTypeSpec type, + HashSet emittedTypes) + { + if (!emittedTypes.Add(type.SafeIdentifier)) + return; + + source.Append(" private static ") + .Append(type.TypeName) + .Append(" ReadBinaryRecord_") + .Append(type.SafeIdentifier) + .AppendLine("(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.Append(" int nullBitmapStart = ReadBinaryRecordNullBitmap(payload, ref position, ") + .Append(GetNullBitmapSize(type).ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(");"); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + source.Append(" ") + .Append(member.Value.TypeName) + .Append(" value") + .Append(i.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(" = default!;"); + } + + if (type.Members.Length > 0) + source.AppendLine(); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + source.Append(" if (!IsBinaryRecordNull(payload, nullBitmapStart, ") + .Append(i.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine("))"); + source.AppendLine(" {"); + source.Append(" value") + .Append(i.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .Append(" = "); + AppendReadBinaryRecordValueExpression(source, member.Value, "payload", "position"); + source.AppendLine(";"); + source.AppendLine(" }"); + } + + source.AppendLine(); + source.Append(" return new ") + .Append(type.TypeName) + .Append('('); + for (int i = 0; i < type.Constructor.ParameterMemberIndexes.Length; i++) + { + if (i > 0) + source.Append(", "); + + source.Append("value") + .Append(type.Constructor.ParameterMemberIndexes[i].ToString(System.Globalization.CultureInfo.InvariantCulture)); + } + + source.AppendLine(");"); + source.AppendLine(" }"); + source.AppendLine(); + + for (int i = 0; i < type.Members.Length; i++) + EmitNestedBinaryRecordTypeReaders(source, type.Members[i].Value, emittedTypes); + } + + private static void EmitNestedBinaryRecordTypeReaders( + StringBuilder source, + BinaryValueSpec value, + HashSet emittedTypes) + { + if (value.ObjectType is not null) + EmitBinaryRecordTypeReader(source, value.ObjectType, emittedTypes); + } + + private static void EmitBinaryRecordTypeSkipper( + StringBuilder source, + BinaryTypeSpec type, + HashSet emittedTypes) + { + if (!emittedTypes.Add(type.SafeIdentifier)) + return; + + source.Append(" private static void SkipBinaryRecord_") + .Append(type.SafeIdentifier) + .AppendLine("(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.Append(" int nullBitmapStart = ReadBinaryRecordNullBitmap(payload, ref position, ") + .Append(GetNullBitmapSize(type).ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(");"); + + for (int i = 0; i < type.Members.Length; i++) + { + BinaryMemberSpec member = type.Members[i]; + source.Append(" if (!IsBinaryRecordNull(payload, nullBitmapStart, ") + .Append(i.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine("))"); + source.AppendLine(" {"); + EmitSkipBinaryRecordValue(source, member.Value, "payload", "position", " "); + source.AppendLine(" }"); + } + + source.AppendLine(" }"); + source.AppendLine(); + + for (int i = 0; i < type.Members.Length; i++) + EmitNestedBinaryRecordTypeSkippers(source, type.Members[i].Value, emittedTypes); + } + + private static void EmitNestedBinaryRecordTypeSkippers( + StringBuilder source, + BinaryValueSpec value, + HashSet emittedTypes) + { + if (value.ObjectType is not null) + EmitBinaryRecordTypeSkipper(source, value.ObjectType, emittedTypes); + } + + private static int GetNullBitmapSize(BinaryTypeSpec type) + => (type.Members.Length + 7) / 8; + + private static void EmitGetBinaryRecordValueSize( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent) + { + if (value.IsArray) + throw new InvalidOperationException("Generated binary record format does not support array payloads."); + + if (value.CanBeNull) + { + source.Append(indent) + .Append("if (") + .Append(GetNullCheckExpression(value, expression)) + .AppendLine(")"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + EmitGetBinaryRecordNonNullValueSize(source, value, GetNonNullExpression(value, expression), indent + " "); + source.Append(indent).AppendLine("}"); + return; + } + + EmitGetBinaryRecordNonNullValueSize(source, value, expression, indent); + } + + private static void EmitGetBinaryRecordNonNullValueSize( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent) + { + if (value.ValueKindName == "Object") + { + BinaryTypeSpec objectType = value.ObjectType ?? throw new InvalidOperationException("Object binary value is missing its type spec."); + source.Append(indent) + .Append("size += GetBinaryRecordSize_") + .Append(objectType.SafeIdentifier) + .Append('(') + .Append(expression) + .AppendLine(");"); + return; + } + + switch (value.ValueKindName) + { + case "String": + source.Append(indent) + .Append("size += GetLengthPrefixedStringSize(") + .Append(expression) + .AppendLine(");"); + return; + case "Guid": + source.Append(indent).AppendLine("size += 16;"); + return; + case "DateOnly": + case "Int32": + case "UInt32": + case "Single": + source.Append(indent).AppendLine("size += sizeof(int);"); + return; + case "TimeOnly": + case "Enum": + case "Int64": + case "Double": + source.Append(indent).AppendLine("size += sizeof(long);"); + return; + case "Boolean": + case "Byte": + case "SByte": + source.Append(indent).AppendLine("size += 1;"); + return; + case "Int16": + case "UInt16": + source.Append(indent).AppendLine("size += sizeof(short);"); + return; + case "Decimal": + source.Append(indent).AppendLine("size += sizeof(int) * 4;"); + return; + default: + throw new InvalidOperationException($"Unsupported generated binary record value kind '{value.ValueKindName}'."); + } + } + + private static void EmitWriteBinaryRecordValue( + StringBuilder source, + BinaryValueSpec value, + string expression, + int memberIndex, + string indent) + { + if (value.IsArray) + throw new InvalidOperationException("Generated binary record format does not support array payloads."); + + if (value.CanBeNull) + { + source.Append(indent) + .Append("if (") + .Append(GetNullCheckExpression(value, expression)) + .AppendLine(")"); + source.Append(indent).AppendLine("{"); + source.Append(indent) + .Append(" SetBinaryRecordNull(destination, nullBitmapStart, ") + .Append(memberIndex.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(");"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + EmitWriteBinaryRecordNonNullValue(source, value, GetNonNullExpression(value, expression), indent + " "); + source.Append(indent).AppendLine("}"); + return; + } + + EmitWriteBinaryRecordNonNullValue(source, value, expression, indent); + } + + private static void EmitWriteBinaryRecordNonNullValue( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent) + { + if (value.ValueKindName == "Object") + { + BinaryTypeSpec objectType = value.ObjectType ?? throw new InvalidOperationException("Object binary value is missing its type spec."); + source.Append(indent) + .Append("WriteBinaryRecord_") + .Append(objectType.SafeIdentifier) + .Append("(destination, ref position, ") + .Append(expression) + .AppendLine(");"); + return; + } + + string methodName = value.ValueKindName switch + { + "String" => "WriteLengthPrefixedString", + "Guid" => "WriteBinaryRecordGuid", + "DateOnly" => "WriteBinaryRecordDateOnly", + "TimeOnly" => "WriteBinaryRecordTimeOnly", + "Enum" => "WriteBinaryRecordEnum", + "Boolean" => "WriteBinaryRecordBoolean", + "Byte" => "WriteBinaryRecordByte", + "SByte" => "WriteBinaryRecordSByte", + "Int16" => "WriteBinaryRecordInt16", + "UInt16" => "WriteBinaryRecordUInt16", + "Int32" => "WriteBinaryRecordInt32", + "UInt32" => "WriteBinaryRecordUInt32", + "Int64" => "WriteInt64", + "Single" => "WriteBinaryRecordSingle", + "Double" => "WriteBinaryRecordDouble", + "Decimal" => "WriteBinaryRecordDecimal", + _ => throw new InvalidOperationException($"Unsupported generated binary record value kind '{value.ValueKindName}'."), + }; + + source.Append(indent) + .Append(methodName) + .Append("(destination, ref position, ") + .Append(expression) + .AppendLine(");"); + } + + private static void AppendReadBinaryRecordValueExpression( + StringBuilder source, + BinaryValueSpec value, + string payloadExpression, + string positionVariable) + { + if (value.IsArray) + throw new InvalidOperationException("Generated binary record format does not support array payloads."); + + if (value.ValueKindName == "Object") + { + BinaryTypeSpec objectType = value.ObjectType ?? throw new InvalidOperationException("Object binary value is missing its type spec."); + source.Append("ReadBinaryRecord_") + .Append(objectType.SafeIdentifier) + .Append('(') + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(')'); + return; + } + + string methodName = value.ValueKindName switch + { + "String" => "ReadBinaryRecordString", + "Guid" => "ReadBinaryRecordGuid", + "DateOnly" => "ReadBinaryRecordDateOnly", + "TimeOnly" => "ReadBinaryRecordTimeOnly", + "Enum" => "ReadBinaryRecordEnum<" + value.EffectiveTypeName + ">", + "Boolean" => "ReadBinaryRecordBoolean", + "Byte" => "ReadBinaryRecordByte", + "SByte" => "ReadBinaryRecordSByte", + "Int16" => "ReadBinaryRecordInt16", + "UInt16" => "ReadBinaryRecordUInt16", + "Int32" => "ReadBinaryRecordInt32", + "UInt32" => "ReadBinaryRecordUInt32", + "Int64" => "ReadInt64", + "Single" => "ReadBinaryRecordSingle", + "Double" => "ReadBinaryRecordDouble", + "Decimal" => "ReadBinaryRecordDecimal", + _ => throw new InvalidOperationException($"Unsupported generated binary record value kind '{value.ValueKindName}'."), + }; + + source.Append(methodName) + .Append('(') + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(')'); + if (value.ValueKindName == "String") + source.Append('!'); + } + + private static void EmitSkipBinaryRecordValue( + StringBuilder source, + BinaryValueSpec value, + string payloadExpression, + string positionVariable, + string indent) + { + if (value.IsArray) + throw new InvalidOperationException("Generated binary record format does not support array payloads."); + + if (value.ValueKindName == "Object") + { + BinaryTypeSpec objectType = value.ObjectType ?? throw new InvalidOperationException("Object binary value is missing its type spec."); + source.Append(indent) + .Append("SkipBinaryRecord_") + .Append(objectType.SafeIdentifier) + .Append('(') + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .AppendLine(");"); + return; + } + + switch (value.ValueKindName) + { + case "String": + source.Append(indent) + .Append("_ = ReadLengthPrefixedBytes(") + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .AppendLine(");"); + return; + case "Guid": + source.Append(indent) + .Append("EnsureAvailable(") + .Append(payloadExpression) + .Append(", ") + .Append(positionVariable) + .AppendLine(", 16);"); + source.Append(indent) + .Append(positionVariable) + .AppendLine(" += 16;"); + return; + case "DateOnly": + case "Int32": + case "UInt32": + case "Single": + source.Append(indent) + .Append("EnsureAvailable(") + .Append(payloadExpression) + .Append(", ") + .Append(positionVariable) + .AppendLine(", sizeof(int));"); + source.Append(indent) + .Append(positionVariable) + .AppendLine(" += sizeof(int);"); + return; + case "TimeOnly": + case "Enum": + case "Int64": + case "Double": + source.Append(indent) + .Append("EnsureAvailable(") + .Append(payloadExpression) + .Append(", ") + .Append(positionVariable) + .AppendLine(", sizeof(long));"); + source.Append(indent) + .Append(positionVariable) + .AppendLine(" += sizeof(long);"); + return; + case "Boolean": + case "Byte": + case "SByte": + source.Append(indent) + .Append("EnsureAvailable(") + .Append(payloadExpression) + .Append(", ") + .Append(positionVariable) + .AppendLine(", 1);"); + source.Append(indent) + .Append(positionVariable) + .AppendLine("++;"); + return; + case "Int16": + case "UInt16": + source.Append(indent) + .Append("EnsureAvailable(") + .Append(payloadExpression) + .Append(", ") + .Append(positionVariable) + .AppendLine(", sizeof(short));"); + source.Append(indent) + .Append(positionVariable) + .AppendLine(" += sizeof(short);"); + return; + case "Decimal": + source.Append(indent) + .Append("EnsureAvailable(") + .Append(payloadExpression) + .Append(", ") + .Append(positionVariable) + .AppendLine(", sizeof(int) * 4);"); + source.Append(indent) + .Append(positionVariable) + .AppendLine(" += sizeof(int) * 4;"); + return; + default: + throw new InvalidOperationException($"Unsupported generated binary record value kind '{value.ValueKindName}'."); + } + } + + private static string GetNullCheckExpression(BinaryValueSpec value, string expression) + => value.IsNullableValueType + ? "!" + expression + ".HasValue" + : expression + " is null"; + + private static string GetNonNullExpression(BinaryValueSpec value, string expression) + => value.IsNullableValueType ? expression + ".Value" : expression; + + private static void EmitBinaryPayloadFieldReaders( + StringBuilder source, + BinaryTypeSpec root, + ImmutableArray fields) + { + for (int i = 0; i < fields.Length; i++) + { + CollectionFieldSpec field = fields[i]; + if (!TryCreateBinaryFieldReaderSpec(root, field, out BinaryFieldReaderSpec reader)) + continue; + + EmitBinaryPayloadFieldReader(source, root, field, reader); + } + } + + private static void EmitBinaryPayloadFieldReader( + StringBuilder source, + BinaryTypeSpec root, + CollectionFieldSpec field, + BinaryFieldReaderSpec reader) + { + source.Append(" internal static bool ") + .Append(CreateBinaryPayloadValueReaderName(field)) + .AppendLine("(global::System.ReadOnlySpan payload, out global::CSharpDB.Primitives.DbValue value)"); + source.AppendLine(" {"); + source.AppendLine(" value = default;"); + EmitBinaryPayloadFieldPathReader( + source, + root, + reader, + "Value", + " return TryReadBinaryPayloadFieldValue(currentPayload, ref position, out value);"); + source.AppendLine(" }"); + source.AppendLine(); + + if (reader.DataKindName == "Integer") + { + source.Append(" internal static bool ") + .Append(CreateBinaryPayloadIntegerReaderName(field)) + .AppendLine("(global::System.ReadOnlySpan payload, out long value)"); + source.AppendLine(" {"); + source.AppendLine(" value = 0;"); + EmitBinaryPayloadFieldPathReader( + source, + root, + reader, + "Integer", + " return TryReadBinaryPayloadFieldInt64(currentPayload, ref position, out value);"); + source.AppendLine(" }"); + source.AppendLine(); + } + else if (reader.DataKindName == "Text") + { + source.Append(" internal static bool ") + .Append(CreateBinaryPayloadTextReaderName(field)) + .AppendLine("(global::System.ReadOnlySpan payload, out string? value)"); + source.AppendLine(" {"); + source.AppendLine(" value = null;"); + EmitBinaryPayloadFieldPathReader( + source, + root, + reader, + "Text", + " return TryReadBinaryPayloadFieldText(currentPayload, ref position, out value);"); + source.AppendLine(" }"); + source.AppendLine(); + + source.Append(" internal static bool ") + .Append(CreateBinaryPayloadTextUtf8ReaderName(field)) + .AppendLine("(global::System.ReadOnlySpan payload, out global::System.ReadOnlySpan value)"); + source.AppendLine(" {"); + source.AppendLine(" value = default;"); + EmitBinaryPayloadFieldPathReader( + source, + root, + reader, + "TextUtf8", + " return TryReadBinaryPayloadFieldTextUtf8(currentPayload, ref position, out value);"); + source.AppendLine(" }"); + source.AppendLine(); + } + } + + private static void EmitBinaryPayloadFieldPathReader( + StringBuilder source, + BinaryTypeSpec root, + BinaryFieldReaderSpec reader, + string recordReadKindName, + string finalReadStatement) + { + source.AppendLine(" if (!global::CSharpDB.Storage.Serialization.CollectionPayloadCodec.TryGetBinaryDocumentPayload(payload, out global::System.ReadOnlySpan currentPayload))"); + source.AppendLine(" return false;"); + source.AppendLine(); + source.AppendLine(" int position = 0;"); + + if (CanUseBinaryRecordFormat(root)) + { + source.AppendLine(" if (IsBinaryRecordPayload(currentPayload))"); + source.AppendLine(" {"); + source.AppendLine(" position = 3;"); + EmitBinaryRecordPayloadFieldPathReader(source, root, reader, recordReadKindName); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" position = 0;"); + } + + BinaryTypeSpec currentType = root; + for (int depth = 0; depth < reader.MemberIndexes.Length; depth++) + { + int memberIndex = reader.MemberIndexes[depth]; + BinaryMemberSpec member = currentType.Members[memberIndex]; + + source.Append(" if (!TryReadExpectedObjectFieldCount(currentPayload, ref position, ") + .Append(currentType.Members.Length.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine("))"); + source.AppendLine(" return false;"); + + for (int i = 0; i < memberIndex; i++) + { + source.AppendLine(" SkipBinaryField(currentPayload, ref position);"); + } + + source.Append(" if (!TryReadExpectedFieldName(currentPayload, ref position, ") + .Append(SymbolDisplay.FormatLiteral(member.JsonName, quote: true)) + .AppendLine("u8))"); + source.AppendLine(" return false;"); + + if (depth == reader.MemberIndexes.Length - 1) + { + source.AppendLine(finalReadStatement); + } + else + { + source.Append(" if (!TryReadBinaryObjectPayload(currentPayload, ref position, out int nestedStart") + .Append(depth.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .Append(", out int nestedLength") + .Append(depth.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine("))"); + source.AppendLine(" return false;"); + source.Append(" currentPayload = currentPayload.Slice(nestedStart") + .Append(depth.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .Append(", nestedLength") + .Append(depth.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(");"); + source.AppendLine(" position = 0;"); + source.AppendLine(); + currentType = member.Value.ObjectType ?? throw new InvalidOperationException("Binary field reader path entered a non-object member."); + } + } + } + + private static void EmitBinaryRecordPayloadFieldPathReader( + StringBuilder source, + BinaryTypeSpec root, + BinaryFieldReaderSpec reader, + string recordReadKindName) + { + BinaryTypeSpec currentType = root; + for (int depth = 0; depth < reader.MemberIndexes.Length; depth++) + { + int memberIndex = reader.MemberIndexes[depth]; + BinaryMemberSpec member = currentType.Members[memberIndex]; + string nullBitmapStart = "nullBitmapStart" + depth.ToString(System.Globalization.CultureInfo.InvariantCulture); + + source.Append(" int ") + .Append(nullBitmapStart) + .Append(" = ReadBinaryRecordNullBitmap(currentPayload, ref position, ") + .Append(GetNullBitmapSize(currentType).ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine(");"); + source.Append(" if (IsBinaryRecordNull(currentPayload, ") + .Append(nullBitmapStart) + .Append(", ") + .Append(memberIndex.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine("))"); + source.AppendLine(" return false;"); + + for (int i = 0; i < memberIndex; i++) + { + source.Append(" if (!IsBinaryRecordNull(currentPayload, ") + .Append(nullBitmapStart) + .Append(", ") + .Append(i.ToString(System.Globalization.CultureInfo.InvariantCulture)) + .AppendLine("))"); + source.AppendLine(" {"); + EmitSkipBinaryRecordValue(source, currentType.Members[i].Value, "currentPayload", "position", " "); + source.AppendLine(" }"); + } + + if (depth == reader.MemberIndexes.Length - 1) + { + EmitBinaryRecordPayloadFinalRead(source, reader, recordReadKindName, " "); + } + else + { + currentType = member.Value.ObjectType ?? throw new InvalidOperationException("Binary field reader path entered a non-object member."); + } + } + } + + private static void EmitBinaryRecordPayloadFinalRead( + StringBuilder source, + BinaryFieldReaderSpec reader, + string recordReadKindName, + string indent) + { + switch (recordReadKindName) + { + case "Value": + if (reader.DataKindName == "Integer") + { + source.Append(indent).Append("value = global::CSharpDB.Primitives.DbValue.FromInteger("); + AppendReadBinaryRecordIntegerAsInt64Expression(source, reader.Value, "currentPayload", "position"); + source.AppendLine(");"); + } + else if (reader.DataKindName == "Text") + { + source.Append(indent).Append("value = global::CSharpDB.Primitives.DbValue.FromText("); + AppendReadBinaryRecordTextExpression(source, reader.Value, "currentPayload", "position"); + source.AppendLine(");"); + } + else + { + throw new InvalidOperationException($"Unsupported generated binary record field data kind '{reader.DataKindName}'."); + } + + source.Append(indent).AppendLine("return true;"); + return; + + case "Integer": + source.Append(indent).Append("value = "); + AppendReadBinaryRecordIntegerAsInt64Expression(source, reader.Value, "currentPayload", "position"); + source.AppendLine(";"); + source.Append(indent).AppendLine("return true;"); + return; + + case "Text": + source.Append(indent).Append("value = "); + AppendReadBinaryRecordTextExpression(source, reader.Value, "currentPayload", "position"); + source.AppendLine(";"); + source.Append(indent).AppendLine("return true;"); + return; + + case "TextUtf8": + if (reader.Value.ValueKindName == "String") + { + source.Append(indent).AppendLine("value = ReadBinaryRecordStringUtf8(currentPayload, ref position);"); + source.Append(indent).AppendLine("return true;"); + } + else + { + source.Append(indent).AppendLine("return false;"); + } + + return; + + default: + throw new InvalidOperationException($"Unsupported generated binary record read kind '{recordReadKindName}'."); + } + } + + private static void AppendReadBinaryRecordIntegerAsInt64Expression( + StringBuilder source, + BinaryValueSpec value, + string payloadExpression, + string positionVariable) + { + switch (value.ValueKindName) + { + case "Byte": + case "SByte": + case "Int16": + case "UInt16": + case "Int32": + case "UInt32": + source.Append("(long)"); + AppendReadBinaryRecordValueExpression(source, value, payloadExpression, positionVariable); + return; + case "Int64": + case "Enum": + source.Append("ReadInt64(") + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(')'); + return; + default: + throw new InvalidOperationException($"Unsupported generated binary record integer field kind '{value.ValueKindName}'."); + } + } + + private static void AppendReadBinaryRecordTextExpression( + StringBuilder source, + BinaryValueSpec value, + string payloadExpression, + string positionVariable) + { + switch (value.ValueKindName) + { + case "String": + source.Append("ReadBinaryRecordString(") + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(')'); + return; + case "Guid": + source.Append("ReadBinaryRecordGuid(") + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(").ToString(\"D\")"); + return; + case "DateOnly": + source.Append("ReadBinaryRecordDateOnly(") + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(").ToString(\"O\", global::System.Globalization.CultureInfo.InvariantCulture)"); + return; + case "TimeOnly": + source.Append("ReadBinaryRecordTimeOnly(") + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(").ToString(\"O\", global::System.Globalization.CultureInfo.InvariantCulture)"); + return; + default: + throw new InvalidOperationException($"Unsupported generated binary record text field kind '{value.ValueKindName}'."); + } + } + + private static string CreateBinaryReaderMethodName(BinaryTypeSpec type, int memberIndex) + => "ReadBinaryArray_" + type.SafeIdentifier + "_" + memberIndex.ToString(System.Globalization.CultureInfo.InvariantCulture); + + private static void EmitBinaryArrayReader( + StringBuilder source, + BinaryValueSpec value, + string methodName) + { + BinaryValueSpec element = value.Element ?? throw new InvalidOperationException("Array binary value is missing its element spec."); + + source.Append(" private static ") + .Append(value.TypeName) + .Append(' ') + .Append(methodName) + .AppendLine("(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag == NullTag)"); + source.AppendLine(" return default!;"); + source.AppendLine(); + source.AppendLine(" EnsureTag(tag, ArrayTag);"); + source.AppendLine(" ulong count = ReadVarint(payload, ref position);"); + source.Append(" var values = new global::System.Collections.Generic.List<") + .Append(element.TypeName) + .AppendLine(">(checked((int)count));"); + source.AppendLine(" for (ulong i = 0; i < count; i++)"); + source.AppendLine(" {"); + source.Append(" values.Add("); + AppendReadBinaryValueExpression(source, element, "payload", "position", methodName + "_Element"); + source.AppendLine(");"); + source.AppendLine(" }"); + source.AppendLine(); + if (value.IsArrayType) + source.AppendLine(" return values.ToArray();"); + else + source.AppendLine(" return values;"); + source.AppendLine(" }"); + source.AppendLine(); + } + + private static void AppendReadBinaryValueExpression( + StringBuilder source, + BinaryValueSpec value, + string payloadExpression, + string positionVariable, + string arrayReaderMethodName) + { + if (value.IsArray) + { + source.Append(arrayReaderMethodName) + .Append('(') + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(')'); + return; + } + + if (value.ValueKindName == "Object") + { + BinaryTypeSpec objectType = value.ObjectType ?? throw new InvalidOperationException("Object binary value is missing its type spec."); + source.Append("ReadBinaryObject_") + .Append(objectType.SafeIdentifier) + .Append('(') + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(')'); + return; + } + + string methodName = value.ValueKindName switch + { + "String" => "ReadBinaryString", + "Guid" => value.IsNullableValueType ? "ReadNullableBinaryGuid" : "ReadBinaryGuid", + "DateOnly" => value.IsNullableValueType ? "ReadNullableBinaryDateOnly" : "ReadBinaryDateOnly", + "TimeOnly" => value.IsNullableValueType ? "ReadNullableBinaryTimeOnly" : "ReadBinaryTimeOnly", + "Enum" => value.IsNullableValueType ? "ReadNullableBinaryEnum<" + value.EffectiveTypeName + ">" : "ReadBinaryEnum<" + value.EffectiveTypeName + ">", + "Boolean" => value.IsNullableValueType ? "ReadNullableBinaryBoolean" : "ReadBinaryBoolean", + "Byte" => value.IsNullableValueType ? "ReadNullableBinaryByte" : "ReadBinaryByte", + "SByte" => value.IsNullableValueType ? "ReadNullableBinarySByte" : "ReadBinarySByte", + "Int16" => value.IsNullableValueType ? "ReadNullableBinaryInt16" : "ReadBinaryInt16", + "UInt16" => value.IsNullableValueType ? "ReadNullableBinaryUInt16" : "ReadBinaryUInt16", + "Int32" => value.IsNullableValueType ? "ReadNullableBinaryInt32" : "ReadBinaryInt32", + "UInt32" => value.IsNullableValueType ? "ReadNullableBinaryUInt32" : "ReadBinaryUInt32", + "Int64" => value.IsNullableValueType ? "ReadNullableBinaryInt64" : "ReadBinaryInt64", + "Single" => value.IsNullableValueType ? "ReadNullableBinarySingle" : "ReadBinarySingle", + "Double" => value.IsNullableValueType ? "ReadNullableBinaryDouble" : "ReadBinaryDouble", + "Decimal" => value.IsNullableValueType ? "ReadNullableBinaryDecimal" : "ReadBinaryDecimal", + _ => throw new InvalidOperationException($"Unsupported generated binary value kind '{value.ValueKindName}'."), + }; + + source.Append(methodName) + .Append('(') + .Append(payloadExpression) + .Append(", ref ") + .Append(positionVariable) + .Append(')'); + if (value.ValueKindName == "String") + source.Append('!'); + } + + private static void EmitGetBinaryValueSize( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent, + string suffix) + { + if (value.IsArray) + { + EmitGetBinaryArrayValueSize(source, value, expression, indent, suffix); + return; + } + + if (value.ValueKindName == "Object") + { + EmitGetBinaryObjectValueSize(source, value, expression, indent); + return; + } + + if (value.IsNullableValueType) + { + source.Append(indent) + .Append("if (!") + .Append(expression) + .AppendLine(".HasValue)"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" size += 1;"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + EmitGetBinaryScalarValueSize(source, value, expression + ".Value", indent + " "); + source.Append(indent).AppendLine("}"); + return; + } + + EmitGetBinaryScalarValueSize(source, value, expression, indent); + } + + private static void EmitGetBinaryArrayValueSize( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent, + string suffix) + { + BinaryValueSpec element = value.Element ?? throw new InvalidOperationException("Array binary value is missing its element spec."); + string countVariable = "count" + suffix; + string bufferVariable = "buffer" + suffix; + string itemVariable = "item" + suffix; + + source.Append(indent) + .Append("if (") + .Append(expression) + .AppendLine(" is null)"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" size += 1;"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" size += 1;"); + source.Append(indent) + .Append(" if (!global::System.Linq.Enumerable.TryGetNonEnumeratedCount<") + .Append(element.TypeName) + .Append(">(") + .Append(expression) + .Append(", out int ") + .Append(countVariable) + .AppendLine("))"); + source.Append(indent).AppendLine(" {"); + source.Append(indent) + .Append(" var ") + .Append(bufferVariable) + .Append(" = new global::System.Collections.Generic.List<") + .Append(element.TypeName) + .AppendLine(">();"); + source.Append(indent) + .Append(" foreach (var ") + .Append(itemVariable) + .Append(" in ") + .Append(expression) + .AppendLine(")"); + source.Append(indent) + .Append(" ") + .Append(bufferVariable) + .Append(".Add(") + .Append(itemVariable) + .AppendLine(");"); + source.Append(indent) + .Append(" size += GetVarintSize((ulong)") + .Append(bufferVariable) + .AppendLine(".Count);"); + source.Append(indent) + .Append(" foreach (var ") + .Append(itemVariable) + .Append(" in ") + .Append(bufferVariable) + .AppendLine(")"); + source.Append(indent).AppendLine(" {"); + EmitGetBinaryValueSize(source, element, itemVariable, indent + " ", suffix + "_item"); + source.Append(indent).AppendLine(" }"); + source.Append(indent).AppendLine(" }"); + source.Append(indent).AppendLine(" else"); + source.Append(indent).AppendLine(" {"); + source.Append(indent) + .Append(" size += GetVarintSize((ulong)") + .Append(countVariable) + .AppendLine(");"); + source.Append(indent) + .Append(" foreach (var ") + .Append(itemVariable) + .Append(" in ") + .Append(expression) + .AppendLine(")"); + source.Append(indent).AppendLine(" {"); + EmitGetBinaryValueSize(source, element, itemVariable, indent + " ", suffix + "_item"); + source.Append(indent).AppendLine(" }"); + source.Append(indent).AppendLine(" }"); + source.Append(indent).AppendLine("}"); + } + + private static void EmitGetBinaryObjectValueSize( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent) + { + BinaryTypeSpec objectType = value.ObjectType ?? throw new InvalidOperationException("Object binary value is missing its type spec."); + + if (value.IsNullableValueType) + { + source.Append(indent) + .Append("if (!") + .Append(expression) + .AppendLine(".HasValue)"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" size += 1;"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + source.Append(indent) + .Append(" size += 1 + GetBinarySize_") + .Append(objectType.SafeIdentifier) + .Append('(') + .Append(expression) + .AppendLine(".Value);"); + source.Append(indent).AppendLine("}"); + return; + } + + if (value.CanBeNull) + { + source.Append(indent) + .Append("if (") + .Append(expression) + .AppendLine(" is null)"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" size += 1;"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + source.Append(indent) + .Append(" size += 1 + GetBinarySize_") + .Append(objectType.SafeIdentifier) + .Append('(') + .Append(expression) + .AppendLine(");"); + source.Append(indent).AppendLine("}"); + return; + } + + source.Append(indent) + .Append("size += 1 + GetBinarySize_") + .Append(objectType.SafeIdentifier) + .Append('(') + .Append(expression) + .AppendLine(");"); + } + + private static void EmitGetBinaryScalarValueSize( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent) + { + switch (value.ValueKindName) + { + case "String": + source.Append(indent) + .Append("size += GetBinaryStringSize(") + .Append(expression) + .AppendLine(");"); + return; + case "Guid": + source.Append(indent) + .Append("size += GetBinaryStringSize(") + .Append(expression) + .AppendLine(".ToString(\"D\"));"); + return; + case "DateOnly": + case "TimeOnly": + source.Append(indent) + .Append("size += GetBinaryStringSize(") + .Append(expression) + .AppendLine(".ToString(\"O\", global::System.Globalization.CultureInfo.InvariantCulture));"); + return; + case "Boolean": + source.Append(indent).AppendLine("size += 1;"); + return; + case "Enum": + case "Byte": + case "SByte": + case "Int16": + case "UInt16": + case "Int32": + case "UInt32": + case "Int64": + case "Single": + case "Double": + source.Append(indent).AppendLine("size += 1 + sizeof(long);"); + return; + case "Decimal": + source.Append(indent).AppendLine("size += 1 + (sizeof(int) * 4);"); + return; + default: + throw new InvalidOperationException($"Unsupported generated binary value kind '{value.ValueKindName}'."); + } + } + + private static void EmitWriteBinaryValue( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent, + string suffix) + { + if (value.IsArray) + { + EmitWriteBinaryArrayValue(source, value, expression, indent, suffix); + return; + } + + if (value.ValueKindName == "Object") + { + EmitWriteBinaryObjectValue(source, value, expression, indent); + return; + } + + if (value.IsNullableValueType) + { + source.Append(indent) + .Append("if (!") + .Append(expression) + .AppendLine(".HasValue)"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" WriteByte(destination, ref position, NullTag);"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + EmitWriteBinaryScalarValue(source, value, expression + ".Value", indent + " "); + source.Append(indent).AppendLine("}"); + return; + } + + EmitWriteBinaryScalarValue(source, value, expression, indent); + } + + private static void EmitWriteBinaryArrayValue( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent, + string suffix) + { + BinaryValueSpec element = value.Element ?? throw new InvalidOperationException("Array binary value is missing its element spec."); + string countVariable = "count" + suffix; + string bufferVariable = "buffer" + suffix; + string itemVariable = "item" + suffix; + + source.Append(indent) + .Append("if (") + .Append(expression) + .AppendLine(" is null)"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" WriteByte(destination, ref position, NullTag);"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" WriteByte(destination, ref position, ArrayTag);"); + source.Append(indent) + .Append(" if (!global::System.Linq.Enumerable.TryGetNonEnumeratedCount<") + .Append(element.TypeName) + .Append(">(") + .Append(expression) + .Append(", out int ") + .Append(countVariable) + .AppendLine("))"); + source.Append(indent).AppendLine(" {"); + source.Append(indent) + .Append(" var ") + .Append(bufferVariable) + .Append(" = new global::System.Collections.Generic.List<") + .Append(element.TypeName) + .AppendLine(">();"); + source.Append(indent) + .Append(" foreach (var ") + .Append(itemVariable) + .Append(" in ") + .Append(expression) + .AppendLine(")"); + source.Append(indent) + .Append(" ") + .Append(bufferVariable) + .Append(".Add(") + .Append(itemVariable) + .AppendLine(");"); + source.Append(indent) + .Append(" WriteVarint(destination, ref position, (ulong)") + .Append(bufferVariable) + .AppendLine(".Count);"); + source.Append(indent) + .Append(" foreach (var ") + .Append(itemVariable) + .Append(" in ") + .Append(bufferVariable) + .AppendLine(")"); + source.Append(indent).AppendLine(" {"); + EmitWriteBinaryValue(source, element, itemVariable, indent + " ", suffix + "_item"); + source.Append(indent).AppendLine(" }"); + source.Append(indent).AppendLine(" }"); + source.Append(indent).AppendLine(" else"); + source.Append(indent).AppendLine(" {"); + source.Append(indent) + .Append(" WriteVarint(destination, ref position, (ulong)") + .Append(countVariable) + .AppendLine(");"); + source.Append(indent) + .Append(" foreach (var ") + .Append(itemVariable) + .Append(" in ") + .Append(expression) + .AppendLine(")"); + source.Append(indent).AppendLine(" {"); + EmitWriteBinaryValue(source, element, itemVariable, indent + " ", suffix + "_item"); + source.Append(indent).AppendLine(" }"); + source.Append(indent).AppendLine(" }"); + source.Append(indent).AppendLine("}"); + } + + private static void EmitWriteBinaryObjectValue( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent) + { + BinaryTypeSpec objectType = value.ObjectType ?? throw new InvalidOperationException("Object binary value is missing its type spec."); + + if (value.IsNullableValueType) + { + source.Append(indent) + .Append("if (!") + .Append(expression) + .AppendLine(".HasValue)"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" WriteByte(destination, ref position, NullTag);"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" WriteByte(destination, ref position, ObjectTag);"); + source.Append(indent) + .Append(" WriteBinary_") + .Append(objectType.SafeIdentifier) + .Append("(destination, ref position, ") + .Append(expression) + .AppendLine(".Value);"); + source.Append(indent).AppendLine("}"); + return; + } + + if (value.CanBeNull) + { + source.Append(indent) + .Append("if (") + .Append(expression) + .AppendLine(" is null)"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" WriteByte(destination, ref position, NullTag);"); + source.Append(indent).AppendLine("}"); + source.Append(indent).AppendLine("else"); + source.Append(indent).AppendLine("{"); + source.Append(indent).AppendLine(" WriteByte(destination, ref position, ObjectTag);"); + source.Append(indent) + .Append(" WriteBinary_") + .Append(objectType.SafeIdentifier) + .Append("(destination, ref position, ") + .Append(expression) + .AppendLine(");"); + source.Append(indent).AppendLine("}"); + return; + } + + source.Append(indent).AppendLine("WriteByte(destination, ref position, ObjectTag);"); + source.Append(indent) + .Append("WriteBinary_") + .Append(objectType.SafeIdentifier) + .Append("(destination, ref position, ") + .Append(expression) + .AppendLine(");"); + } + + private static void EmitWriteBinaryScalarValue( + StringBuilder source, + BinaryValueSpec value, + string expression, + string indent) + { + switch (value.ValueKindName) + { + case "String": + source.Append(indent) + .Append("WriteBinaryString(destination, ref position, ") + .Append(expression) + .AppendLine(");"); + return; + case "Guid": + source.Append(indent) + .Append("WriteBinaryString(destination, ref position, ") + .Append(expression) + .AppendLine(".ToString(\"D\"));"); + return; + case "DateOnly": + case "TimeOnly": + source.Append(indent) + .Append("WriteBinaryString(destination, ref position, ") + .Append(expression) + .AppendLine(".ToString(\"O\", global::System.Globalization.CultureInfo.InvariantCulture));"); + return; + case "Enum": + source.Append(indent) + .Append("WriteBinaryInteger(destination, ref position, global::System.Convert.ToInt64(") + .Append(expression) + .AppendLine(", global::System.Globalization.CultureInfo.InvariantCulture));"); + return; + case "Boolean": + source.Append(indent) + .Append("WriteByte(destination, ref position, ") + .Append(expression) + .AppendLine(" ? TrueTag : FalseTag);"); + return; + case "Byte": + case "SByte": + case "Int16": + case "UInt16": + case "Int32": + case "UInt32": + case "Int64": + source.Append(indent) + .Append("WriteBinaryInteger(destination, ref position, (long)") + .Append(expression) + .AppendLine(");"); + return; + case "Single": + source.Append(indent) + .Append("WriteBinaryDouble(destination, ref position, (double)") + .Append(expression) + .AppendLine(");"); + return; + case "Double": + source.Append(indent) + .Append("WriteBinaryDouble(destination, ref position, ") + .Append(expression) + .AppendLine(");"); + return; + case "Decimal": + source.Append(indent) + .Append("WriteBinaryDecimal(destination, ref position, ") + .Append(expression) + .AppendLine(");"); + return; + default: + throw new InvalidOperationException($"Unsupported generated binary value kind '{value.ValueKindName}'."); + } + } + + private static void EmitBinaryPrimitiveHelpers(StringBuilder source) + { + source.AppendLine(" private static int GetBinaryStringSize(string? value)"); + source.AppendLine(" => value is null ? 1 : 1 + GetLengthPrefixedStringSize(value);"); + source.AppendLine(); + source.AppendLine(" private static int GetLengthPrefixedStringSize(string value)"); + source.AppendLine(" {"); + source.AppendLine(" int byteCount = global::System.Text.Encoding.UTF8.GetByteCount(value);"); + source.AppendLine(" return GetVarintSize((ulong)byteCount) + byteCount;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static int GetLengthPrefixedUtf8Size(global::System.ReadOnlySpan value)"); + source.AppendLine(" => GetVarintSize((ulong)value.Length) + value.Length;"); + source.AppendLine(); + source.AppendLine(" private static int GetVarintSize(ulong value)"); + source.AppendLine(" => global::CSharpDB.Storage.Serialization.Varint.SizeOf(value);"); + source.AppendLine(); + source.AppendLine(" private static bool IsBinaryRecordPayload(global::System.ReadOnlySpan payload)"); + source.AppendLine(" => payload.Length >= 3 && payload[0] == RecordFormatMarker && payload[1] == RecordFormatMagic && payload[2] == RecordFormatVersion;"); + source.AppendLine(); + source.AppendLine(" private static int ReadBinaryRecordNullBitmap(global::System.ReadOnlySpan payload, ref int position, int byteCount)"); + source.AppendLine(" {"); + source.AppendLine(" int start = position;"); + source.AppendLine(" EnsureAvailable(payload, position, byteCount);"); + source.AppendLine(" position += byteCount;"); + source.AppendLine(" return start;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool IsBinaryRecordNull(global::System.ReadOnlySpan payload, int nullBitmapStart, int fieldIndex)"); + source.AppendLine(" => (payload[nullBitmapStart + (fieldIndex >> 3)] & (1 << (fieldIndex & 7))) != 0;"); + source.AppendLine(); + source.AppendLine(" private static void SetBinaryRecordNull(global::System.Span destination, int nullBitmapStart, int fieldIndex)"); + source.AppendLine(" => destination[nullBitmapStart + (fieldIndex >> 3)] = (byte)(destination[nullBitmapStart + (fieldIndex >> 3)] | (1 << (fieldIndex & 7)));"); + source.AppendLine(); + source.AppendLine(" private static string ReadBinaryRecordString(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => global::System.Text.Encoding.UTF8.GetString(ReadLengthPrefixedBytes(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static global::System.ReadOnlySpan ReadBinaryRecordStringUtf8(global::System.ReadOnlySpan payload, scoped ref int position)"); + source.AppendLine(" => ReadLengthPrefixedBytes(payload, ref position);"); + source.AppendLine(); + source.AppendLine(" private static global::System.Guid ReadBinaryRecordGuid(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, 16);"); + source.AppendLine(" global::System.Guid value = new global::System.Guid(payload.Slice(position, 16));"); + source.AppendLine(" position += 16;"); + source.AppendLine(" return value;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static global::System.DateOnly ReadBinaryRecordDateOnly(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => global::System.DateOnly.FromDayNumber(ReadBinaryRecordInt32(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static global::System.TimeOnly ReadBinaryRecordTimeOnly(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => new global::System.TimeOnly(ReadInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static TEnum ReadBinaryRecordEnum(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" where TEnum : struct"); + source.AppendLine(" => (TEnum)global::System.Enum.ToObject(typeof(TEnum), ReadInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static bool ReadBinaryRecordBoolean(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => ReadByte(payload, ref position) != 0;"); + source.AppendLine(); + source.AppendLine(" private static byte ReadBinaryRecordByte(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => ReadByte(payload, ref position);"); + source.AppendLine(); + source.AppendLine(" private static sbyte ReadBinaryRecordSByte(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => unchecked((sbyte)ReadByte(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static short ReadBinaryRecordInt16(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(short));"); + source.AppendLine(" short value = global::System.Buffers.Binary.BinaryPrimitives.ReadInt16LittleEndian(payload[position..]);"); + source.AppendLine(" position += sizeof(short);"); + source.AppendLine(" return value;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static ushort ReadBinaryRecordUInt16(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(ushort));"); + source.AppendLine(" ushort value = global::System.Buffers.Binary.BinaryPrimitives.ReadUInt16LittleEndian(payload[position..]);"); + source.AppendLine(" position += sizeof(ushort);"); + source.AppendLine(" return value;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static int ReadBinaryRecordInt32(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(int));"); + source.AppendLine(" int value = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[position..]);"); + source.AppendLine(" position += sizeof(int);"); + source.AppendLine(" return value;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static uint ReadBinaryRecordUInt32(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(uint));"); + source.AppendLine(" uint value = global::System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(payload[position..]);"); + source.AppendLine(" position += sizeof(uint);"); + source.AppendLine(" return value;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static float ReadBinaryRecordSingle(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => global::System.BitConverter.Int32BitsToSingle(ReadBinaryRecordInt32(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static double ReadBinaryRecordDouble(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => global::System.BitConverter.Int64BitsToDouble(ReadInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static decimal ReadBinaryRecordDecimal(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(int) * 4);"); + source.AppendLine(" int lo = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[position..]);"); + source.AppendLine(" int mid = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[(position + 4)..]);"); + source.AppendLine(" int hi = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[(position + 8)..]);"); + source.AppendLine(" int flags = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[(position + 12)..]);"); + source.AppendLine(" position += sizeof(int) * 4;"); + source.AppendLine(" return new decimal(new int[] { lo, mid, hi, flags });"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static string? ReadBinaryString(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag == NullTag)"); + source.AppendLine(" return null;"); + source.AppendLine(); + source.AppendLine(" EnsureTag(tag, StringTag);"); + source.AppendLine(" return global::System.Text.Encoding.UTF8.GetString(ReadLengthPrefixedBytes(payload, ref position));"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static string ReadRequiredBinaryString(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => ReadBinaryString(payload, ref position)"); + source.AppendLine(" ?? throw new global::CSharpDB.Primitives.CSharpDbException(global::CSharpDB.Primitives.ErrorCode.CorruptDatabase, \"Generated binary collection payload contained null for a required string value.\");"); + source.AppendLine(); + source.AppendLine(" private static global::System.Guid ReadBinaryGuid(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => global::System.Guid.Parse(ReadRequiredBinaryString(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static global::System.Guid? ReadNullableBinaryGuid(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" string? value = ReadBinaryString(payload, ref position);"); + source.AppendLine(" return value is null ? null : global::System.Guid.Parse(value);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static global::System.DateOnly ReadBinaryDateOnly(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => global::System.DateOnly.ParseExact(ReadRequiredBinaryString(payload, ref position), \"O\", global::System.Globalization.CultureInfo.InvariantCulture);"); + source.AppendLine(); + source.AppendLine(" private static global::System.DateOnly? ReadNullableBinaryDateOnly(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" string? value = ReadBinaryString(payload, ref position);"); + source.AppendLine(" return value is null ? null : global::System.DateOnly.ParseExact(value, \"O\", global::System.Globalization.CultureInfo.InvariantCulture);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static global::System.TimeOnly ReadBinaryTimeOnly(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => global::System.TimeOnly.ParseExact(ReadRequiredBinaryString(payload, ref position), \"O\", global::System.Globalization.CultureInfo.InvariantCulture);"); + source.AppendLine(); + source.AppendLine(" private static global::System.TimeOnly? ReadNullableBinaryTimeOnly(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" string? value = ReadBinaryString(payload, ref position);"); + source.AppendLine(" return value is null ? null : global::System.TimeOnly.ParseExact(value, \"O\", global::System.Globalization.CultureInfo.InvariantCulture);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static TEnum ReadBinaryEnum(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" where TEnum : struct"); + source.AppendLine(" => (TEnum)global::System.Enum.ToObject(typeof(TEnum), ReadBinaryInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static TEnum? ReadNullableBinaryEnum(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" where TEnum : struct"); + source.AppendLine(" {"); + source.AppendLine(" long? value = ReadNullableBinaryInt64(payload, ref position);"); + source.AppendLine(" return value.HasValue ? (TEnum)global::System.Enum.ToObject(typeof(TEnum), value.Value) : null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool ReadBinaryBoolean(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag == TrueTag)"); + source.AppendLine(" return true;"); + source.AppendLine(" if (tag == FalseTag)"); + source.AppendLine(" return false;"); + source.AppendLine(); + source.AppendLine(" ThrowUnexpectedTag(tag, TrueTag);"); + source.AppendLine(" return false;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool? ReadNullableBinaryBoolean(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag == NullTag)"); + source.AppendLine(" return null;"); + source.AppendLine(" if (tag == TrueTag)"); + source.AppendLine(" return true;"); + source.AppendLine(" if (tag == FalseTag)"); + source.AppendLine(" return false;"); + source.AppendLine(); + source.AppendLine(" ThrowUnexpectedTag(tag, TrueTag);"); + source.AppendLine(" return null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static byte ReadBinaryByte(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => checked((byte)ReadBinaryInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static byte? ReadNullableBinaryByte(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" long? value = ReadNullableBinaryInt64(payload, ref position);"); + source.AppendLine(" return value.HasValue ? checked((byte)value.Value) : null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static sbyte ReadBinarySByte(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => checked((sbyte)ReadBinaryInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static sbyte? ReadNullableBinarySByte(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" long? value = ReadNullableBinaryInt64(payload, ref position);"); + source.AppendLine(" return value.HasValue ? checked((sbyte)value.Value) : null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static short ReadBinaryInt16(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => checked((short)ReadBinaryInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static short? ReadNullableBinaryInt16(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" long? value = ReadNullableBinaryInt64(payload, ref position);"); + source.AppendLine(" return value.HasValue ? checked((short)value.Value) : null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static ushort ReadBinaryUInt16(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => checked((ushort)ReadBinaryInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static ushort? ReadNullableBinaryUInt16(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" long? value = ReadNullableBinaryInt64(payload, ref position);"); + source.AppendLine(" return value.HasValue ? checked((ushort)value.Value) : null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static int ReadBinaryInt32(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => checked((int)ReadBinaryInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static int? ReadNullableBinaryInt32(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" long? value = ReadNullableBinaryInt64(payload, ref position);"); + source.AppendLine(" return value.HasValue ? checked((int)value.Value) : null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static uint ReadBinaryUInt32(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => checked((uint)ReadBinaryInt64(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static uint? ReadNullableBinaryUInt32(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" long? value = ReadNullableBinaryInt64(payload, ref position);"); + source.AppendLine(" return value.HasValue ? checked((uint)value.Value) : null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static long ReadBinaryInt64(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => ReadBinaryIntegerPayload(payload, ref position, ReadByte(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static long? ReadNullableBinaryInt64(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" return tag == NullTag ? null : ReadBinaryIntegerPayload(payload, ref position, tag);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static long ReadBinaryIntegerPayload(global::System.ReadOnlySpan payload, ref int position, byte tag)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureTag(tag, IntegerTag);"); + source.AppendLine(" return ReadInt64(payload, ref position);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static float ReadBinarySingle(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => checked((float)ReadBinaryDouble(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static float? ReadNullableBinarySingle(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" double? value = ReadNullableBinaryDouble(payload, ref position);"); + source.AppendLine(" return value.HasValue ? checked((float)value.Value) : null;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static double ReadBinaryDouble(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => ReadBinaryDoublePayload(payload, ref position, ReadByte(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static double? ReadNullableBinaryDouble(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" return tag == NullTag ? null : ReadBinaryDoublePayload(payload, ref position, tag);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static double ReadBinaryDoublePayload(global::System.ReadOnlySpan payload, ref int position, byte tag)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureTag(tag, DoubleTag);"); + source.AppendLine(" return global::System.BitConverter.Int64BitsToDouble(ReadInt64(payload, ref position));"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static decimal ReadBinaryDecimal(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" => ReadBinaryDecimalPayload(payload, ref position, ReadByte(payload, ref position));"); + source.AppendLine(); + source.AppendLine(" private static decimal? ReadNullableBinaryDecimal(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" return tag == NullTag ? null : ReadBinaryDecimalPayload(payload, ref position, tag);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static decimal ReadBinaryDecimalPayload(global::System.ReadOnlySpan payload, ref int position, byte tag)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureTag(tag, DecimalTag);"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(int) * 4);"); + source.AppendLine(" int lo = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[position..]);"); + source.AppendLine(" int mid = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[(position + 4)..]);"); + source.AppendLine(" int hi = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[(position + 8)..]);"); + source.AppendLine(" int flags = global::System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload[(position + 12)..]);"); + source.AppendLine(" position += sizeof(int) * 4;"); + source.AppendLine(" return new decimal(new int[] { lo, mid, hi, flags });"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void SkipBinaryValue(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" switch (tag)"); + source.AppendLine(" {"); + source.AppendLine(" case NullTag:"); + source.AppendLine(" case FalseTag:"); + source.AppendLine(" case TrueTag:"); + source.AppendLine(" return;"); + source.AppendLine(" case StringTag:"); + source.AppendLine(" _ = ReadLengthPrefixedBytes(payload, ref position);"); + source.AppendLine(" return;"); + source.AppendLine(" case IntegerTag:"); + source.AppendLine(" case DoubleTag:"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(long));"); + source.AppendLine(" position += sizeof(long);"); + source.AppendLine(" return;"); + source.AppendLine(" case DecimalTag:"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(int) * 4);"); + source.AppendLine(" position += sizeof(int) * 4;"); + source.AppendLine(" return;"); + source.AppendLine(" case ObjectTag:"); + source.AppendLine(" SkipBinaryObject(payload, ref position);"); + source.AppendLine(" return;"); + source.AppendLine(" case ArrayTag:"); + source.AppendLine(" ulong elementCount = ReadVarint(payload, ref position);"); + source.AppendLine(" for (ulong i = 0; i < elementCount; i++)"); + source.AppendLine(" SkipBinaryValue(payload, ref position);"); + source.AppendLine(" return;"); + source.AppendLine(" default:"); + source.AppendLine(" ThrowUnexpectedTag(tag, NullTag);"); + source.AppendLine(" return;"); + source.AppendLine(" }"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void SkipBinaryObject(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" ulong fieldCount = ReadVarint(payload, ref position);"); + source.AppendLine(" for (ulong i = 0; i < fieldCount; i++)"); + source.AppendLine(" {"); + source.AppendLine(" _ = ReadLengthPrefixedBytes(payload, ref position);"); + source.AppendLine(" SkipBinaryValue(payload, ref position);"); + source.AppendLine(" }"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static byte ReadByte(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, 1);"); + source.AppendLine(" return payload[position++];"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static ulong ReadVarint(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, 1);"); + source.AppendLine(" ulong value = global::CSharpDB.Storage.Serialization.Varint.Read(payload[position..], out int bytesRead);"); + source.AppendLine(" EnsureAvailable(payload, position, bytesRead);"); + source.AppendLine(" position += bytesRead;"); + source.AppendLine(" return value;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static global::System.ReadOnlySpan ReadLengthPrefixedBytes(global::System.ReadOnlySpan payload, scoped ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" int length = checked((int)ReadVarint(payload, ref position));"); + source.AppendLine(" EnsureAvailable(payload, position, length);"); + source.AppendLine(" global::System.ReadOnlySpan value = payload.Slice(position, length);"); + source.AppendLine(" position += length;"); + source.AppendLine(" return value;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool TryReadExpectedFieldName(global::System.ReadOnlySpan payload, ref int position, global::System.ReadOnlySpan expected)"); + source.AppendLine(" {"); + source.AppendLine(" int start = position;"); + source.AppendLine(" global::System.ReadOnlySpan fieldName = ReadLengthPrefixedBytes(payload, ref position);"); + source.AppendLine(" if (!fieldName.SequenceEqual(expected))"); + source.AppendLine(" {"); + source.AppendLine(" position = start;"); + source.AppendLine(" return false;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" return true;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool TryReadExpectedObjectFieldCount(global::System.ReadOnlySpan payload, ref int position, int expectedFieldCount)"); + source.AppendLine(" {"); + source.AppendLine(" int current = position;"); + source.AppendLine(" ulong fieldCount = ReadVarint(payload, ref current);"); + source.AppendLine(" if (fieldCount != (ulong)expectedFieldCount)"); + source.AppendLine(" return false;"); + source.AppendLine(); + source.AppendLine(" position = current;"); + source.AppendLine(" return true;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool TrySkipExpectedBinaryField(global::System.ReadOnlySpan payload, ref int position, global::System.ReadOnlySpan expected)"); + source.AppendLine(" {"); + source.AppendLine(" if (!TryReadExpectedFieldName(payload, ref position, expected))"); + source.AppendLine(" return false;"); + source.AppendLine(); + source.AppendLine(" SkipBinaryValue(payload, ref position);"); + source.AppendLine(" return true;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void SkipBinaryField(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" _ = ReadLengthPrefixedBytes(payload, ref position);"); + source.AppendLine(" SkipBinaryValue(payload, ref position);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool TryReadBinaryObjectPayload(global::System.ReadOnlySpan payload, ref int position, out int objectStart, out int objectLength)"); + source.AppendLine(" {"); + source.AppendLine(" objectStart = 0;"); + source.AppendLine(" objectLength = 0;"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag != ObjectTag)"); + source.AppendLine(" return false;"); + source.AppendLine(); + source.AppendLine(" objectStart = position;"); + source.AppendLine(" SkipBinaryObject(payload, ref position);"); + source.AppendLine(" objectLength = position - objectStart;"); + source.AppendLine(" return true;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool TryReadBinaryPayloadFieldValue(global::System.ReadOnlySpan payload, ref int position, out global::CSharpDB.Primitives.DbValue value)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" switch (tag)"); + source.AppendLine(" {"); + source.AppendLine(" case StringTag:"); + source.AppendLine(" value = global::CSharpDB.Primitives.DbValue.FromText(global::System.Text.Encoding.UTF8.GetString(ReadLengthPrefixedBytes(payload, ref position)));"); + source.AppendLine(" return true;"); + source.AppendLine(" case IntegerTag:"); + source.AppendLine(" value = global::CSharpDB.Primitives.DbValue.FromInteger(ReadInt64(payload, ref position));"); + source.AppendLine(" return true;"); + source.AppendLine(" case DoubleTag:"); + source.AppendLine(" value = global::CSharpDB.Primitives.DbValue.FromReal(global::System.BitConverter.Int64BitsToDouble(ReadInt64(payload, ref position)));"); + source.AppendLine(" return true;"); + source.AppendLine(" default:"); + source.AppendLine(" value = default;"); + source.AppendLine(" return false;"); + source.AppendLine(" }"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool TryReadBinaryPayloadFieldInt64(global::System.ReadOnlySpan payload, ref int position, out long value)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag == IntegerTag)"); + source.AppendLine(" {"); + source.AppendLine(" value = ReadInt64(payload, ref position);"); + source.AppendLine(" return true;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" value = 0;"); + source.AppendLine(" return false;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool TryReadBinaryPayloadFieldText(global::System.ReadOnlySpan payload, ref int position, out string? value)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag == StringTag)"); + source.AppendLine(" {"); + source.AppendLine(" value = global::System.Text.Encoding.UTF8.GetString(ReadLengthPrefixedBytes(payload, ref position));"); + source.AppendLine(" return true;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" value = null;"); + source.AppendLine(" return false;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static bool TryReadBinaryPayloadFieldTextUtf8(global::System.ReadOnlySpan payload, scoped ref int position, out global::System.ReadOnlySpan value)"); + source.AppendLine(" {"); + source.AppendLine(" byte tag = ReadByte(payload, ref position);"); + source.AppendLine(" if (tag == StringTag)"); + source.AppendLine(" {"); + source.AppendLine(" value = ReadLengthPrefixedBytes(payload, ref position);"); + source.AppendLine(" return true;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" value = default;"); + source.AppendLine(" return false;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static long ReadInt64(global::System.ReadOnlySpan payload, ref int position)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(payload, position, sizeof(long));"); + source.AppendLine(" long value = global::System.Buffers.Binary.BinaryPrimitives.ReadInt64LittleEndian(payload[position..]);"); + source.AppendLine(" position += sizeof(long);"); + source.AppendLine(" return value;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void EnsureTag(byte actual, byte expected)"); + source.AppendLine(" {"); + source.AppendLine(" if (actual != expected)"); + source.AppendLine(" ThrowUnexpectedTag(actual, expected);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void EnsureAvailable(global::System.ReadOnlySpan payload, int position, int count)"); + source.AppendLine(" {"); + source.AppendLine(" if (count < 0 || position < 0 || payload.Length - position < count)"); + source.AppendLine(" throw new global::CSharpDB.Primitives.CSharpDbException(global::CSharpDB.Primitives.ErrorCode.CorruptDatabase, \"Invalid generated binary collection payload length.\");"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void ThrowUnexpectedTag(byte actual, byte expected)"); + source.AppendLine(" => throw new global::CSharpDB.Primitives.CSharpDbException(global::CSharpDB.Primitives.ErrorCode.CorruptDatabase, $\"Invalid generated binary collection payload tag {actual}; expected {expected}.\");"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryString(global::System.Span destination, ref int position, string? value)"); + source.AppendLine(" {"); + source.AppendLine(" if (value is null)"); + source.AppendLine(" {"); + source.AppendLine(" WriteByte(destination, ref position, NullTag);"); + source.AppendLine(" return;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" WriteByte(destination, ref position, StringTag);"); + source.AppendLine(" WriteLengthPrefixedString(destination, ref position, value);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordGuid(global::System.Span destination, ref int position, global::System.Guid value)"); + source.AppendLine(" {"); + source.AppendLine(" EnsureAvailable(destination, position, 16);"); + source.AppendLine(" if (!value.TryWriteBytes(destination.Slice(position, 16)))"); + source.AppendLine(" throw new global::System.InvalidOperationException(\"Failed to write generated binary record Guid value.\");"); + source.AppendLine(" position += 16;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordDateOnly(global::System.Span destination, ref int position, global::System.DateOnly value)"); + source.AppendLine(" => WriteBinaryRecordInt32(destination, ref position, value.DayNumber);"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordTimeOnly(global::System.Span destination, ref int position, global::System.TimeOnly value)"); + source.AppendLine(" => WriteInt64(destination, ref position, value.Ticks);"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordEnum(global::System.Span destination, ref int position, TEnum value)"); + source.AppendLine(" where TEnum : struct"); + source.AppendLine(" => WriteInt64(destination, ref position, global::System.Convert.ToInt64(value, global::System.Globalization.CultureInfo.InvariantCulture));"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordBoolean(global::System.Span destination, ref int position, bool value)"); + source.AppendLine(" => WriteByte(destination, ref position, value ? (byte)1 : (byte)0);"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordByte(global::System.Span destination, ref int position, byte value)"); + source.AppendLine(" => WriteByte(destination, ref position, value);"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordSByte(global::System.Span destination, ref int position, sbyte value)"); + source.AppendLine(" => WriteByte(destination, ref position, unchecked((byte)value));"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordInt16(global::System.Span destination, ref int position, short value)"); + source.AppendLine(" {"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt16LittleEndian(destination.Slice(position, sizeof(short)), value);"); + source.AppendLine(" position += sizeof(short);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordUInt16(global::System.Span destination, ref int position, ushort value)"); + source.AppendLine(" {"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteUInt16LittleEndian(destination.Slice(position, sizeof(ushort)), value);"); + source.AppendLine(" position += sizeof(ushort);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordInt32(global::System.Span destination, ref int position, int value)"); + source.AppendLine(" {"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(position, sizeof(int)), value);"); + source.AppendLine(" position += sizeof(int);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordUInt32(global::System.Span destination, ref int position, uint value)"); + source.AppendLine(" {"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(destination.Slice(position, sizeof(uint)), value);"); + source.AppendLine(" position += sizeof(uint);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordSingle(global::System.Span destination, ref int position, float value)"); + source.AppendLine(" => WriteBinaryRecordInt32(destination, ref position, global::System.BitConverter.SingleToInt32Bits(value));"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordDouble(global::System.Span destination, ref int position, double value)"); + source.AppendLine(" => WriteInt64(destination, ref position, global::System.BitConverter.DoubleToInt64Bits(value));"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryRecordDecimal(global::System.Span destination, ref int position, decimal value)"); + source.AppendLine(" {"); + source.AppendLine(" int[] bits = decimal.GetBits(value);"); + source.AppendLine(" global::System.Span span = destination.Slice(position, sizeof(int) * 4);"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(span, bits[0]);"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(span[4..], bits[1]);"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(span[8..], bits[2]);"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(span[12..], bits[3]);"); + source.AppendLine(" position += sizeof(int) * 4;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryInteger(global::System.Span destination, ref int position, long value)"); + source.AppendLine(" {"); + source.AppendLine(" WriteByte(destination, ref position, IntegerTag);"); + source.AppendLine(" WriteInt64(destination, ref position, value);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryDouble(global::System.Span destination, ref int position, double value)"); + source.AppendLine(" {"); + source.AppendLine(" WriteByte(destination, ref position, DoubleTag);"); + source.AppendLine(" WriteInt64(destination, ref position, global::System.BitConverter.DoubleToInt64Bits(value));"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBinaryDecimal(global::System.Span destination, ref int position, decimal value)"); + source.AppendLine(" {"); + source.AppendLine(" WriteByte(destination, ref position, DecimalTag);"); + source.AppendLine(" int[] bits = decimal.GetBits(value);"); + source.AppendLine(" global::System.Span span = destination.Slice(position, sizeof(int) * 4);"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(span, bits[0]);"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(span[4..], bits[1]);"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(span[8..], bits[2]);"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(span[12..], bits[3]);"); + source.AppendLine(" position += sizeof(int) * 4;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteByte(global::System.Span destination, ref int position, byte value)"); + source.AppendLine(" {"); + source.AppendLine(" destination[position] = value;"); + source.AppendLine(" position++;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteVarint(global::System.Span destination, ref int position, ulong value)"); + source.AppendLine(" {"); + source.AppendLine(" position += global::CSharpDB.Storage.Serialization.Varint.Write(destination[position..], value);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteBytes(global::System.Span destination, ref int position, global::System.ReadOnlySpan value)"); + source.AppendLine(" {"); + source.AppendLine(" value.CopyTo(destination[position..]);"); + source.AppendLine(" position += value.Length;"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteLengthPrefixedString(global::System.Span destination, ref int position, string value)"); + source.AppendLine(" {"); + source.AppendLine(" int byteCount = global::System.Text.Encoding.UTF8.GetByteCount(value);"); + source.AppendLine(" WriteVarint(destination, ref position, (ulong)byteCount);"); + source.AppendLine(" position += global::System.Text.Encoding.UTF8.GetBytes(value.AsSpan(), destination.Slice(position, byteCount));"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteLengthPrefixedUtf8(global::System.Span destination, ref int position, global::System.ReadOnlySpan value)"); + source.AppendLine(" {"); + source.AppendLine(" WriteVarint(destination, ref position, (ulong)value.Length);"); + source.AppendLine(" WriteBytes(destination, ref position, value);"); + source.AppendLine(" }"); + source.AppendLine(); + source.AppendLine(" private static void WriteInt64(global::System.Span destination, ref int position, long value)"); + source.AppendLine(" {"); + source.AppendLine(" global::System.Buffers.Binary.BinaryPrimitives.WriteInt64LittleEndian(destination.Slice(position, sizeof(long)), value);"); + source.AppendLine(" position += sizeof(long);"); + source.AppendLine(" }"); + } + private readonly struct CollectionGenerationResult { public CollectionGenerationResult(CollectionModelTarget? target, ImmutableArray diagnostics) @@ -1048,7 +4098,8 @@ public CollectionModelTarget( string jsonContextTypeName, string partialTypeKeyword, string safeIdentifier, - ImmutableArray fields) + ImmutableArray fields, + BinaryTypeSpec? binaryModel) { NamespaceName = namespaceName; TypeName = typeName; @@ -1057,6 +4108,7 @@ public CollectionModelTarget( PartialTypeKeyword = partialTypeKeyword; SafeIdentifier = safeIdentifier; Fields = fields; + BinaryModel = binaryModel; } public string? NamespaceName { get; } @@ -1072,6 +4124,8 @@ public CollectionModelTarget( public string SafeIdentifier { get; } public ImmutableArray Fields { get; } + + public BinaryTypeSpec? BinaryModel { get; } } private readonly struct CollectionFieldSpec @@ -1113,6 +4167,193 @@ public CollectionFieldSpec( public string? AccessorHelperSource { get; } } + private sealed class BinaryTypeSpec + { + public BinaryTypeSpec( + string typeName, + string safeIdentifier, + ImmutableArray members, + BinaryConstructorSpec constructor) + { + TypeName = typeName; + SafeIdentifier = safeIdentifier; + Members = members; + Constructor = constructor; + } + + public string TypeName { get; } + + public string SafeIdentifier { get; } + + public ImmutableArray Members { get; } + + public BinaryConstructorSpec Constructor { get; } + } + + private readonly struct BinaryConstructorSpec + { + public BinaryConstructorSpec(ImmutableArray parameterMemberIndexes) + { + ParameterMemberIndexes = parameterMemberIndexes; + } + + public ImmutableArray ParameterMemberIndexes { get; } + } + + private readonly struct BinaryFieldReaderSpec + { + public BinaryFieldReaderSpec( + ImmutableArray memberIndexes, + BinaryValueSpec value, + string dataKindName) + { + MemberIndexes = memberIndexes; + Value = value; + DataKindName = dataKindName; + } + + public ImmutableArray MemberIndexes { get; } + + public BinaryValueSpec Value { get; } + + public string DataKindName { get; } + } + + private readonly struct BinaryMemberCandidate + { + public BinaryMemberCandidate( + ISymbol member, + ITypeSymbol type, + BinaryValueSpec value) + { + Member = member; + Type = type; + Value = value; + } + + public ISymbol Member { get; } + + public ITypeSymbol Type { get; } + + public BinaryValueSpec Value { get; } + } + + private readonly struct BinaryMemberSpec + { + public BinaryMemberSpec( + string clrName, + string escapedClrName, + string jsonName, + BinaryValueSpec value) + { + ClrName = clrName; + EscapedClrName = escapedClrName; + JsonName = jsonName; + Value = value; + } + + public string ClrName { get; } + + public string EscapedClrName { get; } + + public string JsonName { get; } + + public BinaryValueSpec Value { get; } + } + + private sealed class BinaryValueSpec + { + private BinaryValueSpec( + string typeName, + string effectiveTypeName, + string valueKindName, + bool canBeNull, + bool isNullableValueType, + bool isArray, + bool isArrayType, + BinaryValueSpec? element, + BinaryTypeSpec? objectType) + { + TypeName = typeName; + EffectiveTypeName = effectiveTypeName; + ValueKindName = valueKindName; + CanBeNull = canBeNull; + IsNullableValueType = isNullableValueType; + IsArray = isArray; + IsArrayType = isArrayType; + Element = element; + ObjectType = objectType; + } + + public static BinaryValueSpec ForScalar( + string typeName, + string effectiveTypeName, + string valueKindName, + bool canBeNull, + bool isNullableValueType) + => new( + typeName, + effectiveTypeName, + valueKindName, + canBeNull, + isNullableValueType, + isArray: false, + isArrayType: false, + element: null, + objectType: null); + + public static BinaryValueSpec ForObject( + string typeName, + string effectiveTypeName, + bool canBeNull, + bool isNullableValueType, + BinaryTypeSpec objectType) + => new( + typeName, + effectiveTypeName, + "Object", + canBeNull, + isNullableValueType, + isArray: false, + isArrayType: false, + element: null, + objectType); + + public static BinaryValueSpec ForArray( + string typeName, + bool isArrayType, + bool canBeNull, + BinaryValueSpec element) + => new( + typeName, + typeName, + "Array", + canBeNull, + isNullableValueType: false, + isArray: true, + isArrayType, + element, + objectType: null); + + public string TypeName { get; } + + public string EffectiveTypeName { get; } + + public string ValueKindName { get; } + + public bool CanBeNull { get; } + + public bool IsNullableValueType { get; } + + public bool IsArray { get; } + + public bool IsArrayType { get; } + + public BinaryValueSpec? Element { get; } + + public BinaryTypeSpec? ObjectType { get; } + } + private readonly struct FieldPathSegment { public FieldPathSegment( diff --git a/src/CSharpDB.Generators/README.md b/src/CSharpDB.Generators/README.md index a601d2ce..1cd96bb3 100644 --- a/src/CSharpDB.Generators/README.md +++ b/src/CSharpDB.Generators/README.md @@ -5,6 +5,8 @@ Source generator for CSharpDB generated collection models. Use this package with `CSharpDB.Engine` when you want: - generated collection codecs backed by a `System.Text.Json` source-generated context +- generated binary direct-payload writes for supported document graphs, with + source-generated JSON fallback for unsupported binary shapes - generated `CollectionField<,>` descriptors such as `User.Collection.Email` - flattened nested descriptors such as `User.Collection.Address_City` and `User.Collection.Orders_Sku` - trim-safe typed collection access through `Database.GetGeneratedCollectionAsync(...)` @@ -41,6 +43,10 @@ renamed with `JsonPropertyName`. Unsupported public members are ignored with a build warning (`CDBGEN007`) so generator coverage gaps fail loudly instead of silently omitting descriptors. +When every serialized public member in the document graph maps to the supported +binary collection payload shape, generated collections write the binary +direct-payload format. If a member does not fit that binary shape, generated +collections keep the existing source-generated JSON payload path. At runtime, `GetGeneratedCollectionAsync(...)` also expects existing collection indexes for that document type to bind through registered generated diff --git a/src/CSharpDB.Storage/Indexing/OrderedTextIndexKeyCodec.cs b/src/CSharpDB.Storage/Indexing/OrderedTextIndexKeyCodec.cs index c1810fa6..2c6f3f42 100644 --- a/src/CSharpDB.Storage/Indexing/OrderedTextIndexKeyCodec.cs +++ b/src/CSharpDB.Storage/Indexing/OrderedTextIndexKeyCodec.cs @@ -32,11 +32,19 @@ public static long ComputeKey(string text) } Pack: + return PackUtf8Prefix(utf8Prefix, totalByteCount); + } + + public static long ComputeKey(ReadOnlySpan utf8) + => PackUtf8Prefix(utf8, utf8.Length); + + private static long PackUtf8Prefix(ReadOnlySpan utf8, int totalByteCount) + { int symbolCount = Math.Min(totalByteCount, MaxPrefixBytes); ulong packed = 0; for (int i = 0; i < symbolCount; i++) - packed = (packed << BitsPerSymbol) | (uint)(utf8Prefix[i] + 1); + packed = (packed << BitsPerSymbol) | (uint)(utf8[i] + 1); if (totalByteCount < MaxPrefixBytes) packed <<= BitsPerSymbol; diff --git a/src/CSharpDB.Storage/Serialization/CollectionBinaryDocumentCodec.cs b/src/CSharpDB.Storage/Serialization/CollectionBinaryDocumentCodec.cs index b5d54e7c..2f74aee1 100644 --- a/src/CSharpDB.Storage/Serialization/CollectionBinaryDocumentCodec.cs +++ b/src/CSharpDB.Storage/Serialization/CollectionBinaryDocumentCodec.cs @@ -116,18 +116,36 @@ public static bool TryReadValue(ReadOnlySpan payload, byte[][] pathSegment return TryReadValueFromObject(payload, 0, pathSegmentsUtf8, out value); } + public static bool TryReadValue(ReadOnlySpan payload, ReadOnlySpan pathSegmentUtf8, out DbValue value) + => TryReadValueFromObject(payload, pathSegmentUtf8, out value); + public static bool TryReadInt64(ReadOnlySpan payload, byte[][] pathSegmentsUtf8, out long value) { ArgumentNullException.ThrowIfNull(pathSegmentsUtf8); return TryReadInt64FromObject(payload, 0, pathSegmentsUtf8, out value); } + public static bool TryReadInt64(ReadOnlySpan payload, ReadOnlySpan pathSegmentUtf8, out long value) + => TryReadInt64FromObject(payload, pathSegmentUtf8, out value); + public static bool TryReadString(ReadOnlySpan payload, byte[][] pathSegmentsUtf8, out string? value) { ArgumentNullException.ThrowIfNull(pathSegmentsUtf8); return TryReadStringFromObject(payload, 0, pathSegmentsUtf8, out value); } + public static bool TryReadString(ReadOnlySpan payload, ReadOnlySpan pathSegmentUtf8, out string? value) + => TryReadStringFromObject(payload, pathSegmentUtf8, out value); + + public static bool TryReadStringUtf8(ReadOnlySpan payload, byte[][] pathSegmentsUtf8, out ReadOnlySpan value) + { + ArgumentNullException.ThrowIfNull(pathSegmentsUtf8); + return TryReadStringUtf8FromObject(payload, 0, pathSegmentsUtf8, out value); + } + + public static bool TryReadStringUtf8(ReadOnlySpan payload, ReadOnlySpan pathSegmentUtf8, out ReadOnlySpan value) + => TryReadStringUtf8FromObject(payload, pathSegmentUtf8, out value); + public static bool TryReadBoolean(ReadOnlySpan payload, byte[][] pathSegmentsUtf8, out bool value) { ArgumentNullException.ThrowIfNull(pathSegmentsUtf8); @@ -180,6 +198,12 @@ public static bool TryTextEquals(ReadOnlySpan payload, byte[][] pathSegmen return TryTextEqualsFromObject(payload, 0, pathSegmentsUtf8, expectedValue); } + public static bool TryTextEquals(ReadOnlySpan payload, ReadOnlySpan pathSegmentUtf8, string expectedValue) + { + ArgumentNullException.ThrowIfNull(expectedValue); + return TryTextEqualsFromObject(payload, pathSegmentUtf8, expectedValue); + } + public static byte[] EncodeJsonUtf8(ReadOnlySpan payload) { var output = new ArrayBufferWriter(); @@ -208,6 +232,20 @@ private static bool TryReadValueFromObject( return TryConvertValue(tag, payload.Slice(valueStart, valueLength), out value); } + private static bool TryReadValueFromObject( + ReadOnlySpan payload, + ReadOnlySpan pathSegmentUtf8, + out DbValue value) + { + if (!TryFindValue(payload, pathSegmentUtf8, out byte tag, out int valueStart, out int valueLength)) + { + value = default; + return false; + } + + return TryConvertValue(tag, payload.Slice(valueStart, valueLength), out value); + } + private static bool TryTextEqualsFromObject( ReadOnlySpan payload, int pathIndex, @@ -218,6 +256,15 @@ private static bool TryTextEqualsFromObject( TryTextEquals(tag, payload.Slice(valueStart, valueLength), expectedValue); } + private static bool TryTextEqualsFromObject( + ReadOnlySpan payload, + ReadOnlySpan pathSegmentUtf8, + string expectedValue) + { + return TryFindValue(payload, pathSegmentUtf8, out byte tag, out int valueStart, out int valueLength) && + TryTextEquals(tag, payload.Slice(valueStart, valueLength), expectedValue); + } + private static bool TryReadInt64FromObject( ReadOnlySpan payload, int pathIndex, @@ -236,6 +283,23 @@ private static bool TryReadInt64FromObject( return true; } + private static bool TryReadInt64FromObject( + ReadOnlySpan payload, + ReadOnlySpan pathSegmentUtf8, + out long value) + { + if (!TryFindValue(payload, pathSegmentUtf8, out byte tag, out int valueStart, out int valueLength) || + tag != IntegerTag || + valueLength != sizeof(long)) + { + value = 0; + return false; + } + + value = BinaryPrimitives.ReadInt64LittleEndian(payload.Slice(valueStart, valueLength)); + return true; + } + private static bool TryReadStringFromObject( ReadOnlySpan payload, int pathIndex, @@ -253,6 +317,55 @@ private static bool TryReadStringFromObject( return true; } + private static bool TryReadStringFromObject( + ReadOnlySpan payload, + ReadOnlySpan pathSegmentUtf8, + out string? value) + { + if (!TryFindValue(payload, pathSegmentUtf8, out byte tag, out int valueStart, out int valueLength) || + tag != StringTag) + { + value = null; + return false; + } + + value = Encoding.UTF8.GetString(payload.Slice(valueStart, valueLength)); + return true; + } + + private static bool TryReadStringUtf8FromObject( + ReadOnlySpan payload, + int pathIndex, + byte[][] pathSegmentsUtf8, + out ReadOnlySpan value) + { + if (!TryFindValue(payload, pathIndex, pathSegmentsUtf8, out byte tag, out int valueStart, out int valueLength) || + tag != StringTag) + { + value = default; + return false; + } + + value = payload.Slice(valueStart, valueLength); + return true; + } + + private static bool TryReadStringUtf8FromObject( + ReadOnlySpan payload, + ReadOnlySpan pathSegmentUtf8, + out ReadOnlySpan value) + { + if (!TryFindValue(payload, pathSegmentUtf8, out byte tag, out int valueStart, out int valueLength) || + tag != StringTag) + { + value = default; + return false; + } + + value = payload.Slice(valueStart, valueLength); + return true; + } + private static bool TryReadBooleanFromObject( ReadOnlySpan payload, int pathIndex, @@ -542,6 +655,14 @@ private static bool TryFindValue( return false; } + private static bool TryFindValue( + ReadOnlySpan payload, + ReadOnlySpan pathSegmentUtf8, + out byte tag, + out int valueStart, + out int valueLength) + => TryFindCurrentValue(payload, pathSegmentUtf8, out tag, out valueStart, out valueLength); + private static bool TryFindCurrentValue( ReadOnlySpan payload, int pathIndex, @@ -572,6 +693,35 @@ private static bool TryFindCurrentValue( return false; } + private static bool TryFindCurrentValue( + ReadOnlySpan payload, + ReadOnlySpan pathSegmentUtf8, + out byte tag, + out int valueStart, + out int valueLength) + { + int position = 0; + ulong fieldCount = ReadVarint(payload, ref position); + for (ulong i = 0; i < fieldCount; i++) + { + ReadOnlySpan fieldName = ReadLengthPrefixedBytes(payload, ref position); + if (!fieldName.SequenceEqual(pathSegmentUtf8)) + { + if (!TrySkipValue(payload, ref position)) + break; + + continue; + } + + return TryReadValuePayload(payload, ref position, out tag, out valueStart, out valueLength); + } + + tag = default; + valueStart = 0; + valueLength = 0; + return false; + } + private static bool TryReadValuePayload( ReadOnlySpan payload, ref int position, diff --git a/src/CSharpDB.Storage/Serialization/CollectionPayloadCodec.cs b/src/CSharpDB.Storage/Serialization/CollectionPayloadCodec.cs index 7b145e2a..32968c29 100644 --- a/src/CSharpDB.Storage/Serialization/CollectionPayloadCodec.cs +++ b/src/CSharpDB.Storage/Serialization/CollectionPayloadCodec.cs @@ -1,3 +1,5 @@ +using System.Buffers; +using System.Runtime.CompilerServices; using System.Text; using CSharpDB.Primitives; @@ -12,6 +14,9 @@ public static class CollectionPayloadCodec internal const byte LegacyJsonFormatMarker = 0xC1; internal const byte BinaryFormatMarker = 0xC2; internal const byte BinaryFormatVersion = 0x01; + internal const byte GeneratedRecordFormatMarker = 0xD0; + internal const byte GeneratedRecordFormatMagic = 0xF0; + internal const byte GeneratedRecordFormatVersion = 0x01; public static bool IsDirectPayload(ReadOnlySpan payload) => TryReadHeader(payload, out _); @@ -24,20 +29,19 @@ internal static bool TryReadValidatedHeader(ReadOnlySpan payload, out Head internal static bool TryReadFastHeader(ReadOnlySpan payload, out Header header) { - header = default; - - try + if (TryReadFastHeaderFields( + payload, + out CollectionPayloadFormat format, + out int keyStart, + out int keyByteCount, + out int documentStart)) { - if (TryReadLegacyHeader(payload, out header)) - return HasPlausibleJsonPayload(payload[header.DocumentStart..]); - - return TryReadBinaryHeader(payload, out header); - } - catch (Exception ex) when (ex is CSharpDbException or ArgumentOutOfRangeException or IndexOutOfRangeException or OverflowException) - { - header = default; - return false; + header = new Header(format, keyStart, keyByteCount, documentStart); + return true; } + + header = default; + return false; } internal static ReadOnlySpan GetKeyUtf8(ReadOnlySpan payload, Header header) @@ -104,6 +108,129 @@ public static byte[] EncodeBinary(string key, ReadOnlySpan documentPayload return payload; } + public static byte[] EncodeBinary( + string key, + int documentPayloadLength, + SpanAction writeDocumentPayload, + TState state) + { + ArgumentNullException.ThrowIfNull(key); + ArgumentNullException.ThrowIfNull(writeDocumentPayload); + ArgumentOutOfRangeException.ThrowIfNegative(documentPayloadLength); + + int keyByteCount = Encoding.UTF8.GetByteCount(key); + int keyLengthSize = Varint.SizeOf((ulong)keyByteCount); + byte[] payload = GC.AllocateUninitializedArray(2 + keyLengthSize + keyByteCount + documentPayloadLength); + payload[0] = BinaryFormatMarker; + payload[1] = BinaryFormatVersion; + + int position = 2; + position += Varint.Write(payload.AsSpan(position), (ulong)keyByteCount); + position += Encoding.UTF8.GetBytes(key.AsSpan(), payload.AsSpan(position, keyByteCount)); + writeDocumentPayload(payload.AsSpan(position, documentPayloadLength), state); + return payload; + } + + public static byte[] EncodeBinary( + string key, + int documentPayloadLength, + out int documentPayloadStart) + { + ArgumentNullException.ThrowIfNull(key); + ArgumentOutOfRangeException.ThrowIfNegative(documentPayloadLength); + + int keyByteCount = Encoding.UTF8.GetByteCount(key); + int keyLengthSize = Varint.SizeOf((ulong)keyByteCount); + byte[] payload = GC.AllocateUninitializedArray(2 + keyLengthSize + keyByteCount + documentPayloadLength); + payload[0] = BinaryFormatMarker; + payload[1] = BinaryFormatVersion; + + int position = 2; + position += Varint.Write(payload.AsSpan(position), (ulong)keyByteCount); + position += Encoding.UTF8.GetBytes(key.AsSpan(), payload.AsSpan(position, keyByteCount)); + documentPayloadStart = position; + return payload; + } + + public static bool TryDecodeDirectPayloadKey(ReadOnlySpan payload, out string key) + { + if (TryReadFastHeaderFields( + payload, + out _, + out int keyStart, + out int keyByteCount, + out _)) + { + key = Encoding.UTF8.GetString(payload.Slice(keyStart, keyByteCount)); + return true; + } + + key = string.Empty; + return false; + } + + public static bool TryDirectPayloadKeyEquals( + ReadOnlySpan payload, + ReadOnlySpan expectedKeyUtf8, + out bool equals) + { + if (TryReadFastHeaderFields( + payload, + out _, + out int keyStart, + out int keyByteCount, + out _)) + { + equals = payload.Slice(keyStart, keyByteCount).SequenceEqual(expectedKeyUtf8); + return true; + } + + equals = false; + return false; + } + + public static bool TryDirectPayloadKeyEquals( + ReadOnlySpan payload, + string expectedKey, + out bool equals) + { + ArgumentNullException.ThrowIfNull(expectedKey); + + if (TryReadFastHeaderFields( + payload, + out _, + out int keyStart, + out int keyByteCount, + out _)) + { + equals = KeyUtf8EqualsString(payload.Slice(keyStart, keyByteCount), expectedKey); + return true; + } + + equals = false; + return false; + } + + public static bool TryGetBinaryDocumentPayload( + ReadOnlySpan payload, + out ReadOnlySpan documentPayload) + { + if (TryReadFastHeaderFields( + payload, + out CollectionPayloadFormat format, + out _, + out _, + out int documentStart) && + format == CollectionPayloadFormat.Binary) + { + documentPayload = payload[documentStart..]; + return true; + } + + documentPayload = default; + return false; + } + public static bool KeyEquals(ReadOnlySpan payload, ReadOnlySpan expectedKeyUtf8) { var header = ReadHeader(payload); @@ -119,9 +246,18 @@ public static string DecodeKey(ReadOnlySpan payload) public static string DecodeJson(ReadOnlySpan payload) { var header = ReadHeader(payload); - return header.Format == CollectionPayloadFormat.LegacyJson - ? Encoding.UTF8.GetString(payload[header.DocumentStart..]) - : CollectionBinaryDocumentCodec.DecodeJson(payload[header.DocumentStart..]); + if (header.Format == CollectionPayloadFormat.LegacyJson) + return Encoding.UTF8.GetString(payload[header.DocumentStart..]); + + ReadOnlySpan documentPayload = payload[header.DocumentStart..]; + if (IsGeneratedRecordDocumentPayload(documentPayload)) + { + throw new CSharpDbException( + ErrorCode.CorruptDatabase, + "Generated record collection payloads require their generated collection codec for JSON conversion."); + } + + return CollectionBinaryDocumentCodec.DecodeJson(documentPayload); } public static bool JsonEquals(ReadOnlySpan payload, ReadOnlySpan expectedUtf8) @@ -130,7 +266,9 @@ public static bool JsonEquals(ReadOnlySpan payload, ReadOnlySpan exp if (header.Format == CollectionPayloadFormat.LegacyJson) return payload[header.DocumentStart..].SequenceEqual(expectedUtf8); - return CollectionBinaryDocumentCodec.EncodeJsonUtf8(payload[header.DocumentStart..]).AsSpan().SequenceEqual(expectedUtf8); + ReadOnlySpan documentPayload = payload[header.DocumentStart..]; + return !IsGeneratedRecordDocumentPayload(documentPayload) && + CollectionBinaryDocumentCodec.EncodeJsonUtf8(documentPayload).AsSpan().SequenceEqual(expectedUtf8); } public static ReadOnlySpan GetJsonUtf8(ReadOnlySpan payload) @@ -177,7 +315,7 @@ private static bool TryReadHeader(ReadOnlySpan payload, out Header header) return HasPlausibleJsonPayload(payload[header.DocumentStart..]); if (TryReadBinaryHeader(payload, out header)) - return CollectionBinaryDocumentCodec.IsValidDocument(payload[header.DocumentStart..]); + return IsKnownBinaryDocumentPayload(payload[header.DocumentStart..]); return false; } @@ -187,6 +325,61 @@ private static bool TryReadHeader(ReadOnlySpan payload, out Header header) } } + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static bool TryReadFastHeaderFields( + ReadOnlySpan payload, + out CollectionPayloadFormat format, + out int keyStart, + out int keyByteCount, + out int documentStart) + { + int lengthOffset; + if (payload.Length >= 3 && + payload[0] == BinaryFormatMarker && + payload[1] == BinaryFormatVersion) + { + format = CollectionPayloadFormat.Binary; + lengthOffset = 2; + } + else if (payload.Length >= 2 && payload[0] == LegacyJsonFormatMarker) + { + format = CollectionPayloadFormat.LegacyJson; + lengthOffset = 1; + } + else + { + return FailFastHeaderFields(out format, out keyStart, out keyByteCount, out documentStart); + } + + if (!TryReadInt32Varint(payload, lengthOffset, out keyByteCount, out int keyLengthBytes)) + return FailFastHeaderFields(out format, out keyStart, out keyByteCount, out documentStart); + + keyStart = lengthOffset + keyLengthBytes; + if (keyByteCount > payload.Length - keyStart) + return FailFastHeaderFields(out format, out keyStart, out keyByteCount, out documentStart); + + documentStart = keyStart + keyByteCount; + if (documentStart >= payload.Length) + return FailFastHeaderFields(out format, out keyStart, out keyByteCount, out documentStart); + + return format != CollectionPayloadFormat.LegacyJson || + HasPlausibleJsonPayload(payload[documentStart..]); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static bool FailFastHeaderFields( + out CollectionPayloadFormat format, + out int keyStart, + out int keyByteCount, + out int documentStart) + { + format = default; + keyStart = 0; + keyByteCount = 0; + documentStart = 0; + return false; + } + private static bool TryReadLegacyHeader(ReadOnlySpan payload, out Header header) { header = default; @@ -219,6 +412,16 @@ private static bool TryReadBinaryHeader(ReadOnlySpan payload, out Header h return true; } + private static bool IsKnownBinaryDocumentPayload(ReadOnlySpan payload) + => IsGeneratedRecordDocumentPayload(payload) || + CollectionBinaryDocumentCodec.IsValidDocument(payload); + + private static bool IsGeneratedRecordDocumentPayload(ReadOnlySpan payload) + => payload.Length >= 3 && + payload[0] == GeneratedRecordFormatMarker && + payload[1] == GeneratedRecordFormatMagic && + payload[2] == GeneratedRecordFormatVersion; + private static bool HasPlausibleJsonPayload(ReadOnlySpan jsonUtf8) { for (int i = 0; i < jsonUtf8.Length; i++) @@ -240,6 +443,89 @@ private static bool HasPlausibleJsonPayload(ReadOnlySpan jsonUtf8) return false; } + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static bool TryReadInt32Varint( + ReadOnlySpan payload, + int offset, + out int value, + out int bytesRead) + { + ulong result = 0; + int shift = 0; + + for (int i = 0; i < 5; i++) + { + int index = offset + i; + if ((uint)index >= (uint)payload.Length) + break; + + byte b = payload[index]; + result |= (ulong)(b & 0x7F) << shift; + if ((b & 0x80) == 0) + { + if (result > int.MaxValue) + break; + + value = (int)result; + bytesRead = i + 1; + return true; + } + + shift += 7; + } + + value = 0; + bytesRead = 0; + return false; + } + + private static bool KeyUtf8EqualsString(ReadOnlySpan keyUtf8, string expectedKey) + { + if (keyUtf8.Length == expectedKey.Length) + { + bool ascii = true; + for (int i = 0; i < expectedKey.Length; i++) + { + char c = expectedKey[i]; + if (c > 0x7F) + { + ascii = false; + break; + } + + if (keyUtf8[i] != (byte)c) + return false; + } + + if (ascii) + return true; + } + + int byteCount = Encoding.UTF8.GetByteCount(expectedKey); + if (byteCount != keyUtf8.Length) + return false; + + const int StackallocKeyThreshold = 256; + byte[]? rented = null; + Span expectedKeyUtf8 = byteCount <= StackallocKeyThreshold + ? stackalloc byte[StackallocKeyThreshold] + : (rented = ArrayPool.Shared.Rent(byteCount)); + + try + { + int written = Encoding.UTF8.GetBytes(expectedKey.AsSpan(), expectedKeyUtf8); + return keyUtf8.SequenceEqual(expectedKeyUtf8[..written]); + } + finally + { + if (rented is not null) + { + expectedKeyUtf8[..byteCount].Clear(); + ArrayPool.Shared.Return(rented); + } + } + } + internal enum CollectionPayloadFormat : byte { LegacyJson = 1, diff --git a/tests/CSharpDB.Benchmarks/CSharpDB.Benchmarks.csproj b/tests/CSharpDB.Benchmarks/CSharpDB.Benchmarks.csproj index 1efe096a..fab30757 100644 --- a/tests/CSharpDB.Benchmarks/CSharpDB.Benchmarks.csproj +++ b/tests/CSharpDB.Benchmarks/CSharpDB.Benchmarks.csproj @@ -23,6 +23,7 @@ + diff --git a/tests/CSharpDB.Benchmarks/HISTORY.md b/tests/CSharpDB.Benchmarks/HISTORY.md index 39c2713a..5818707b 100644 --- a/tests/CSharpDB.Benchmarks/HISTORY.md +++ b/tests/CSharpDB.Benchmarks/HISTORY.md @@ -7,7 +7,135 @@ This file preserves the benchmark release logs, failed-run notes, investigation Performance benchmarks for the CSharpDB embedded database engine. -The current main README is promoted from the April 21, 2026 release-core run and the April 22, 2026 UTC guardrail compare. Earlier April 21 release-prep failures are retained below as investigation history, not as the current published state. +The current main README is promoted from the April 26, 2026 release-core run and the April 27, 2026 UTC guardrail compare. Earlier release-prep failures are retained below as investigation history, not as the current published state. + +## April 26-27, 2026 Release-Core Promotion + +The stable README was refreshed after the collection binary payload work from a balanced release-core run, not from the diagnostic collection-only sweep. + +```powershell +dotnet run -c Release --project .\tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --release-core --repeat 3 --repro +pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Run-Perf-Guardrails.ps1 -Mode release +pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Compare-Baseline.ps1 ` + -ThresholdsPath .\tests\CSharpDB.Benchmarks\perf-thresholds.json ` + -CurrentMicroResultsDir .\tests\CSharpDB.Benchmarks\results\.tmp-current-micro-run ` + -ReportPath .\tests\CSharpDB.Benchmarks\results\perf-guardrails-last.md +pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Update-BenchmarkReadme.ps1 -RunManifest .\tests\CSharpDB.Benchmarks\release-core-manifest.json +``` + +| Item | Result | +|------|--------| +| Release-core status | `completed` | +| Release-core completion | `Sunday, April 26, 2026 about 4:03 PM PT` | +| Release-core duration | `about 68 minutes` | +| Initial guardrail wrapper compare | `Compared 185 rows against baseline. PASS=174, WARN=0, SKIP=0, FAIL=11` | +| Final guardrail compare | `Compared 185 rows against baseline. PASS=185, WARN=0, SKIP=0, FAIL=0` | +| Final guardrail report | `tests/CSharpDB.Benchmarks/results/perf-guardrails-last.md` | +| Final compare timestamp | `2026-04-27 01:18:39Z` | +| Runner | `Intel i9-11900K, 16 logical cores, Windows 10.0.26300, .NET SDK 10.0.203, .NET runtime 10.0.7` | +| Commit | `b7cb52ee2c30f31538e96480b6d055ff52439c26 plus uncommitted collection binary payload and benchmark updates` | + +The first release guardrail wrapper completed benchmark collection but failed the compare with volatile samples in `CollationIndexBenchmarks`, `CollectionIndexBenchmarks`, and `CompositeIndexBenchmarks`. A focused retry refreshed only those three micro CSVs, then the official compare passed with `PASS=185, WARN=0, SKIP=0, FAIL=0`. The release-core source artifacts were not replaced by targeted micro runs. + +| Artifact | Path | +|----------|------| +| Durable master comparison | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/master-table-20260426-215529-median-of-3.csv` | +| Durable SQL batching | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/durable-sql-batching-20260426-221413-median-of-3.csv` | +| Concurrent durable write | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/concurrent-write-diagnostics-20260426-223659-median-of-3.csv` | +| Hybrid storage mode | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-storage-mode-20260426-224331-median-of-3.csv` | +| Hybrid hot-set read | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-hot-set-read-20260426-225908-median-of-3.csv` | +| Hybrid cold open | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-cold-open-20260426-225949-median-of-3.csv` | +| SQLite comparison | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/sqlite-compare-20260426-230045-median-of-3.csv` | + +## April 26, 2026 Collection Binary Payload Fast-Path Investigation + +This was a focused diagnostic and recovery run for collection binary payload work. It was not a release-core promotion and does not update the stable README scorecard. + +The work investigated regressions seen after source-generated collection fast paths, generated record payload encoding, and targeted UTF-8 span plumbing for text index/read/compare paths. A same-machine HEAD baseline worktree was created at `.tmp/baseline-head-collection-20260426`, and the full `*Collection*` benchmark filter was run against both HEAD and the working tree. + +Initial collection comparison: + +| Item | Result | +|------|--------| +| Current rows | `70` | +| Matched HEAD baseline rows | `60` | +| Faster matched rows | `50` | +| Slower matched rows | `10` | +| Median matched speedup | `+4.1%` | +| Mean matched speedup | `+4.8%` | +| Comparison CSV | `.tmp/collection-benchmark-comparison-current-vs-head-20260426.csv` | + +The most actionable regressions were small direct payload/header and unbound field-reader paths: + +| Benchmark | HEAD baseline | Before recovery | Notes | +|-----------|--------------:|----------------:|-------| +| Collection field read (missing field) | `201.81 ns` | `223.22 ns` | Allocations unchanged in the first comparison; path shared header parsing and per-call property-name metadata. | +| Collection decode (direct payload) | `317.90 ns` | `333.80 ns` | Error bars overlapped, but the direct payload decode path parsed the same header twice through `DecodeKey` and `DecodeDocument`. | +| Collection cold get (file-backed, cache-pressured) | `51.588 us` | `57.036 us` | Treated as noisy file/cache/async behavior, not directly tied to the codec path. | + +Recovery changes: + +- `CollectionDocumentCodec.Decode` now parses direct payload headers once and decodes the key/document from that single header. +- `CollectionIndexedFieldReader` now uses stack/rented UTF-8 spans for top-level string-property reads instead of allocating a `byte[]` plus `byte[][]` per call. +- `CollectionBinaryDocumentCodec` now has single-segment `ReadOnlySpan` lookup overloads for top-level binary document field access. +- `CollectionPayloadCodec` fast header parsing was adjusted for the common binary payload marker first. + +Focused recovery benchmark comparison: + +| Benchmark | HEAD baseline | Before recovery | After recovery | After vs before | After vs baseline | Allocation after | +|-----------|--------------:|----------------:|---------------:|----------------:|------------------:|-----------------:| +| Collection field read (missing field) | `201.81 ns` | `223.22 ns` | `136.73 ns` | `+38.7%` | `+32.2%` | `0 B` | +| Collection decode (direct payload) | `317.90 ns` | `333.80 ns` | `155.20 ns` | `+53.5%` | `+51.2%` | `328 B` | +| Collection field read (early field) | `120.07 ns` | `108.60 ns` | `49.77 ns` | `+54.2%` | `+58.5%` | `48 B` | +| Collection field read (middle field) | `107.15 ns` | `95.52 ns` | `67.96 ns` | `+28.9%` | `+36.6%` | `0 B` | +| Collection field read (late field) | `158.71 ns` | `146.20 ns` | `112.27 ns` | `+23.2%` | `+29.3%` | `0 B` | +| Collection field compare (late text field) | `187.12 ns` | `159.55 ns` | `97.60 ns` | `+38.8%` | `+47.8%` | `0 B` | +| Collection field compare (late text field, bound accessor) | `160.29 ns` | `128.03 ns` | `92.83 ns` | `+27.5%` | `+42.1%` | `0 B` | + +Collection path-index follow-up: + +The April 7 path-index spot-check table remains a historical snapshot and was not overwritten. After the recovery changes, the focused `*CollectionIndexBenchmarks*` filter was rerun on the final working tree to cover the same path-index area. This is still diagnostic data, not a release-core promotion; read it against the April 26 same-machine HEAD artifact, not as a replacement for the April 7 dated snapshot. + +| Benchmark | April 26 HEAD baseline | April 26 final | Final vs baseline | Allocation final | +|-----------|-----------------------:|---------------:|------------------:|-----------------:| +| Collection FindByIndex nested path equality | `2,408.209 us` | `1,116.247 us` | `+53.6%` | `754.83 KB` | +| Collection FindByPath nested path equality | `2,485.552 us` | `1,096.847 us` | `+55.9%` | `754.83 KB` | +| Collection FindByIndex array path equality | `2,720.115 us` | `1,303.300 us` | `+52.1%` | `1254.25 KB` | +| Collection FindByPath array path equality | `2,706.758 us` | `1,310.859 us` | `+51.6%` | `1254.25 KB` | +| Collection FindByPath nested array path equality | `4,211.142 us` | `2,176.966 us` | `+48.3%` | `1742.67 KB` | +| Collection FindByPath integer range | `1,017.047 us` | `587.753 us` | `+42.2%` | `307.28 KB` | +| Collection FindByPath text range | `1,103.269 us` | `632.086 us` | `+42.7%` | `499.5 KB` | +| Collection FindByPath Guid equality | `12.388 us` | `5.034 us` | `+59.4%` | `15.26 KB` | +| Collection FindByPath DateOnly range | `2,176.684 us` | `1,093.622 us` | `+49.8%` | `835.63 KB` | + +Artifacts: + +| Artifact | Path | +|----------|------| +| Preserved April 26 HEAD benchmark reports | `.tmp/baseline-head-collection-20260426-results` | +| Final collection index rerun report | `BenchmarkDotNet.Artifacts/results/CSharpDB.Benchmarks.Micro.CollectionIndexBenchmarks-report.csv` | + +Commands run: + +```powershell +dotnet run -c Release --project tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --filter "*Collection*" +dotnet run -c Release --project tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --filter "*CollectionFieldExtractionBenchmarks*" +dotnet run -c Release --project tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --filter "*CollectionPayloadBenchmarks*" +dotnet run -c Release --project tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --filter "*CollectionIndexBenchmarks*" +dotnet build CSharpDB.slnx -c Release --no-restore +dotnet test CSharpDB.slnx -c Release --no-build -m:1 -- RunConfiguration.DisableParallelization=true +``` + +Verification: + +| Check | Result | +|-------|--------| +| Release build | `passed` | +| Non-parallel unit tests | `passed: 1,652 tests` | +| Focused field extraction benchmarks | `completed` | +| Focused payload benchmarks | `completed` | +| Focused collection index benchmarks | `completed` | +| Recovery comparison CSV | `.tmp/collection-recovery-comparison-20260426.csv` | ## April 21-22, 2026 Release-Core Promotion @@ -788,6 +916,8 @@ The important result for async I/O batching is still the staged-vs-direct compar ### Collection Path Index Spot Checks (April 7, 2026) +This April 7 table is retained unchanged as a dated historical spot check. See the April 26 collection binary payload investigation above for the newer diagnostic path-index rerun; the two sections should not be treated as a release-to-release promotion comparison. + | Metric | Mean | Allocated | Notes | |--------|------|-----------|-------| | Collection `FindByIndex` nested path equality (`$.address.city`) | 995.823 us | 820.75 KB | Public string-path index lookup over a nested scalar path with many matches | diff --git a/tests/CSharpDB.Benchmarks/Micro/GeneratedCollectionCodecBenchmarks.cs b/tests/CSharpDB.Benchmarks/Micro/GeneratedCollectionCodecBenchmarks.cs new file mode 100644 index 00000000..2b6f8308 --- /dev/null +++ b/tests/CSharpDB.Benchmarks/Micro/GeneratedCollectionCodecBenchmarks.cs @@ -0,0 +1,378 @@ +using System.Buffers; +using System.Diagnostics.CodeAnalysis; +using System.Text; +using System.Text.Json; +using System.Text.Json.Serialization; +using BenchmarkDotNet.Attributes; +using CSharpDB.Benchmarks.Infrastructure; +using CSharpDB.Engine; +using CSharpDB.Primitives; +using CSharpDB.Storage.Serialization; + +namespace CSharpDB.Benchmarks.Micro; + +[MemoryDiagnoser] +[Config(typeof(CollectionInProcessBenchmarkConfig))] +public class GeneratedCollectionEncodeBenchmarks : GeneratedCollectionCodecBenchmarkBase +{ + [Benchmark(Baseline = true, Description = "Generated collection encode (source-gen JSON payload)")] + public byte[] Encode_SourceGeneratedJsonPayload() + => JsonCodec.Encode(Key, JsonDocument); + + [Benchmark(Description = "Generated collection encode (source-gen binary payload)")] + public byte[] Encode_SourceGeneratedBinaryPayload() + => BinaryCodec.Encode(Key, BinaryDocument); +} + +[MemoryDiagnoser] +[Config(typeof(CollectionInProcessBenchmarkConfig))] +public class GeneratedCollectionIndexedFieldBenchmarks : GeneratedCollectionCodecBenchmarkBase +{ + private long _sink; + + [Benchmark(Baseline = true, Description = "Generated collection indexed field read (source-gen JSON payload)")] + public void ReadIndexedField_SourceGeneratedJsonPayload() + { + if (CollectionIndexedFieldReader.TryReadValue(JsonPayload, ScoreAccessor, out var value) && + value.Type == DbType.Integer) + { + _sink = value.AsInteger; + } + else + { + _sink = -1; + } + } + + [Benchmark(Description = "Generated collection indexed field read (source-gen binary payload)")] + public void ReadIndexedField_SourceGeneratedBinaryPayload() + { + if (ScoreField.TryReadPayloadInt64(BinaryPayload, out long score)) + { + _sink = score; + } + else + { + _sink = -1; + } + } +} + +[MemoryDiagnoser] +[Config(typeof(CollectionInProcessBenchmarkConfig))] +public class GeneratedCollectionTextFieldBenchmarks : GeneratedCollectionCodecBenchmarkBase +{ + private int _sink; + + [Benchmark(Baseline = true, Description = "Generated collection text field read (source-gen JSON payload)")] + public void ReadTextField_SourceGeneratedJsonPayload() + { + if (CollectionIndexedFieldReader.TryReadString(JsonPayload, EmailAccessor, out string? email) && + email is not null) + { + _sink = email.Length; + } + else + { + _sink = -1; + } + } + + [Benchmark(Description = "Generated collection text field read (source-gen binary UTF-8 payload)")] + public void ReadTextField_SourceGeneratedBinaryPayload() + { + if (EmailField.TryReadPayloadStringUtf8(BinaryPayload, out ReadOnlySpan emailUtf8)) + { + _sink = emailUtf8.Length; + } + else + { + _sink = -1; + } + } +} + +[MemoryDiagnoser] +[Config(typeof(CollectionInProcessBenchmarkConfig))] +public class GeneratedCollectionPayloadKeyBenchmarks : GeneratedCollectionCodecBenchmarkBase +{ + [Benchmark(Baseline = true, Description = "Generated collection key match (source-gen JSON payload)")] + public bool PayloadMatchesKey_SourceGeneratedJsonPayload() + => JsonCodec.PayloadMatchesKey(JsonPayload, Key); + + [Benchmark(Description = "Generated collection key match (source-gen binary payload)")] + public bool PayloadMatchesKey_SourceGeneratedBinaryPayload() + => BinaryCodec.PayloadMatchesKey(BinaryPayload, Key); +} + +[MemoryDiagnoser] +[Config(typeof(CollectionInProcessBenchmarkConfig))] +public class GeneratedCollectionDecodeBenchmarks : GeneratedCollectionCodecBenchmarkBase +{ + [Benchmark(Baseline = true, Description = "Generated collection decode (source-gen JSON payload)")] + public JsonSourceBenchDoc Decode_SourceGeneratedJsonPayload() + => JsonCodec.DecodeDocument(JsonPayload); + + [Benchmark(Description = "Generated collection decode (source-gen binary payload)")] + public GeneratedBinaryBenchDoc Decode_SourceGeneratedBinaryPayload() + => BinaryCodec.DecodeDocument(BinaryPayload); +} + +public abstract class GeneratedCollectionCodecBenchmarkBase +{ + protected const string Key = "doc:42"; + + private CollectionModelRegistration? _jsonRegistration; + + private protected CollectionDocumentCodec BinaryCodec { get; private set; } = null!; + + private protected CollectionDocumentCodec JsonCodec { get; private set; } = null!; + + private protected GeneratedBinaryBenchDoc BinaryDocument { get; private set; } = null!; + + private protected JsonSourceBenchDoc JsonDocument { get; private set; } = null!; + + private protected byte[] BinaryPayload { get; private set; } = null!; + + private protected byte[] JsonPayload { get; private set; } = null!; + + private protected CollectionFieldAccessor ScoreAccessor { get; private set; } = null!; + + private protected CollectionFieldAccessor EmailAccessor { get; private set; } = null!; + + private protected CollectionField ScoreField { get; private set; } = null!; + + private protected CollectionField EmailField { get; private set; } = null!; + + [GlobalSetup] + public void GlobalSetup() + { + _jsonRegistration = CollectionModelRegistry.Register(new JsonSourceBenchDocCollectionModel()); + + BinaryCodec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + JsonCodec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + + BinaryDocument = new GeneratedBinaryBenchDoc( + "Alice Example", + 37, + "Alpha", + "alice@example.com", + 912, + "west", + "active", + 7, + "gold", + 3, + new GeneratedBinaryBenchProfile("enterprise", new GeneratedBinaryBenchAddress("Seattle", 98101))); + JsonDocument = new JsonSourceBenchDoc( + BinaryDocument.Name, + BinaryDocument.Age, + BinaryDocument.Category, + BinaryDocument.Email, + BinaryDocument.Score, + BinaryDocument.Region, + BinaryDocument.Status, + BinaryDocument.Revision, + BinaryDocument.Tier, + BinaryDocument.Flags, + new JsonSourceBenchProfile( + BinaryDocument.Profile.Segment, + new JsonSourceBenchAddress( + BinaryDocument.Profile.Address.City, + BinaryDocument.Profile.Address.ZipCode))); + + BinaryPayload = BinaryCodec.Encode(Key, BinaryDocument); + JsonPayload = JsonCodec.Encode(Key, JsonDocument); + ScoreAccessor = CollectionFieldAccessor.FromFieldPath("score"); + EmailAccessor = CollectionFieldAccessor.FromFieldPath("email"); + ScoreField = GeneratedBinaryBenchDoc.Collection.Score; + EmailField = GeneratedBinaryBenchDoc.Collection.Email; + + if (!CollectionPayloadCodec.IsBinaryPayload(BinaryPayload)) + throw new InvalidOperationException("Generated binary benchmark payload did not use the generated binary path."); + + if (!CollectionPayloadCodec.IsDirectPayload(JsonPayload) || CollectionPayloadCodec.IsBinaryPayload(JsonPayload)) + throw new InvalidOperationException("Generated JSON benchmark payload did not use the direct JSON path."); + } + + [GlobalCleanup] + public void GlobalCleanup() + => _jsonRegistration?.Dispose(); +} + +[CollectionModel(typeof(GeneratedCollectionCodecJsonContext))] +public sealed partial record GeneratedBinaryBenchDoc( + string Name, + int Age, + string Category, + string Email, + int Score, + string Region, + string Status, + int Revision, + string Tier, + int Flags, + GeneratedBinaryBenchProfile Profile); + +public sealed partial record GeneratedBinaryBenchProfile(string Segment, GeneratedBinaryBenchAddress Address); + +public sealed partial record GeneratedBinaryBenchAddress(string City, int ZipCode); + +public sealed record JsonSourceBenchDoc( + string Name, + int Age, + string Category, + string Email, + int Score, + string Region, + string Status, + int Revision, + string Tier, + int Flags, + JsonSourceBenchProfile Profile); + +public sealed record JsonSourceBenchProfile(string Segment, JsonSourceBenchAddress Address); + +public sealed record JsonSourceBenchAddress(string City, int ZipCode); + +internal sealed class JsonSourceBenchDocCollectionModel : ICollectionModel +{ + public ICollectionDocumentCodec CreateCodec(IRecordSerializer recordSerializer) + => new JsonSourceBenchDocCollectionCodec(recordSerializer); + + public bool TryGetField( + string fieldPath, + [NotNullWhen(true)] out CollectionField? field) + { + field = null; + return false; + } +} + +internal sealed class JsonSourceBenchDocCollectionCodec : ICollectionDocumentCodec +{ + private const int StackallocKeyThreshold = 256; + + private readonly IRecordSerializer _recordSerializer; + private readonly bool _usesDirectPayloadFormat; + + public JsonSourceBenchDocCollectionCodec(IRecordSerializer recordSerializer) + { + _recordSerializer = recordSerializer ?? throw new ArgumentNullException(nameof(recordSerializer)); + _usesDirectPayloadFormat = recordSerializer is DefaultRecordSerializer; + } + + public byte[] Encode(string key, JsonSourceBenchDoc document) + { + ArgumentNullException.ThrowIfNull(key); + ArgumentNullException.ThrowIfNull(document); + + if (_usesDirectPayloadFormat) + { + byte[] jsonUtf8 = JsonSerializer.SerializeToUtf8Bytes( + document, + GeneratedCollectionCodecJsonContext.Default.JsonSourceBenchDoc); + return CollectionPayloadCodec.Encode(key, jsonUtf8); + } + + string json = JsonSerializer.Serialize( + document, + GeneratedCollectionCodecJsonContext.Default.JsonSourceBenchDoc); + return _recordSerializer.Encode( + [ + DbValue.FromText(key), + DbValue.FromText(json), + ]); + } + + public (string Key, JsonSourceBenchDoc Document) Decode(ReadOnlySpan payload) + => (DecodeKey(payload), DecodeDocument(payload)); + + public JsonSourceBenchDoc DecodeDocument(ReadOnlySpan payload) + { + if (_usesDirectPayloadFormat && CollectionPayloadCodec.IsDirectPayload(payload)) + { + if (!CollectionPayloadCodec.IsBinaryPayload(payload)) + { + return JsonSerializer.Deserialize( + CollectionPayloadCodec.GetJsonUtf8(payload), + GeneratedCollectionCodecJsonContext.Default.JsonSourceBenchDoc) + ?? throw new InvalidOperationException("Generated JSON benchmark payload deserialized to null."); + } + + string json = CollectionPayloadCodec.DecodeJson(payload); + return JsonSerializer.Deserialize( + json, + GeneratedCollectionCodecJsonContext.Default.JsonSourceBenchDoc) + ?? throw new InvalidOperationException("Generated JSON benchmark payload deserialized to null."); + } + + var values = _recordSerializer.Decode(payload); + return JsonSerializer.Deserialize( + values[1].AsText, + GeneratedCollectionCodecJsonContext.Default.JsonSourceBenchDoc) + ?? throw new InvalidOperationException("Generated JSON benchmark payload deserialized to null."); + } + + public string DecodeKey(ReadOnlySpan payload) + { + if (_usesDirectPayloadFormat && CollectionPayloadCodec.TryDecodeDirectPayloadKey(payload, out string key)) + return key; + + var values = _recordSerializer.DecodeUpTo(payload, 0); + return values[0].AsText; + } + + public bool TryDecodeDocumentForKey( + ReadOnlySpan payload, + string expectedKey, + [NotNullWhen(true)] out JsonSourceBenchDoc? document) + { + if (!PayloadMatchesKey(payload, expectedKey)) + { + document = null; + return false; + } + + document = DecodeDocument(payload); + return true; + } + + public bool PayloadMatchesKey(ReadOnlySpan payload, string expectedKey) + { + ArgumentNullException.ThrowIfNull(expectedKey); + + if (_usesDirectPayloadFormat && CollectionPayloadCodec.TryDirectPayloadKeyEquals(payload, expectedKey, out bool directEquals)) + return directEquals; + + int byteCount = Encoding.UTF8.GetByteCount(expectedKey); + byte[]? rented = null; + Span utf8 = byteCount <= StackallocKeyThreshold + ? stackalloc byte[StackallocKeyThreshold] + : (rented = ArrayPool.Shared.Rent(byteCount)); + + try + { + int written = Encoding.UTF8.GetBytes(expectedKey.AsSpan(), utf8); + ReadOnlySpan expectedKeyUtf8 = utf8[..written]; + + if (_recordSerializer.TryColumnTextEquals(payload, 0, expectedKeyUtf8, out bool equals)) + return equals; + + return DecodeKey(payload) == expectedKey; + } + finally + { + if (rented is not null) + { + utf8[..byteCount].Clear(); + ArrayPool.Shared.Return(rented); + } + } + } +} + +[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)] +[JsonSerializable(typeof(GeneratedBinaryBenchDoc))] +[JsonSerializable(typeof(JsonSourceBenchDoc))] +internal sealed partial class GeneratedCollectionCodecJsonContext : JsonSerializerContext; diff --git a/tests/CSharpDB.Benchmarks/README.md b/tests/CSharpDB.Benchmarks/README.md index dd10a534..1a2d3c0e 100644 --- a/tests/CSharpDB.Benchmarks/README.md +++ b/tests/CSharpDB.Benchmarks/README.md @@ -26,7 +26,7 @@ Current release health: |---|---| | Latest release guardrail | `PASS` | | Latest compare | `PASS=185, WARN=0, SKIP=0, FAIL=0` | -| Promotion state | Current published tables are promoted from the April 21, 2026 release-core suite | +| Promotion state | Current published tables are promoted from the April 26, 2026 release-core suite | | Durability default | CSharpDB values are durable unless a row explicitly says otherwise | ## Core Performance Scorecard @@ -40,25 +40,25 @@ The generated block below contains the scorecard first, then the current core re | Field | Value | |---|---| -| Published snapshot | April 21, 2026 release-core snapshot | -| Run date | Release-core artifacts captured April 21, 2026; release guardrail compare captured April 22, 2026 UTC | -| Promotion status | Promoted after release-core suite and release guardrail compare passed | +| Published snapshot | April 26, 2026 release-core snapshot | +| Run date | Release-core artifacts captured April 26, 2026 PT; release guardrail compare captured April 27, 2026 UTC | +| Promotion status | Promoted after release-core suite and release guardrail compare passed; focused micro retry replaced volatile guardrail samples | | Latest release guardrail | PASS=185, WARN=0, SKIP=0, FAIL=0 | -| Runner | Intel i9-11900K, 16 logical cores, Windows 10.0.26300, .NET SDK 10.0.202, .NET runtime 10.0.6 | +| Runner | Intel i9-11900K, 16 logical cores, Windows 10.0.26300, .NET SDK 10.0.203, .NET runtime 10.0.7 | | Repro mode | priority=High, affinity=0xFF when captured with --repro | -| Commit | 4e12bbe6eed58ffa7b7cf85371093008c292ec13 plus uncommitted release-prep fixes | +| Commit | b7cb52ee2c30f31538e96480b6d055ff52439c26 plus uncommitted collection binary payload and benchmark updates | ### Approved Source Artifacts | Artifact | Command | Source CSV | |---|---|---| -| `master` | `--master-table --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/master-table-20260421-212338-median-of-3.csv` | -| `batching` | `--durable-sql-batching --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/durable-sql-batching-20260421-214227-median-of-3.csv` | -| `concurrent` | `--concurrent-write-diagnostics --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/concurrent-write-diagnostics-20260421-220505-median-of-3.csv` | -| `storage` | `--hybrid-storage-mode --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-storage-mode-20260421-221135-median-of-3.csv` | -| `hotset` | `--hybrid-hot-set-read --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-hot-set-read-20260421-222712-median-of-3.csv` | -| `coldopen` | `--hybrid-cold-open --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-cold-open-20260421-222743-median-of-3.csv` | -| `sqlite` | `--sqlite-compare --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/sqlite-compare-20260421-222824-median-of-3.csv` | +| `master` | `--master-table --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/master-table-20260426-215529-median-of-3.csv` | +| `batching` | `--durable-sql-batching --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/durable-sql-batching-20260426-221413-median-of-3.csv` | +| `concurrent` | `--concurrent-write-diagnostics --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/concurrent-write-diagnostics-20260426-223659-median-of-3.csv` | +| `storage` | `--hybrid-storage-mode --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-storage-mode-20260426-224331-median-of-3.csv` | +| `hotset` | `--hybrid-hot-set-read --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-hot-set-read-20260426-225908-median-of-3.csv` | +| `coldopen` | `--hybrid-cold-open --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-cold-open-20260426-225949-median-of-3.csv` | +| `sqlite` | `--sqlite-compare --repeat 3 --repro` | `tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/sqlite-compare-20260426-230045-median-of-3.csv` | ### Scorecard @@ -66,16 +66,16 @@ These are the headline rows readers should use first. Detailed tables below map | Area | Metric | Result | Source | |---|---|---|---| -| Release health | Latest guardrail compare | PASS: PASS=185, FAIL=0 | `Run-Perf-Guardrails.ps1 -Mode release` | -| SQL durable write | Single INSERT | 279.4 ops/sec | `master` | -| SQL durable write | Batch x100 | 26.71K rows/sec | `master` | -| SQL hot read | Point lookup | 1.33M ops/sec | `master` | -| SQL concurrent read | 8 readers, reused snapshots x32 | 10.77M COUNT(*) ops/sec | `master` | -| Collection hot read | Point Get | 1.67M ops/sec | `master` | -| Single-writer ingest | InsertBatch B1000 | 204.03K rows/sec | `batching` | -| Concurrent durable write | W8, 250us commit window | 1.04K commits/sec | `concurrent` | -| Resident hot set | Hybrid hot-set SQL burst | 535.39K ops/sec | `hotset` | -| Local SQLite reference | SQLite WAL+FULL B1000 | 192.06K rows/sec | `sqlite` | +| Release health | Latest guardrail compare | PASS: PASS=185, FAIL=0 | `Compare-Baseline.ps1` after focused micro retry | +| SQL durable write | Single INSERT | 450.4 ops/sec | `master` | +| SQL durable write | Batch x100 | 41.88K rows/sec | `master` | +| SQL hot read | Point lookup | 1.27M ops/sec | `master` | +| SQL concurrent read | 8 readers, reused snapshots x32 | 9.97M COUNT(*) ops/sec | `master` | +| Collection hot read | Point Get | 1.60M ops/sec | `master` | +| Single-writer ingest | InsertBatch B1000 | 233.06K rows/sec | `batching` | +| Concurrent durable write | W8, 250us commit window | 891.0 commits/sec | `concurrent` | +| Resident hot set | Hybrid hot-set SQL burst | 311.62K ops/sec | `hotset` | +| Local SQLite reference | SQLite WAL+FULL B1000 | 203.30K rows/sec | `sqlite` | ## Current Core Results @@ -85,21 +85,21 @@ These detailed tables are generated from the approved source artifacts listed ab | Surface | Single write | Batch x100 | Point read | Concurrent read | |---|---|---|---|---| -| SQL file-backed | 279.4 ops/sec | 26.71K rows/sec | 1.33M ops/sec | 10.77M COUNT(*) ops/sec | -| SQL hybrid incremental-durable | 277.3 ops/sec | 26.19K rows/sec | 1.37M ops/sec | 10.35M COUNT(*) ops/sec | -| SQL in-memory | 212.04K ops/sec | 889.37K rows/sec | 1.35M ops/sec | 10.71M COUNT(*) ops/sec | -| Collection file-backed | 273.5 ops/sec | 25.92K docs/sec | 1.67M ops/sec | - | -| Collection hybrid incremental-durable | 264.8 ops/sec | 25.02K docs/sec | 1.66M ops/sec | - | -| Collection in-memory | 222.67K ops/sec | 912.56K docs/sec | 1.59M ops/sec | - | +| SQL file-backed | 450.4 ops/sec | 41.88K rows/sec | 1.27M ops/sec | 9.97M COUNT(*) ops/sec | +| SQL hybrid incremental-durable | 449.3 ops/sec | 41.77K rows/sec | 1.29M ops/sec | 10.27M COUNT(*) ops/sec | +| SQL in-memory | 194.98K ops/sec | 708.75K rows/sec | 1.24M ops/sec | 10.09M COUNT(*) ops/sec | +| Collection file-backed | 447.3 ops/sec | 42.28K docs/sec | 1.60M ops/sec | - | +| Collection hybrid incremental-durable | 450.8 ops/sec | 42.34K docs/sec | 1.66M ops/sec | - | +| Collection in-memory | 205.25K ops/sec | 872.89K docs/sec | 1.59M ops/sec | - | ### Single-Writer Durable Ingest | Batch shape | Rows/sec | P50 | P99 | |---|---|---|---| -| InsertBatch B1 | 277.0 rows/sec | 3.4365 ms | 7.7126 ms | -| InsertBatch B100 | 25.87K rows/sec | 3.6300 ms | 8.3708 ms | -| InsertBatch B1000 | 204.03K rows/sec | 4.1034 ms | 9.5599 ms | -| InsertBatch B10000 | 798.25K rows/sec | 8.7019 ms | 119.0664 ms | +| InsertBatch B1 | 358.4 rows/sec | 2.7074 ms | 4.1941 ms | +| InsertBatch B100 | 33.92K rows/sec | 2.7875 ms | 4.2581 ms | +| InsertBatch B1000 | 233.06K rows/sec | 3.5123 ms | 7.7143 ms | +| InsertBatch B10000 | 618.81K rows/sec | 10.9217 ms | 162.9867 ms | ### Concurrent Durable Writes @@ -107,45 +107,45 @@ Each row is total successful commits/sec across one shared engine. The intended | Scenario | Commits/sec | Commits/flush | P50 | P99 | |---|---|---|---|---| -| W4, window 0 | 270.5 commits/sec | 1.00 | 14.2462 ms | 23.3782 ms | -| W4, window 250us | 517.4 commits/sec | 1.99 | 7.3000 ms | 16.8764 ms | -| W8, window 0 | 266.2 commits/sec | 1.00 | 29.7311 ms | 42.0490 ms | -| W8, window 250us | 1.04K commits/sec | 3.98 | 7.3218 ms | 15.3828 ms | +| W4, window 0 | 231.1 commits/sec | 1.00 | 17.0038 ms | 23.3867 ms | +| W4, window 250us | 449.6 commits/sec | 1.99 | 8.6405 ms | 15.8344 ms | +| W8, window 0 | 240.5 commits/sec | 1.00 | 32.8008 ms | 41.7800 ms | +| W8, window 250us | 891.0 commits/sec | 3.92 | 8.4358 ms | 16.9718 ms | ### Storage Mode Hot Steady State | Mode | SQL insert | SQL batch x100 | SQL point lookup | Collection put | Collection batch x100 | Collection get | |---|---|---|---|---|---|---| -| File-backed | 276.9 ops/sec | 26.09K rows/sec | 1.30M ops/sec | 263.3 ops/sec | 26.28K docs/sec | 1.58M ops/sec | -| Hybrid incremental-durable | 266.5 ops/sec | 25.62K rows/sec | 1.32M ops/sec | 279.1 ops/sec | 26.40K docs/sec | 1.67M ops/sec | -| In-memory | 217.87K ops/sec | 852.21K rows/sec | 1.29M ops/sec | 236.68K ops/sec | 911.14K docs/sec | 1.74M ops/sec | +| File-backed | 383.4 ops/sec | 33.66K rows/sec | 777.29K ops/sec | 387.5 ops/sec | 34.87K docs/sec | 862.93K ops/sec | +| Hybrid incremental-durable | 377.7 ops/sec | 33.97K rows/sec | 721.75K ops/sec | 379.1 ops/sec | 34.56K docs/sec | 864.55K ops/sec | +| In-memory | 123.41K ops/sec | 449.84K rows/sec | 711.22K ops/sec | 124.92K ops/sec | 519.03K docs/sec | 872.07K ops/sec | ### Resident Hot-Set Reads | Mode | SQL hot burst | SQL P50 | Collection hot burst | Collection P50 | |---|---|---|---|---| -| File-backed | 41.41K ops/sec | 0.0257 ms | 38.17K ops/sec | 0.0270 ms | -| Hybrid incremental-durable | 36.25K ops/sec | 0.0267 ms | 43.53K ops/sec | 0.0226 ms | -| Hybrid hot-set incremental-durable | 535.39K ops/sec | 0.0012 ms | 761.72K ops/sec | 0.0011 ms | -| In-memory | 94.14K ops/sec | 0.0020 ms | 99.35K ops/sec | 0.0021 ms | +| File-backed | 27.88K ops/sec | 0.0346 ms | 28.98K ops/sec | 0.0329 ms | +| Hybrid incremental-durable | 27.16K ops/sec | 0.0366 ms | 29.22K ops/sec | 0.0341 ms | +| Hybrid hot-set incremental-durable | 311.62K ops/sec | 0.0031 ms | 295.45K ops/sec | 0.0029 ms | +| In-memory | 84.82K ops/sec | 0.0049 ms | 122.39K ops/sec | 0.0044 ms | ### Cold Open And First Read | Mode | SQL open+first lookup P50 | Collection open+first get P50 | |---|---|---| -| File-backed | 13.8203 ms | 13.2537 ms | -| Hybrid incremental-durable | 13.4234 ms | 13.4727 ms | -| Hybrid hot-set incremental-durable | 57.0473 ms | 81.2153 ms | -| In-memory | 8.0593 ms | 12.0583 ms | +| File-backed | 18.1275 ms | 20.0627 ms | +| Hybrid incremental-durable | 19.9048 ms | 19.4950 ms | +| Hybrid hot-set incremental-durable | 89.5360 ms | 134.5282 ms | +| In-memory | 15.8719 ms | 21.7409 ms | ### Local SQLite Matched Rows | Engine / row | Throughput | P50 | P99 | |---|---|---|---| -| CSharpDB InsertBatch B1000 | 204.03K rows/sec | 4.1034 ms | 9.5599 ms | -| SQLite WAL+FULL prepared B1000 | 192.06K rows/sec | 4.7479 ms | 18.1078 ms | -| CSharpDB SQL point lookup | 1.33M ops/sec | 0.0005 ms | 0.0021 ms | -| SQLite WAL+FULL point lookup | 138.33K ops/sec | 0.0058 ms | 0.0188 ms | +| CSharpDB InsertBatch B1000 | 233.06K rows/sec | 3.5123 ms | 7.7143 ms | +| SQLite WAL+FULL prepared B1000 | 203.30K rows/sec | 4.6622 ms | 22.0984 ms | +| CSharpDB SQL point lookup | 1.27M ops/sec | 0.0005 ms | 0.0026 ms | +| SQLite WAL+FULL point lookup | 70.21K ops/sec | 0.0119 ms | 0.0369 ms | ## Core Benchmark Map diff --git a/tests/CSharpDB.Benchmarks/release-core-manifest.json b/tests/CSharpDB.Benchmarks/release-core-manifest.json index 386c5c49..a41cab4e 100644 --- a/tests/CSharpDB.Benchmarks/release-core-manifest.json +++ b/tests/CSharpDB.Benchmarks/release-core-manifest.json @@ -1,49 +1,49 @@ { "schemaVersion": 1, "metadata": { - "Published snapshot": "April 21, 2026 release-core snapshot", - "Run date": "Release-core artifacts captured April 21, 2026; release guardrail compare captured April 22, 2026 UTC", - "Promotion status": "Promoted after release-core suite and release guardrail compare passed", + "Published snapshot": "April 26, 2026 release-core snapshot", + "Run date": "Release-core artifacts captured April 26, 2026 PT; release guardrail compare captured April 27, 2026 UTC", + "Promotion status": "Promoted after release-core suite and release guardrail compare passed; focused micro retry replaced volatile guardrail samples", "Latest release guardrail": "PASS=185, WARN=0, SKIP=0, FAIL=0", - "Runner": "Intel i9-11900K, 16 logical cores, Windows 10.0.26300, .NET SDK 10.0.202, .NET runtime 10.0.6", + "Runner": "Intel i9-11900K, 16 logical cores, Windows 10.0.26300, .NET SDK 10.0.203, .NET runtime 10.0.7", "Repro mode": "priority=High, affinity=0xFF when captured with --repro", - "Commit": "4e12bbe6eed58ffa7b7cf85371093008c292ec13 plus uncommitted release-prep fixes" + "Commit": "b7cb52ee2c30f31538e96480b6d055ff52439c26 plus uncommitted collection binary payload and benchmark updates" }, "artifacts": [ { "id": "master", "command": "--master-table --repeat 3 --repro", - "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/master-table-20260421-212338-median-of-3.csv" + "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/master-table-20260426-215529-median-of-3.csv" }, { "id": "batching", "command": "--durable-sql-batching --repeat 3 --repro", - "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/durable-sql-batching-20260421-214227-median-of-3.csv" + "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/durable-sql-batching-20260426-221413-median-of-3.csv" }, { "id": "concurrent", "command": "--concurrent-write-diagnostics --repeat 3 --repro", - "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/concurrent-write-diagnostics-20260421-220505-median-of-3.csv" + "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/concurrent-write-diagnostics-20260426-223659-median-of-3.csv" }, { "id": "storage", "command": "--hybrid-storage-mode --repeat 3 --repro", - "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-storage-mode-20260421-221135-median-of-3.csv" + "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-storage-mode-20260426-224331-median-of-3.csv" }, { "id": "hotset", "command": "--hybrid-hot-set-read --repeat 3 --repro", - "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-hot-set-read-20260421-222712-median-of-3.csv" + "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-hot-set-read-20260426-225908-median-of-3.csv" }, { "id": "coldopen", "command": "--hybrid-cold-open --repeat 3 --repro", - "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-cold-open-20260421-222743-median-of-3.csv" + "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/hybrid-cold-open-20260426-225949-median-of-3.csv" }, { "id": "sqlite", "command": "--sqlite-compare --repeat 3 --repro", - "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/sqlite-compare-20260421-222824-median-of-3.csv" + "path": "tests/CSharpDB.Benchmarks/bin/Release/net10.0/results/sqlite-compare-20260426-230045-median-of-3.csv" } ], "sections": [ @@ -59,7 +59,7 @@ { "value": "Release health" }, { "value": "Latest guardrail compare" }, { "value": "PASS: PASS=185, FAIL=0" }, - { "value": "`Run-Perf-Guardrails.ps1 -Mode release`" } + { "value": "`Compare-Baseline.ps1` after focused micro retry" } ] }, { diff --git a/tests/CSharpDB.Tests/CollectionBinaryDocumentCodecTests.cs b/tests/CSharpDB.Tests/CollectionBinaryDocumentCodecTests.cs index 8fe68697..c253ec9d 100644 --- a/tests/CSharpDB.Tests/CollectionBinaryDocumentCodecTests.cs +++ b/tests/CSharpDB.Tests/CollectionBinaryDocumentCodecTests.cs @@ -198,6 +198,12 @@ public void BinaryCollectionPayload_CodecReader_ReadsNestedDocumentSlice() [System.Text.Encoding.UTF8.GetBytes("city")], out string? city)); Assert.Equal("Seattle", city); + Assert.True( + CollectionBinaryDocumentCodec.TryReadStringUtf8( + nestedDocument, + [System.Text.Encoding.UTF8.GetBytes("city")], + out ReadOnlySpan cityUtf8)); + Assert.Equal("Seattle", System.Text.Encoding.UTF8.GetString(cityUtf8)); } [Fact] diff --git a/tests/CSharpDB.Tests/GeneratedCollectionModelTests.cs b/tests/CSharpDB.Tests/GeneratedCollectionModelTests.cs index de27cc7a..37c3f75f 100644 --- a/tests/CSharpDB.Tests/GeneratedCollectionModelTests.cs +++ b/tests/CSharpDB.Tests/GeneratedCollectionModelTests.cs @@ -1,7 +1,9 @@ using System.Text.Json; using System.Text.Json.Serialization; +using System.Text; using System.Diagnostics.CodeAnalysis; using CSharpDB.Engine; +using CSharpDB.Primitives; using CSharpDB.Storage.Serialization; namespace CSharpDB.Tests; @@ -27,6 +29,134 @@ public async Task GeneratedCollectionModel_UsesGeneratedDescriptorIndexWithoutMa Assert.Equal("alpha@example.com", matches[0].Value.Email); } + [Fact] + public void GeneratedCollectionModel_EncodesGeneratedDirectPayloadsAsBinary() + { + var codec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + var expected = new GeneratedUser("alpha@example.com", 30); + + byte[] payload = codec.Encode("u1", expected); + var actual = codec.Decode(payload); + + Assert.True(CollectionPayloadCodec.IsDirectPayload(payload)); + Assert.True(CollectionPayloadCodec.IsBinaryPayload(payload)); + Assert.Equal("u1", actual.Key); + Assert.Equal(expected, actual.Document); + } + + [Fact] + public void GeneratedCollectionModel_BinaryPayloadUsesCompactRecordFormat() + { + var codec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + var expected = new GeneratedUser("alpha@example.com", 30); + + byte[] payload = codec.Encode("u1", expected); + ReadOnlySpan documentPayload = CollectionPayloadCodec.GetBinaryDocumentPayload(payload); + + Assert.Equal(0xD0, documentPayload[0]); + Assert.Equal(0xF0, documentPayload[1]); + Assert.Equal(0x01, documentPayload[2]); + Assert.True(documentPayload.Length < 32); + Assert.Equal(expected, codec.DecodeDocument(payload)); + } + + [Fact] + public void GeneratedCollectionModel_PayloadMatchesKey_UsesDirectUtf8KeyComparison() + { + var codec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + + byte[] payload = codec.Encode("u:é", new GeneratedUser("alpha@example.com", 30)); + + Assert.True(codec.PayloadMatchesKey(payload, "u:é")); + Assert.False(codec.PayloadMatchesKey(payload, "u:e")); + Assert.True(CollectionPayloadCodec.TryDirectPayloadKeyEquals(payload, "u:é", out bool equals)); + Assert.True(equals); + } + + [Fact] + public void GeneratedCollectionModel_GeneratedFieldsReadDirectBinaryPayloads() + { + var codec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + byte[] payload = codec.Encode("u1", new GeneratedUser("alpha@example.com", 30)); + + Assert.True(GeneratedUser.Collection.Age.TryReadPayloadInt64(payload, out long age)); + Assert.Equal(30, age); + Assert.True(GeneratedUser.Collection.Age.TryReadPayloadValue(payload, out DbValue ageValue)); + Assert.Equal(DbType.Integer, ageValue.Type); + Assert.Equal(30, ageValue.AsInteger); + + Assert.True(GeneratedUser.Collection.Email.TryReadPayloadString(payload, out string? email)); + Assert.Equal("alpha@example.com", email); + Assert.True(GeneratedUser.Collection.Email.TryReadPayloadStringUtf8(payload, out ReadOnlySpan emailUtf8)); + Assert.Equal("alpha@example.com", Encoding.UTF8.GetString(emailUtf8)); + Assert.True(GeneratedUser.Collection.Email.TryReadPayloadValue(payload, out DbValue emailValue)); + Assert.Equal(DbType.Text, emailValue.Type); + Assert.Equal("alpha@example.com", emailValue.AsText); + } + + [Fact] + public void GeneratedCollectionModel_GeneratedNestedFieldsReadDirectBinaryPayloads() + { + var codec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + byte[] payload = codec.Encode("u1", new NestedGeneratedUser("Alice", new NestedGeneratedAddress("Seattle", 98101))); + + Assert.True(NestedGeneratedUser.Collection.Address_ZipCode.TryReadPayloadInt64(payload, out long zipCode)); + Assert.Equal(98101, zipCode); + Assert.True(NestedGeneratedUser.Collection.Address_City.TryReadPayloadString(payload, out string? city)); + Assert.Equal("Seattle", city); + Assert.True(NestedGeneratedUser.Collection.Address_City.TryReadPayloadStringUtf8(payload, out ReadOnlySpan cityUtf8)); + Assert.Equal("Seattle", Encoding.UTF8.GetString(cityUtf8)); + } + + [Fact] + public void GeneratedCollectionModel_BinaryPayloadHonorsJsonPropertyNameAttributes() + { + var codec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + + byte[] payload = codec.Encode("u1", new RenamedGeneratedUser("alpha@example.com", 24)); + var actual = codec.DecodeDocument(payload); + + Assert.True(CollectionPayloadCodec.IsBinaryPayload(payload)); + Assert.Equal("alpha@example.com", actual.Email); + Assert.Equal(24, actual.Age); + Assert.True(RenamedGeneratedUser.Collection.Email.TryReadPayloadString(payload, out string? email)); + Assert.Equal("alpha@example.com", email); + Assert.True(RenamedGeneratedUser.Collection.Age.TryReadPayloadInt64(payload, out long age)); + Assert.Equal(24, age); + } + + [Fact] + public void GeneratedCollectionModel_KeepsJsonPayloadForUnsupportedBinaryShapes() + { + var codec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + var expected = new DateTimeGeneratedUser( + "alpha@example.com", + new DateTime(2026, 4, 26, 12, 30, 0, DateTimeKind.Utc)); + + byte[] payload = codec.Encode("u1", expected); + var actual = codec.Decode(payload); + + Assert.True(CollectionPayloadCodec.IsDirectPayload(payload)); + Assert.False(CollectionPayloadCodec.IsBinaryPayload(payload)); + Assert.Equal("u1", actual.Key); + Assert.Equal(expected, actual.Document); + } + + [Fact] + public void GeneratedCollectionModel_KeepsJsonPayloadForSinglePassEnumerableShapes() + { + var codec = new CollectionDocumentCodec(new DefaultRecordSerializer()); + + byte[] payload = codec.Encode("u1", new EnumerableGeneratedUser("Alice", ["alpha", "beta"])); + var actual = codec.Decode(payload); + + Assert.True(CollectionPayloadCodec.IsDirectPayload(payload)); + Assert.False(CollectionPayloadCodec.IsBinaryPayload(payload)); + Assert.Equal("u1", actual.Key); + Assert.Equal("Alice", actual.Document.Name); + Assert.Equal(["alpha", "beta"], actual.Document.Tags); + } + [Fact] public async Task GeneratedCollectionModel_RangeQuery_WorksAfterReopen() { @@ -449,6 +579,11 @@ internal sealed partial record GeneratedUser(string Email, int Age); internal sealed record UnannotatedGeneratedCollectionUser(string Email); +#pragma warning disable CDBGEN007 +[CollectionModel(typeof(DateTimeGeneratedUserJsonContext))] +internal sealed partial record DateTimeGeneratedUser(string Email, DateTime UpdatedAt); +#pragma warning restore CDBGEN007 + [CollectionModel(typeof(RenamedGeneratedUserJsonContext))] internal sealed partial record RenamedGeneratedUser( [property: JsonPropertyName("email_address")] string Email, @@ -457,6 +592,9 @@ internal sealed partial record RenamedGeneratedUser( [CollectionModel(typeof(TaggedGeneratedUserJsonContext))] internal sealed partial record TaggedGeneratedUser(string Name, IReadOnlyList Tags); +[CollectionModel(typeof(EnumerableGeneratedUserJsonContext))] +internal sealed partial record EnumerableGeneratedUser(string Name, IEnumerable Tags); + [CollectionModel(typeof(NestedGeneratedUserJsonContext))] internal sealed partial record NestedGeneratedUser(string Name, NestedGeneratedAddress Address); @@ -480,6 +618,10 @@ public static class Collection [JsonSerializable(typeof(GeneratedUser))] internal sealed partial class GeneratedUserGeneratedJsonContext : JsonSerializerContext; +[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)] +[JsonSerializable(typeof(DateTimeGeneratedUser))] +internal sealed partial class DateTimeGeneratedUserJsonContext : JsonSerializerContext; + [JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)] [JsonSerializable(typeof(RenamedGeneratedUser))] internal sealed partial class RenamedGeneratedUserJsonContext : JsonSerializerContext; @@ -488,6 +630,10 @@ internal sealed partial class RenamedGeneratedUserJsonContext : JsonSerializerCo [JsonSerializable(typeof(TaggedGeneratedUser))] internal sealed partial class TaggedGeneratedUserJsonContext : JsonSerializerContext; +[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)] +[JsonSerializable(typeof(EnumerableGeneratedUser))] +internal sealed partial class EnumerableGeneratedUserJsonContext : JsonSerializerContext; + [JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)] [JsonSerializable(typeof(NestedGeneratedUser))] internal sealed partial class NestedGeneratedUserJsonContext : JsonSerializerContext; diff --git a/tests/CSharpDB.Tests/OrderedTextIndexKeyCodecTests.cs b/tests/CSharpDB.Tests/OrderedTextIndexKeyCodecTests.cs index 838909ca..bc6a2674 100644 --- a/tests/CSharpDB.Tests/OrderedTextIndexKeyCodecTests.cs +++ b/tests/CSharpDB.Tests/OrderedTextIndexKeyCodecTests.cs @@ -25,6 +25,7 @@ public void ComputeKey_MatchesReferenceUtf8Packing(string text) long expected = ComputeReferenceKey(text); Assert.Equal(expected, OrderedTextIndexKeyCodec.ComputeKey(text)); + Assert.Equal(expected, OrderedTextIndexKeyCodec.ComputeKey(Encoding.UTF8.GetBytes(text))); } private static long ComputeReferenceKey(string text) From 35f4321d95c9d58ee7b046fe53345a6fcc9084e1 Mon Sep 17 00:00:00 2001 From: Maximum Code Date: Mon, 27 Apr 2026 15:08:53 -0700 Subject: [PATCH 2/4] Add sidebar mockup and styles for CSharpDB Studio admin UI - Created sidebar.html for the Object Explorer interface, including sections for Pinned, Recent, Tables, Forms, Reports, Procedures, and Views. - Implemented a search bar, toolbar with filter chips, and item drill-down functionality. - Added styles.css to define design tokens and frame styles, ensuring consistency with the live admin UI. - Included responsive design elements and hover effects for improved user interaction. --- docs/admin-forms-access-parity/README.md | 126 +++ docs/admin-reports-access-parity/README.md | 151 ++++ www/admin-ui-mockups/command-palette.html | 304 +++++++ www/admin-ui-mockups/dashboard.html | 504 +++++++++++ www/admin-ui-mockups/data-tab.html | 495 +++++++++++ www/admin-ui-mockups/forms-mobile.html | 760 ++++++++++++++++ www/admin-ui-mockups/heavy-tab.html | 409 +++++++++ www/admin-ui-mockups/index.html | 539 ++++++++++++ www/admin-ui-mockups/query-tab.html | 602 +++++++++++++ www/admin-ui-mockups/reports-designer.html | 973 +++++++++++++++++++++ www/admin-ui-mockups/reports-mobile.html | 817 +++++++++++++++++ www/admin-ui-mockups/sidebar.html | 474 ++++++++++ www/admin-ui-mockups/styles.css | 355 ++++++++ 13 files changed, 6509 insertions(+) create mode 100644 docs/admin-forms-access-parity/README.md create mode 100644 docs/admin-reports-access-parity/README.md create mode 100644 www/admin-ui-mockups/command-palette.html create mode 100644 www/admin-ui-mockups/dashboard.html create mode 100644 www/admin-ui-mockups/data-tab.html create mode 100644 www/admin-ui-mockups/forms-mobile.html create mode 100644 www/admin-ui-mockups/heavy-tab.html create mode 100644 www/admin-ui-mockups/index.html create mode 100644 www/admin-ui-mockups/query-tab.html create mode 100644 www/admin-ui-mockups/reports-designer.html create mode 100644 www/admin-ui-mockups/reports-mobile.html create mode 100644 www/admin-ui-mockups/sidebar.html create mode 100644 www/admin-ui-mockups/styles.css diff --git a/docs/admin-forms-access-parity/README.md b/docs/admin-forms-access-parity/README.md new file mode 100644 index 00000000..64070b14 --- /dev/null +++ b/docs/admin-forms-access-parity/README.md @@ -0,0 +1,126 @@ +# Admin Forms Access Parity Plan + +This document captures the current Admin Forms review against Microsoft Access-style +form design and data-entry expectations. It focuses on gaps that affect whether +CSharpDB forms can compete with Access for database-backed line-of-business apps. + +## Current Baseline + +The current forms surface already includes: + +- drag/drop absolute-layout designer +- generated forms from table and view schema +- database-backed form metadata +- runtime data entry with create, update, delete, paging, and navigation +- record search and go-to-primary-key navigation +- labels, text, textarea, number, date, checkbox, radio, select, lookup, + computed, data grid, and child-tabs controls +- lookup lists loaded from tables +- computed fields with simple formulas and child-table aggregates +- one-to-many child grids and nested child tabs +- print support +- schema-change warnings +- designer undo/redo, copy/paste, duplicate, layers, alignment, tab order, and + mobile/tablet/desktop breakpoint editing + +## Added Review Findings + +### P1: Runtime ignores responsive form layouts + +The designer stores mobile and tablet overrides in `RendererHints`, but the +runtime renderer always uses `ControlDefinition.Rect`. Any breakpoint layout work +saved in the designer will not affect actual data-entry rendering. + +Primary code path: + +- `src/CSharpDB.Admin.Forms/Components/Designer/FormRenderer.razor` +- `src/CSharpDB.Admin.Forms/Components/Designer/DesignerState.cs` + +Expected fix: + +- Add a runtime breakpoint/layout resolver shared with the designer. +- Render controls from the effective breakpoint rectangle, not only the desktop + rectangle. +- Honor breakpoint visibility at runtime. +- Add renderer tests for desktop, tablet, and mobile overrides. + +### P1: Inferred validation is not enforced + +`InferRules` creates required, range, regex, and one-of rules, but `Evaluate` +currently checks only `maxLength` and manually added `required` rules. Default +generated forms can save invalid required, range, regex, or enum data unless the +database rejects it later. + +Primary code path: + +- `src/CSharpDB.Admin.Forms/Services/DefaultValidationInferenceService.cs` +- `src/CSharpDB.Admin.Forms/Services/DefaultFormGenerator.cs` + +Expected fix: + +- Evaluate inferred `required`, `maxLength`, `range`, `regex`, and `oneOf` + rules for generated controls. +- Keep validation override behavior intact. +- Normalize numeric and choice values before comparing. +- Add tests for generated forms and override combinations. + +## Access-Parity Roadmap + +### Phase 1: Correctness Before Expansion + +| Feature | Status | Notes | +| --- | --- | --- | +| Runtime breakpoint rendering | Planned | Make mobile/tablet designer work visible in data entry. | +| Complete inferred validation | Planned | Enforce generated required/range/regex/choice rules before save. | +| Form runtime regression tests | Planned | Cover renderer layout, validation, lookup, computed, and child grid behavior. | + +### Phase 2: Record Source, Filtering, and Sorting + +| Feature | Status | Notes | +| --- | --- | --- | +| First-class record source model | Planned | Move beyond a single `TableName`; support table, view, and saved SQL/query sources with editability metadata. | +| Default filter and default sort | Planned | Store form-level defaults and apply them in the runtime record service. | +| User sorting | Planned | Let operators sort by visible/searchable fields at runtime. | +| Advanced filtering | Planned | Add filter-by-field, filter-by-selection, multi-condition filters, saved filters, and clear-filter flows. | + +### Phase 3: Access-Style Form Experiences + +| Feature | Status | Notes | +| --- | --- | --- | +| Layout View | Planned | Let designers adjust a form while real data is visible. | +| Multiple form modes | Planned | Add Single Form, Multiple Items, Datasheet, and Split Form equivalents. | +| Form sections | Planned | Add header, detail, footer, and optional print sections. | +| Embedded subforms | Planned | Support arbitrary embedded form definitions, not only child grids/tabs. | + +### Phase 4: Actions, Events, and App Behavior + +| Feature | Status | Notes | +| --- | --- | --- | +| Command button control | Planned | Add buttons that can run form actions. | +| Action model | Planned | Support actions such as open form, save, delete, navigate, apply filter, clear filter, run SQL/procedure, and show message. | +| Event hooks | Planned | Add form/control events such as on load, before save, after save, before field change, after field change, and button click. | +| Conditional UI rules | Planned | Add visible/enabled/read-only expressions for controls. | + +### Phase 5: Broader Control and Property Coverage + +| Feature | Status | Notes | +| --- | --- | --- | +| Control palette expansion | Planned | Add list box, option group, toggle button, image, attachment/blob, chart, navigation, line, rectangle, page break, and subreport-like controls. | +| Formatting properties | Planned | Add font, color, border, alignment, numeric/date format, input mask, default value, and required indicators. | +| Lookup improvements | Planned | Add searchable combo behavior, display/value column configuration, row limits, and dependent lookups. | +| Child grid improvements | Planned | Add typed editors, validation, sort/filter, column sizing, and batch edit behavior. | + +## Product Positioning + +The current implementation is a strong database-backed generated-form system. +To credibly compete with Microsoft Access as an app builder, the next work should +move from "render fields over records" toward "design complete data-entry +workflows." The highest leverage model changes are: + +- runtime layout resolver +- full validation engine +- richer record-source/filter/sort model +- action/event model +- form-mode model + +Those foundations should be added before expanding the control palette too far. diff --git a/docs/admin-reports-access-parity/README.md b/docs/admin-reports-access-parity/README.md new file mode 100644 index 00000000..1195d992 --- /dev/null +++ b/docs/admin-reports-access-parity/README.md @@ -0,0 +1,151 @@ +# Admin Reports Access Parity Plan + +This document captures the current Admin Reports review against Microsoft +Access-style report design, preview, print, and distribution expectations. It +focuses on gaps that affect whether CSharpDB reports can compete with Access for +database-backed operational reporting. + +## Current Baseline + +The current reports surface already includes: + +- visual band-based report designer +- database-backed report metadata +- report sources from tables, views, and supported saved queries +- default report generation from source schema +- page settings for Letter/A4, portrait/landscape, and margins +- report header, page header, detail, page footer, report footer, group header, + and group footer bands +- grouping and sorting definitions +- labels, bound text, calculated text, lines, and boxes +- calculated expressions for page number, print date, field references, numeric + arithmetic, and `SUM`/`COUNT`/`AVG`/`MIN`/`MAX` aggregates +- preview pagination with page headers and footers +- print support through the browser +- schema-drift warnings + +## Added Review Findings + +### P1: Saved-query previews are unbounded before trimming + +Saved-query reports execute `source.BaseSql` directly and only trim rows after +the full result has been materialized in memory. Large saved queries can make +preview slow, memory-heavy, or effectively unusable. Table and view reports use +the preview query builder with a row limit, but saved-query reports bypass that +path. + +Primary code path: + +- `src/CSharpDB.Admin.Reports/Services/DefaultReportPreviewService.cs` +- `src/CSharpDB.Admin.Reports/Services/ReportPreviewQueryBuilder.cs` + +Expected fix: + +- Apply the preview row cap before materializing saved-query results. +- Preserve report group/sort ordering by wrapping the saved query or using an + equivalent safe query-builder path. +- Keep saved-query SQL validation parameterless unless parameter support is + added in the same work. +- Add tests that prove large saved-query previews fetch only the capped row + window. + +### P2: Preview and print are capped, with no full output/export pipeline + +The preview service intentionally caps output at `10,000` rows and `250` pages. +That is reasonable for an interactive preview, but Access-style reports also +need a full render/export path for print-ready output and distribution. + +Primary code path: + +- `src/CSharpDB.Admin.Reports/Services/DefaultReportPreviewService.cs` +- `src/CSharpDB.Admin.Reports/Pages/Preview.razor` + +Expected fix: + +- Separate preview rendering from full report rendering. +- Add export targets such as PDF, HTML, CSV, and spreadsheet-friendly output. +- Keep preview caps for the UI, but make full export explicit and cancellable. +- Show clear warnings when printing a capped preview rather than the full report. + +## Access-Parity Roadmap + +### Phase 1: Runtime Safety and Output Foundations + +| Feature | Status | Notes | +| --- | --- | --- | +| Bounded saved-query previews | Planned | Apply preview row limits before materializing saved-query results. | +| Full report render pipeline | Planned | Separate capped preview from full print/export rendering. | +| Export support | Planned | Add PDF first, then HTML, CSV, and spreadsheet-friendly exports. | +| Print warnings | Planned | Make truncated preview printing explicit in the UI. | + +### Phase 2: Record Sources, Parameters, and Filters + +| Feature | Status | Notes | +| --- | --- | --- | +| Parameterized report sources | Planned | Support saved query/report parameters with prompt UI and typed values. | +| Runtime filters | Planned | Let users run a report with ad hoc filters without editing the design. | +| Saved report filter definitions | Planned | Store default filters with the report definition. | +| Source query builder integration | Planned | Let reports be based on query-designer definitions, not only raw tables/views/saved queries. | + +### Phase 3: Grouping, Totals, and Pagination Semantics + +| Feature | Status | Notes | +| --- | --- | --- | +| Grouping options | Planned | Add group intervals, header/footer toggles, keep together, repeat section, and force-new-page behavior. | +| Running totals | Planned | Add per-group and whole-report running sums. | +| Total placement helpers | Planned | Add guided totals in group headers, group footers, report header, and report footer. | +| Page header/footer options | Planned | Support Access-style choices such as suppressing page headers on report-header/report-footer pages. | +| Overflow behavior | Planned | Add text growth/shrink, clipping rules, and band overflow tests. | + +### Phase 4: Design Productivity + +| Feature | Status | Notes | +| --- | --- | --- | +| Layout View | Planned | Let users adjust a report while seeing real data. | +| Report Wizard | Planned | Guide source, fields, grouping, sorting, layout, and totals selection. | +| Label Wizard | Planned | Generate printable mailing/product labels from source fields. | +| Style themes | Planned | Add reusable report styles for fonts, spacing, borders, and colors. | + +### Phase 5: Broader Report Controls and Formatting + +| Feature | Status | Notes | +| --- | --- | --- | +| Control palette expansion | Planned | Add image/logo, rich text, barcode, chart, page break, subreport, and attachment/blob controls. | +| Conditional formatting | Planned | Add value/expression-based formatting rules for report controls. | +| Data bars and highlights | Planned | Add visual summaries for numeric ranges and thresholds. | +| Advanced formatting | Planned | Add borders, fill colors, font styles, text wrapping, alignment, culture-aware formats, and background images. | +| Subreports | Planned | Embed another report definition with parent/child linking. | + +### Phase 6: Distribution and Operations + +| Feature | Status | Notes | +| --- | --- | --- | +| Email/report delivery | Planned | Export and attach reports, with host-provided delivery hooks. | +| Scheduled reports | Research | Run recurring reports and store generated artifacts. | +| Report artifact history | Research | Store generated report snapshots for auditing and re-download. | +| Large-report cancellation | Planned | Add cancellation/progress for long render/export jobs. | + +## Product Positioning + +The current implementation is a useful printable preview engine with a banded +designer. To compete with Microsoft Access as a report builder, the next work +should move toward "reliable report production and distribution." The highest +leverage foundations are: + +- bounded source execution +- separate preview and full-render pipelines +- parameter/filter model +- richer grouping and pagination model +- export/distribution model +- conditional formatting and expanded controls + +Those foundations should come before broadening the designer surface too far. + +## References + +- [Introduction to reports in Access](https://support.microsoft.com/en-us/office/introduction-to-reports-in-access-e0869f59-7536-4d19-8e05-7158dcd3681c) +- [Create a simple report](https://support.microsoft.com/en-us/office/create-a-simple-report-408e92a8-11a4-418d-a378-7f1d99c25304) +- [Create a grouped or summary report](https://support.microsoft.com/en-au/office/create-a-grouped-or-summary-report-f23301a1-3e0a-4243-9002-4a23ac0fdbf3) +- [Summing in reports](https://support.microsoft.com/en-us/office/summing-in-reports-ad4e310d-64e9-4699-8d33-b8ae9639fbf4) +- [Highlight data with conditional formatting](https://support.microsoft.com/en-us/office/highlight-data-with-conditional-formatting-7f7c0bd4-7c37-421d-adad-a260125c8129) +- [Distribute a report](https://support.microsoft.com/en-us/office/distribute-a-report-561a9066-00ab-41ee-8f07-a0734810a778) diff --git a/www/admin-ui-mockups/command-palette.html b/www/admin-ui-mockups/command-palette.html new file mode 100644 index 00000000..b7a51d0b --- /dev/null +++ b/www/admin-ui-mockups/command-palette.html @@ -0,0 +1,304 @@ + + + + +Command Palette — CSharpDB Studio Mockup + + + + + +
+ + +
+
+ + CSharpDB Studio + — relational.db +
+ + +
+ Connected +
+ +
+ +
+
+
Dashboard
+
Customers
+
+
+
+ +
+ Connected +
+
+ + +
+ + +
+
+ + + Esc +
+ +
+ All + Actions + Tables · 4 + Views · 1 + Forms · 2 + Reports · 1 + Procedures · 1 +
+ +
+ +
Tables
+
+
+
Customers 1,247 rows
+
+ Open + ↵ +
+
+
+
+
CustomerAddresses 2,109 rows
+
Open
+
+
+
+
CustomerNotes 514 rows
+
Open
+
+ +
Forms
+
+
+
Customer Form → Customers
+
Form
+
+ +
Reports
+
+
+
Customer Sales Report → vw_customer_sales
+
Report
+
+ +
Procedures
+
+
+
recalc_customer_balances
+
Proc
+
+ +
Actions
+
+
+
Run: SELECT * FROM Customers LIMIT 50
+
+ Action + Ctrl ↵ +
+
+
+
+
New Form for Customers
+
Action
+
+
+
+
Drop table Customersrequires confirm
+
Danger
+
+ +
+ +
+ ↑↓ Navigate + ↵ Open + Ctrl ↵ Open in new tab + Tab Filter by type + Esc Dismiss +
+ 9 results in 3 ms +
+
+ + +
+
Reality check vs. the live screen
+
    +
  • ALREADY EXISTS — sidebar text-filter input that narrows the visible tree (case-insensitive substring match), keyboard shortcuts Ctrl+N (new query) / Ctrl+B (toggle sidebar) / Ctrl+Enter (run query) / Ctrl+W (close tab) / Ctrl+Shift+L (theme), right-click context menus on every object kind with "Open / Design / Select Top 50 / Drop" actions, modal confirm for danger ops, autocomplete inside the SQL editor.
    • +
  • DOES NOT EXIST TODAY — there is no Ctrl+K palette or any global "search anything" entry point. Users must locate items via the sidebar tree or open them from a tab.
    • +
  • NEW: Ctrl+K palette — fuzzy search across tables, views, forms, reports, procedures, and saved queries in one input.
    • +
  • NEW: Action items inline — "Run: SELECT TOP 50 FROM X", "New Form for X", "Drop X". Surfaces what's currently buried in right-click menus.
    • +
  • NEW: Type filter chips with keyboard Tab cycling.
    • +
  • NEW: Title-bar launcher ("Search anything…") for discoverability.
    • +
  • Danger actions still route through the existing Modal.ConfirmAsync with isDanger:true — reuses what's there.
    • +
+
+ + diff --git a/www/admin-ui-mockups/dashboard.html b/www/admin-ui-mockups/dashboard.html new file mode 100644 index 00000000..f909ab36 --- /dev/null +++ b/www/admin-ui-mockups/dashboard.html @@ -0,0 +1,504 @@ + + + + +Dashboard — CSharpDB Studio Mockup + + + + + +
+ + +
+
+ + CSharpDB Studio + — relational.db +
+ + +
+ + + +
+
+ Connected + + + +
+ + +
+ + + + +
+
+
Dashboard
+
Customers
+
Query 1
+ +
+ +
+
+
+
relational.db
+

Dashboard

+
+
+ + +
+
+ + +
+
+
Tables
+
24
+
+2 this week
+ +
+
+
Views
+
6
+
no change
+ +
+
+
Procedures
+
11
+
+1 today
+ +
+
+
Indexes
+
38
+
across 14 tables
+ +
+
+
Storage
+
2.4 GB
+
+
2.4 GB used3.5 GB cap
+
+
+ +
+ +
+
+
+

Top tables by row count

+ View all 24 → +
+
+
Customers
+
1,247
+
+ + + + +
+
→
+
+
+
Orders
+
8,912
+
+ + + + +
+
→
+
+
+
OrderItems
+
24,310
+
+ + + + +
+
→
+
+
+
Invoices
+
5,103
+
+ + + + +
+
→
+
+
+
Products
+
412
+
+ + + + +
+
→
+
+
+ +
+
+

Recent activity

+ Open log → +
+
+
+
+
Ran query on Customers
+
SELECT * FROM Customers WHERE status = 'active' LIMIT 50
+
+
2m ago
+
+
+
+
+
Edited form Customer Form
+
Added field "loyaltyTier"
+
+
14m ago
+
+
+
+
+
Created procedure recalc_invoices
+
11 statements · validated
+
+
1h ago
+
+
+
+
+
Pipeline nightly_etl finished with warnings
+
3 of 1,204 rows skipped (type mismatch)
+
+
3h ago
+
+
+
+ + +
+
+
+

Pinned

+ +
+
+
Customers
+
Orders
+
Customer Form
+
Sales Report
+
"Active customers"
+
recalc_invoices
+
+
+ +
+

Quick actions

+
+ + + + + + +
+
+ +
+
+

Health

+ Run full check → +
+
+
Last integrity check2h ago · OK
+
Open writer transactions1
+
Storage pressure68%
+
Background jobs2 running
+
WAL fsync modeHybridIncrementalDurable
+
+
+
+
+
+
+
+ + +
+ Connected + relational.db + 24 tables · 6 views · 11 procs + 2 background jobs +
+ 2.4 GB / 3.5 GB + 3 notifications +
+
+ + +
+
Reality check vs. the live screen
+
    +
  • ALREADY EXISTS — current Welcome tab shows app icon, app name, four shortcut hints (Ctrl+N / Ctrl+B / Ctrl+Enter / Ctrl+Shift+L), and a grid of table cards with a "Load row counts" button. Status bar already reports table/view/procedure counts. Title bar already has Open Database / New Query / New Table / New Form / New Procedure / New Pipeline / Storage buttons plus theme/sidebar/help toggles. ANALYZE and storage stats already accessible via the Storage tab and Query tab "Stats" shortcuts.
    • +
  • NEW: Stat cards row with deltas and a storage-pressure bar.
    • +
  • NEW: Top tables panel with sparklines for row-count growth — today you'd run an ad-hoc query.
    • +
  • NEW: Recent activity feed — there's no audit/activity surface today.
    • +
  • NEW: Pinned objects grid — pinning doesn't exist anywhere yet.
    • +
  • RESTYLE: Quick action tiles — same actions as the title-bar buttons, in a more discoverable card form.
    • +
  • NEW: Health panel — surfaces integrity, open writers, storage pressure, background jobs, WAL mode in one place. Today these are scattered across StorageTab's 11 stacked sections.
    • +
  • NEW: Title-bar database switcher dropdown + search-style palette launcher — today there's just an "Open Database" button that opens a path prompt (no recent-databases list).
    • +
  • NEW: Status-bar storage pressure / background-job ticker / notification counter — current status bar shows connection state, db path, and counts.
    • +
+
+ + diff --git a/www/admin-ui-mockups/data-tab.html b/www/admin-ui-mockups/data-tab.html new file mode 100644 index 00000000..b1c1199f --- /dev/null +++ b/www/admin-ui-mockups/data-tab.html @@ -0,0 +1,495 @@ + + + + +Data Tab — CSharpDB Studio Mockup + + + + + +
+ +
+
+ + CSharpDB Studio + — relational.db +
+ + +
+ Connected + + +
+ +
+ + +
+
+
Dashboard
+
Customers
+
Query 1
+ +
+ + +
+
+ tables + / + Customers +
+ +
+ + + + +
+ +
+ + +
+ +
+ + + +
+ +
+ + +
+ +
+ Rows 1–25 of 1,247 + + +
+
+ + +
+ Filters: + + status + = + 'active' + + + + created_at + ≥ + 2026-01-01 + + + + name + LIKE + '%Smith%' + + + + + Showing 42 of 1,247 rows + +
+ + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
id nameemailstatuscreated_atorder_countnotes
1Alice Smithalice@example.comactive2026-01-12 09:1412NULL
2Bob Leebob@example.comactive2026-01-14 11:023VIP — handle with care
3Carol Smithcarol@example.compending2026-02-03 16:480NULL
4David Smithdsmith@example.comactive2026-02-19 08:227Net 30 terms
5Eve Smitherseve@example.cominactive2026-03-01 14:1022Migrated from legacy
6Frank Smithyfrank@example.comactive2026-03-12 10:551NULL
7Grace Smith-Jonesgrace@example.comactive2026-03-22 09:175Refer-a-friend
8Henry Smithfieldhank@example.compending2026-04-02 12:380NULL
+ + +
+ 2 selected + Apply an action to the selected rows: +
+ + + + +
+
+
+
+
+ +
+ Connected + relational.db + Customers · 1,247 rows + 3 filters · 42 visible +
+ Last refresh 2s ago + 2.4 GB / 3.5 GB +
+
+ + +
+
Reality check vs. the live screen
+
    +
  • ALREADY EXISTS — Data/Schema view switcher, Add Row, Delete (with selection count), Save, Discard, Refresh, Drop Table buttons in toolbar. Per-column filter row with Contains/StartsWith/EndsWith/Exact mode selector. Sortable headers with ↑↓ indicators. Type badges (INTEGER/TEXT/REAL/BLOB). PK badges. Pagination bar with page-size dropdown and first/prev/next/last. Inline cell editing on double-click. Right-click context menu for row actions. NULL and BLOB cell rendering.
    • +
  • NEW: Breadcrumb header ("tables / Customers") — today the object name is right-aligned in the toolbar info area.
    • +
  • NEW: Filter summary chip strip — a recap of all active filters as removable pills. The per-column filter row stays; this adds an overview that's currently missing.
    • +
  • NEW: Schema sub-tabs — split the Schema view's three stacked sections (Columns, Indexes, Triggers) into sub-tabs so users don't scroll past the first section.
    • +
  • NEW: Export / Import buttons — no export today.
    • +
  • NEW: Bulk action bar at bottom when rows are selected — Bulk edit, Copy as INSERT, Export, Delete. Delete exists today via toolbar/context menu, but Bulk edit and Copy as INSERT are new.
    • +
  • RESTYLE — type icons in column headers (replacing or supplementing the existing text badges). Status pills for enum-ish columns. Right-aligned numbers, distinct date coloring. Visual polish on top of features that already work.
    • +
  • NEW: Status-bar pills for table-specific facts (row count, active filter count, last refresh).
    • +
+
+ + diff --git a/www/admin-ui-mockups/forms-mobile.html b/www/admin-ui-mockups/forms-mobile.html new file mode 100644 index 00000000..88a82b95 --- /dev/null +++ b/www/admin-ui-mockups/forms-mobile.html @@ -0,0 +1,760 @@ + + + + +Elastic Forms — CSharpDB Studio Mockup + + + + + +
+ +
CSharpDB.Admin.Forms · UI proposal
+

Make forms mobile-friendly with an "Elastic" layout mode

+

+ Today every form rendered by FormRenderer.razor uses absolute pixel coordinates + — controls have a fixed X / Y / W / H from Rect, and on a phone the + form simply scrolls horizontally. The LayoutDefinition model already carries a + Breakpoints list and ControlDefinition already has + RendererHints, so the schema is responsive-ready. This proposal adds a designer + switch and a runtime that uses them. +

+ + +

1 · Designer · new "Layout mode" switch

+ +
+
+
+
+ Layout mode +
+ + +
+
+
+ Preview +
+ + + +
+
+
+ 12-col grid · 8 px gutter +
+
+
+ 2 controls overflow at Mobile +
+
+ +
+
+
+
Customer · CST-00142
+
vw_customer_form · tablet preview · 720 × auto
+ +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ +
+ + +
+
+
tablet · 768 px breakpoint
+
+
+
+ + +
+

Layout settings

+ +
+ Mode + Elastic · 12-col grid +
+
+ Auto-stack on mobile +
+
+
+ Touch targets + ≥ 44 px on mobile +
+ +

Breakpoints

+ +
+ Desktop + ≥ 1024 px · 12 cols +
+
+ Tablet + ≥ 640 px · 8 cols +
+
+ Mobile + < 640 px · stacked +
+ +

Auto-elastic

+
+ For legacy forms +
+
+

+ When a form has no breakpoint overrides, the renderer infers a 12-col grid from the + pixel layout (X/W → column span, Y → row order) so existing forms get a sensible + mobile view without redesign. +

+
+
+ + +

2 · Same form at three breakpoints

+

+ One form definition, three viewport sizes. Designer can switch between them with the toolbar + Preview segmented control; users see the right one based on their screen. +

+ +
+ +
+
+
Customer · CST-00142
+
desktop · 12-col
+ +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+
+ + +
+
+
desktop · ≥ 1024 px
+
+ + +
+
+
Customer · CST-00142
+
tablet · 8-col
+ +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+
+ + +
+
+
tablet · ≥ 640 px
+
+ + +
+
+
+
Customer · CST-00142
+
mobile · stacked
+ +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+
+ +
+
+
mobile · < 640 px
+
+
+ + +

3 · Today vs. Elastic on a 380 px phone

+ +
+ +
+
+
+
+
Customer · CST-00142
+
+ TODAY · pixel-positioned · 800 px wide +
+
+
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+
+
Horizontal scroll · cramped · no save button visible
+
+ + +
+
+
+
Customer · CST-00142
+
+ ELASTIC · auto-stacked · 380 px +
+ +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+
+ +
+
+
Single-column · 44 px touch targets · primary action visible
+
+
+ + +

4 · Implementation notes (no code changed)

+ +
+
    +
  • + ● + Schema is already responsive-ready. + LayoutDefinition has Breakpoints and + ControlDefinition has RendererHints. Per-breakpoint + geometry can ride in RendererHints["breakpoint:mobile"] = { x, y, w, h, hidden } + with no model migration. DesignerState.IsVisibleAtBreakpoint and + GetEffectiveRect already handle this design-side. +
    • +
  • + ● + Add a LayoutMode field to LayoutDefinition + — values: FixedPixel (today's behavior) and Elastic. + Existing forms default to FixedPixel, so nothing changes for them. +
    • +
  • + ● + Runtime FormRenderer.razor picks the layout using + a CSS container query (or a one-time window.matchMedia probe) and emits + either: +
    — position: absolute; left/top/width/height (FixedPixel, today), +
    — CSS Grid with grid-column: span N per control (Elastic). +
    • +
  • + ● + Auto-elastic for legacy forms — when a form is FixedPixel but the + viewport is below the tablet breakpoint, the renderer can synthesize a column span + from the original Rect.Width / canvasWidth × 12 and order by + Rect.Y. Doesn't require redesigning every form. +
    • +
  • + ● + Designer changes — toolbar gains a Layout-mode segmented control and a + Preview-device segmented control. The existing 8 px snap-to-grid stays. Property + inspector grows a "Span at desktop / tablet / mobile" row per control. The "2 + controls overflow at Mobile" pill is a lint warning that links to the offenders. +
    • +
  • + ● + Runtime niceties — touch targets ≥ 44 px on mobile, primary action sticks to + the bottom of the viewport, the toolbar ([DataEntry.razor toolbar](../../src/CSharpDB.Admin.Forms/Pages/DataEntry.razor)) becomes a flex-wrap row that + collapses Print/Edit-Form behind a "⋯" overflow on small screens. Print stylesheet + ignores breakpoints and renders the desktop layout as today. +
    • +
  • + ● + Out of scope for this proposal — touch-drag in the designer (designer can + stay desktop-only), mobile-specific control variants (e.g. native date picker), and + an offline data-entry mode. +
    • +
+
+ +

+ Back to all mockups. +

+
+ + diff --git a/www/admin-ui-mockups/heavy-tab.html b/www/admin-ui-mockups/heavy-tab.html new file mode 100644 index 00000000..7b246ed9 --- /dev/null +++ b/www/admin-ui-mockups/heavy-tab.html @@ -0,0 +1,409 @@ + + + + +Heavy tab → vertical sub-nav (Storage example) — CSharpDB Studio Mockup + + + + + +
+ +
+
+ + CSharpDB Studio + — relational.db +
+ + +
+ Connected +
+ +
+ + +
+
+
Dashboard
+
Storage
+ +
+ +
+ + + + +
+
Storage / Overview
+

Summary

+

An at-a-glance look at the database file. Pick a category from the rail on the left to drill into details.

+ +
+ + + +
+ +
+
+

File

healthy
+
+
+
+
2.4 GB
+
Database file
+
+
+
68%
+
of 3.5 GB cap
+
+
+
+ + + + + +
Page size4 KB
Physical pages614,400
Freelist pages12
WAL file128 MB
+
+
+ +
+

Health

3 warnings
+
+ + + + + +
Index checksall OK
Integrity issues3 warnings
Last integrity check2h ago
Open writer txns1
+
+
+ +
+

Recent maintenance

last 7 days
+
+ + + + + +
Last backup16h ago · customers.backup.db
Last vacuum4d ago · 312 MB reclaimed
Last reindex2d ago · 38 indexes
Last FK migration—
+
+
+ +
+

Page type histogram

+
+ + + + + +
btree-leaf412,099
btree-internal8,213
overflow194,076
freelist12
+
+
+
+ +
+
+

Top integrity issues

+ View all 3 → +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SeverityCodeMessagePage
WARNWAL_TRAILING_BYTESWAL file has 24 trailing bytes after the last commit frame.—
WARNFREELIST_GAPFreelist contains a gap; consider running vacuum.512
WARNINDEX_FREE_SPACE_HIGHIndex ix_customers_email has >50% free space across leaf pages.3,212
+
+
+
+
+
+ +
+ Connected + relational.db + 3 storage warnings +
+ 2.4 GB / 3.5 GB +
+
+ + +
+
The pattern: split heavy tabs into vertical sub-nav
+
    +
  • ALREADY EXISTS in StorageTab — Database header, WAL, Space usage, Fragmentation, Maintenance (Reindex / Vacuum), Backup & Restore, FK migration, Page-type histogram, Index checks, Integrity issues, Page drill-down. Each is a fully-featured section.
    • +
  • PROBLEM — all eleven sections are vertically stacked in one long scroll. Users have to scroll past the file header every time they want to reach maintenance, and there's no overview of where to start.
    • +
  • NEW: Vertical sub-nav rail on the left of the tab. Categories: Overview, Inspect, Health, Maintenance. Each item is one click away — no scrolling.
    • +
  • NEW: Summary screen as the default view — surfaces the most useful facts (file size with cap, health pills, recent maintenance, top warnings) so a user landing on Storage gets value without picking a section.
    • +
  • NEW: Status badges in the rail (e.g. "WARN 3" on Integrity issues) so users see what needs attention without opening every section.
    • +
  • SAME PATTERN APPLIES TO:
    • +
  • — Pipeline tab — currently stacks Designer, Run Result, Stored Pipelines, Recent Runs vertically. Rail items: Designer / Run / Stored / History / Rejects.
    • +
  • — Procedure tab — currently stacks Definition, Parameters, Execution. Rail items: Definition / Parameters / Run / Results.
    • +
  • — Schema view of Data tab — currently stacks Columns, Alter Table, Indexes, Triggers. Could become sub-tabs as proposed in data-tab.html.
    • +
  • Implementation note: the rail items are routes within a single tab; switching them does not open a new top-level tab. State (e.g. an unsaved JSON edit in Pipeline) is preserved across rail switches.
    • +
+
+ + diff --git a/www/admin-ui-mockups/index.html b/www/admin-ui-mockups/index.html new file mode 100644 index 00000000..3dcc088f --- /dev/null +++ b/www/admin-ui-mockups/index.html @@ -0,0 +1,539 @@ + + + + +CSharpDB Admin — UI Improvement Mockups + + + + + +
+

CSharpDB Admin · UI Improvement Mockups

+

Static HTML previews of proposed enhancements to the Blazor Server admin + (src/CSharpDB.Admin). Nothing here is wired to the live app — these are visual + proposals for discussion, now reconciled against the actual current screens.

+

Open each file directly in a browser to see the mocked screen with annotations.

+ +
+
NEW Capability that doesn't exist today
+
RESTYLE Existing feature, reorganized or visually refreshed
+
EXISTS Already in the live admin (don't propose as new)
+
GAP Missing today, this mockup adds it
+
+ +

The big ideas at a glance

+
+
    +
  • Replace the sparse Welcome tab with a real database dashboard: stat cards, top tables with sparklines, recent activity feed, pinned objects, quick actions, and a consolidated health panel.
    • +
  • Add a command palette (Ctrl+K) — there is no global search today; users must navigate the sidebar tree or right-click for actions.
    • +
  • Sidebar gains Pinned + Recent + drill-down under each table. The existing search/filter and right-click context menus stay; pin star and per-group "+" become discoverable.
    • +
  • Data tab refresh is mostly visual — most "features" I first proposed (per-column filters, sortable headers, type badges, pagination, inline edit, row selection, save/discard) already exist. The genuine adds are: breadcrumb header, filter summary chips, schema sub-tabs, export/import, bulk-edit / copy-as-INSERT, status-bar table facts.
    • +
  • Query tab gains result sub-tabs (Results / Plan / Messages / Stats), an Explain Plan button, a Cancel button, and a query-history rail. Run / Format / Save / Designer mode / autocomplete / completions popup all already exist.
    • +
  • Heavy tabs (Storage / Pipeline / Procedure) currently stack 4–11 sections in one long scroll. Replace with a vertical sub-nav rail and a Summary screen so users land on something useful.
    • +
  • Forms reflow on mobile via a new Elastic layout mode in CSharpDB.Admin.Forms — today every control is absolute-positioned in pixels, so phone users get horizontal scroll. Schema already has Breakpoints and RendererHints; the runtime just doesn't use them.
    • +
  • Reports get a Reader view in CSharpDB.Admin.Reports — paper layout stays untouched (it's supposed to look like paper, and Print/PDF need it), but a derived Reader view turns bands into sections and repeated detail rows into cards. Phone defaults to Reader.
    • +
  • Report designer surfaces the Reader view via a Paper / Reader / Split toolbar toggle, a live phone-sized preview pane, a Reader-hint subsection in the Selection inspector, an "Auto-derive all hints" action, and a Reader-lint panel — so the runtime view never drifts from designer intent.
    • +
  • Title bar gains a database switcher (recent databases dropdown) and a search-style command-palette launcher. Today there's just an "Open Database" button that opens a path prompt.
    • +
+
+ +

Mockups

+
+ + +
+
+ DASHBOARD · relational.db + ┌─ Tables ──┬─ Views ──┬─ Procs ──┐ + │ 24 │ 6 │ 11 │ + └───────────┴──────────┴──────────┘ + Top tables ▮▮▮▮▮▮▮▮ Customers + ▮▮▮▮▮▮ Orders + ▮▮▮▮ Invoices +
+ +
+
+

Database Dashboard

+

Replaces the sparse Welcome tab. Stats, top tables with sparklines, recent activity, pinned objects, quick actions, health panel. Mostly net-new content.

+
+
+ High impact · mostly new + Open mockup → +
+ + + +
+
+ ⌘ Search anything + ──────────────────────────── + ▶ Open table: Customers + Run: Select Top 50 + New Form for Customers +
+ +
+
+

Command Palette (Ctrl+K)

+

One keystroke to fuzzy-find tables, forms, reports, procedures, or actions. There is no global search in the app today — this is purely additive.

+
+
+ Fully new + Open mockup → +
+ + + +
+
+ ★ PINNED + ▦ Customers ▦ Orders + ⏱ RECENT + ▦ Invoices ◫ Customer Form + ▾ TABLES 24 + + ▦ Customers + ▦ Orders +
+ +
+
+

Improved Object Explorer

+

Pinned + Recent sections, type filter chips, drill-down under tables (columns / indexes / triggers as nested rows). The base tree, search, and right-click menus already exist.

+
+
+ High value · adds + restyle + Open mockup → +
+ + + +
+
+ tables / Customers · 1,247 rows + [Data] Schema Indexes Triggers + ▼ status = active ▼ created > 2026-01 + id │ name │ email │ status + ───┼───────────┼───────────┼─────── + 1│ A. Smith │ as@e.com │ active + 2│ B. Lee │ bl@e.com │ active +
+ +
+
+

Data Tab refresh

+

Most filter / sort / pagination / type-badge / inline-edit features already exist. Real adds: breadcrumb, filter chip summary, schema sub-tabs, export, bulk action bar.

+
+
+ Mostly visual + gap-fill + Open mockup → +
+ + + +
+
+ ▶ Run · Format · Explain · SQL + SELECT c.id, c.name, COUNT(o.id) + FROM Customers c … + ───────────────────────────── + [Results · 1247] Plan Messages + id │ name │ orders + 1 │ Alice Smith │ 12 +
+ +
+
+

Query Tab refresh

+

Result sub-tabs (Results / Plan / Messages / Stats), Explain & Cancel buttons, query-history rail. Run / Format / Save / Designer / autocomplete already exist.

+
+
+ Adds + restyle + Open mockup → +
+ + + +
+
+
+ DESKTOP + [ first ][ last ][ email ] + [ status ][ tier ][ created ] + [ notes ──────────────────── ] +
+
+ 📱 + [first ] + [last ] + [email ] + [status] + [ Save ] +
+
+ +
+
+

Elastic Forms (mobile-friendly)

+

Forms render with absolute pixel coords today and don't reflow on phones. Add an Elastic layout mode + auto-stack so the same form works on desktop, tablet, and mobile — schema is already responsive-ready.

+
+
+ Mobile · forms project + Open mockup → +
+ + + +
+
+
+ PAPER · Letter + ID │ Cust │ Orders │ Total + ━━━━━━━━━━━━━━━━━━━━━━ + 1 │ A.S. │ 12 │ 4,950 + 4 │ D.S. │ 7 │ 2,723 +
+
+ 📱 Reader + ▾ Tier·Gold + ▦ Alice S. + 12 · $4,950 + ▦ David S. +
+
+ +
+
+

Mobile-friendly Reports (Reader view)

+

Reports are paper-shaped and shouldn't reflow. Add a derived Reader view: bands → sections, repeated rows → cards, group footers → summary lines. Phone defaults to Reader; Print & PDF stay on Paper.

+
+
+ Mobile · reports project + Open mockup → +
+ + + +
+
+
+ Tools + ▦ + T +
+
+ Detail band + {id} {name} {total} + [title][field] +
+
+ 📱 Reader + ▦ Alice + $4,950 +
+
+ +
+
+

Report Designer · Reader-view support

+

Paper / Reader / Split toggle in the toolbar, live phone preview alongside the paper canvas, Reader-hint subsection in the Selection inspector, and a one-click "Auto-derive all hints" + lint panel.

+
+
+ Designer · reports project + Open mockup → +
+ + + +
+
+ ▾ Storage + ► Summary + Database header + WAL + Space usage + Integrity issues ⚠ 3 + Vacuum / Reindex + Backup & Restore +
+ +
+
+

Heavy tabs → vertical sub-nav

+

Storage, Pipeline, and Procedure tabs each stack 4–11 long sections. Add a left rail with a Summary screen so users land on something useful and can jump without scrolling.

+
+
+ Pattern fix · cross-cutting + Open mockup → +
+ + +
+ +

Reality check: what's actually in the live admin today

+

After reading the actual tab components, the picture is more mature than my first pass assumed. Here's the delta per area.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
AreaAlready exists in the live adminGenuinely missing / proposed here
Welcome / first-runApp icon, name, four shortcut hints, table-card grid with optional row counts.NEW Stat cards, top tables with sparklines, recent activity, pinned grid, quick actions, health panel.
Object ExplorerTree of Tables / Forms / Reports / System Catalog / Views / Indexes / Triggers / Procedures with counts; text filter; right-click menus on every kind; collapse-all; resize handle; active-tab highlighting. + NEW Pinned & Recent sections, type filter chips, drill-down under each table (columns / indexes / triggers nested), match highlighting in filter results.
+ RESTYLE per-item star + ⋯, per-group "+" on hover. +
Data tabData ↔ Schema view switcher, Add Row / Delete (with count) / Save / Discard / Refresh, per-column filter row with Contains/StartsWith/EndsWith/Exact, sort indicators, type badges, PK badges, pagination, inline edit, row selection, NULL/BLOB rendering, modified-cell styling, right-click context menu. + NEW Breadcrumb header, filter summary chips, schema sub-tabs, export/import, bulk-edit + copy-as-INSERT, status-bar table facts.
+ RESTYLE column type icons (vs text badges), enum status pills, value-type cell coloring. +
Query tabRun / Analyze / Clear / Format / Save buttons, saved-query dropdown, Stats shortcuts, SQL ↔ Designer mode toggle, resizable editor with persisted height, line numbers, syntax highlighting, autocomplete popup with arrow-key nav, Ctrl+Enter, EXEC procedure parsing with named args.NEW Result sub-tabs (Results / Plan / Messages / Stats), Explain Plan button, Cancel button, query-history rail per-tab and global.
Storage tab11 stacked sections: DB header, WAL, Space usage, Fragmentation, Maintenance (Reindex / Vacuum), Backup / Restore, FK migration, Page-type histogram, Index checks, Integrity issues, Page drill-down. Refresh, hex dump toggle.NEW Vertical sub-nav rail, default Summary screen, status badges in nav, recent-maintenance recap.
Pipeline tabVisual designer + JSON mode, Validate / Dry Run / Run / Save / Reset, stored pipelines list with revisions, recent runs with metrics, run inspector with rejects, resume.NEW Sub-nav rail to navigate Designer / Run Result / Stored / History / Rejects without scrolling. (Same pattern as Storage.)
Procedure tabDefinition (name / desc / enabled / body SQL), parameters table (name/type/required/default/desc), execution panel with args JSON, per-statement results.NEW Sub-nav rail or sub-tabs for Definition / Parameters / Run / Results so the body editor isn't competing with the args panel.
Forms (Admin.Forms)Visual form designer with grid + snap, toolbox, layers panel, property inspector, child tabs, validation overrides, default form generation from a table schema, undo/redo, navigation, print, child datagrids. Schema model already has LayoutDefinition.Breakpoints and ControlDefinition.RendererHints; DesignerState already exposes IsVisibleAtBreakpoint and GetEffectiveRect. + NEW LayoutMode = Elastic on LayoutDefinition; runtime FormRenderer that emits CSS Grid spans (instead of position: absolute in pixels) and picks a breakpoint based on viewport.
+ NEW Designer toolbar gains Layout-mode + Device-preview segmented controls; property inspector gains per-breakpoint span / hidden flags; "controls overflow at this breakpoint" lint pill.
+ NEW Auto-elastic fallback for legacy forms — synthesize a 12-col span from the existing Rect.Width / canvasWidth so old forms get a usable mobile view without a redesign. +
Reports (Admin.Reports)Visual report designer with bands (ReportHeader / PageHeader / GroupHeader / Detail / GroupFooter / PageFooter / ReportFooter); Label / BoundText / CalculatedText / Line / Box controls; pixel-positioned within bands; preview pagination at fixed Letter 816 × 1056 px page; overflow-x: auto on the page surface; print stylesheet that hides toolbar; Export PDF; default report generation from a source schema; RendererHints on ReportDefinition and a PropertyBag on each control already available for extension. + NEW Reader view — derived from the same ReportPreviewResult; bands become collapsible sections, repeated Detail rows become cards, GroupFooter becomes a summary row. Auto-pick on phones; user toggle persists per report.
+ NEW Per-control reader hints in Props["readerHint"] (role: title / subtitle / field / hidden, custom label, showOnPhone). No schema migration.
+ NEW Paper-view zoom controls (Fit-width / 100% / 150% / 200%) and a sticky page pager so Paper is still tolerable on a phone when the user explicitly wants it.
+ EXISTS Print and Export PDF continue to use Paper unchanged. +
Reports designer3-column layout: Toolbox (Pointer / Label / BoundText / Calculated / Line / Box) + Groups + Sorts on the left; per-band paper canvas with absolute pixel positions and pointer drag-resize in the center; Page Settings (Paper / Orientation / 4 margins) + Selection inspector (X/Y/W/H, type-specific props, Bring/Send/Delete) on the right. Pointer-driven; no touch affordances. + NEW Toolbar Paper / Reader / Split view toggle + Reader-device toggle (Tablet / Phone). Split mode shows the live Reader projection beside the paper canvas.
+ NEW Selection inspector grows a Reader-view subsection (Role: title / subtitle / field / hidden, custom label, "show on phone", quick-action buttons). Stored in Props["readerHint"] — no schema migration.
+ NEW Tiny role badges on canvas controls show what the Reader pickup will be.
+ NEW "Auto-derive all hints" action seeds hints from inference rules; Reader-lint panel surfaces issues with one-click fixes.
+ DELIBERATE NON-GOAL The designer chrome itself stays desktop-only — the 3-column pointer-driven UI is right for building paper reports; only the Reader preview goes mobile-sized. +
Title / status barsLogo + db name, Open Database (path prompt), New Query / Table / Form / Procedure / Pipeline / Storage buttons, theme toggle, sidebar toggle, keyboard-shortcuts modal. Status bar shows connection, db path, table/view/proc counts.NEW Database switcher dropdown with recent databases, command-palette launcher, status-bar storage pressure / background-job ticker / notification counter.
Modals / toastsConfirm modal with danger styling, prompt modal, toast container with success / warning / error, context menu with separators and danger items.NEW Side drawers for non-blocking edits (e.g. column properties); rolling notification log accessible from the status bar.
+ +

Smaller wins worth doing

+
+
    +
  • Tab strip: dirty-dot indicator (already mocked), middle-click to close, overflow dropdown when there are too many tabs.
    • +
  • Density toggle (Compact / Comfortable) so the same screens work on a 13" laptop and a 27" monitor.
    • +
  • Empty states with calls-to-action — every empty list should suggest the next step ("No saved queries yet — press Ctrl+S in any query tab").
    • +
  • Inline help: a "?" button per tab that opens a side-panel cheat-sheet for the active tab, instead of a single global keyboard-shortcut modal.
    • +
  • Toast → inbox: keep a rolling notification log accessible from the status bar so users can re-read what happened.
    • +
+
+
+ + diff --git a/www/admin-ui-mockups/query-tab.html b/www/admin-ui-mockups/query-tab.html new file mode 100644 index 00000000..c1b19b4d --- /dev/null +++ b/www/admin-ui-mockups/query-tab.html @@ -0,0 +1,602 @@ + + + + +Query Tab — CSharpDB Studio Mockup + + + + + +
+ +
+
+ + CSharpDB Studio + — relational.db +
+ + +
+ Connected +
+ +
+ + +
+
+
Dashboard
+
Customers
+
Active customers
+ +
+ +
+ +
+ + + + + + + + +
+ + +
+
+ 1,247 rows · 18.4 ms · page 1 +
+
+ + +
+ Saved + + + + + + Stats + + +
+ + +
+
+ +
+
+ 1234567 +
+ 
-- Pull active customers with recent orders
+SELECT  c.id,
+        c.name,
+        c.email,
+        COUNT(o.id) AS order_count
+FROM Customers AS c
+LEFT JOIN Orders AS o ON o.customer_id = c.i
+ +
+
idint · pk
+
image_urltext
+
is_activeint (bool)
+
imported_atdatetime
+
+
+ +
+ + +
+
+ + + + +
+ Rows 1–25 of 1,247 + + + +
+
+ +
+ + + + + + + + + + + + + + + + + + + + +
idnameemailorder_countlast_order_at
1Alice Smithalice@example.com122026-04-22 09:14
2Bob Leebob@example.com32026-04-12 11:02
3Carol Smithcarol@example.com92026-04-19 16:48
4David Smithdsmith@example.com72026-04-19 08:22
5Eve Smitherseve@example.com222026-04-21 14:10
6Frank Smithyfrank@example.com12026-03-12 10:55
7Grace Smith-Jonesgrace@example.com52026-04-22 09:17
8Henry Smithfieldhank@example.com0—
+
+
+
+ + +
+
+ History + +
+
+ This tab + All + Errors +
+
+
+
✓ 18 ms · 1,247 rowsjust now
+
SELECT c.id, c.name, c.email, COUNT(o.id)…
+
+ Restore + Save + Copy +
+
+
+
✓ 4 ms · 1 row2m
+
SELECT COUNT(*) FROM Customers
+
+
+
✗ syntax error5m
+
SELECT * FROMM Orders
+
+
+
✓ 32 ms · 8,912 rows11m
+
SELECT * FROM Orders WHERE created_at > '2026-01-01'
+
+
+
✓ 9 ms · 6 rows34m
+
SELECT * FROM sys.table_stats
+
+
+
✓ 145 ms · 24,310 rows1h
+
SELECT * FROM OrderItems WHERE qty > 5
+
+
+
+
+
+
+
+ +
+ Connected + relational.db + Active customers · dirty + Last run 18 ms +
+ 87 queries today +
+
+ + +
+
Reality check vs. the live screen
+
    +
  • ALREADY EXISTS — Run / Analyze / Clear / Format buttons, SQL ↔ Designer mode toggle (a Visual Designer panel exists today!), saved-query name input + Save + Load dropdown + Refresh, Stats shortcuts (Table Stats / Column Stats), per-tab SQL persistence, resizable editor splitter with persisted height, line-numbered editor with syntax highlighting, autocomplete popup with arrow-key nav (Ctrl-Space to trigger explicitly), Ctrl+Enter to run, EXEC procedure parsing with named args, paged results that over-fetch one row to avoid blocking COUNT(*).
    • +
  • NEW: Result sub-tabs — Results / Plan / Messages / Stats. Today the result panel only shows rows. Plan would expose the existing query plan; Messages would collect non-query notices; Stats would show the existing elapsed/rows breakdown plus per-statement timings.
    • +
  • NEW: Explain Plan button in toolbar — runs the query in plan-only mode and switches to the Plan sub-tab.
    • +
  • NEW: Cancel button for in-flight queries.
    • +
  • NEW: History rail on the right — per-tab and global recent-queries list with status (✓/✗), elapsed, and row count. Hover reveals Restore / Save / Copy. Today there's no query history.
    • +
  • NEW: Result toolbar in the result tab strip — Export, Copy as JSON, Pop out results.
    • +
  • RESTYLE — toolbar visually groups primary action (Run) + cancel + secondary tools + view mode + result summary, instead of the current single flat row.
    • +
+
+ + diff --git a/www/admin-ui-mockups/reports-designer.html b/www/admin-ui-mockups/reports-designer.html new file mode 100644 index 00000000..3a53caad --- /dev/null +++ b/www/admin-ui-mockups/reports-designer.html @@ -0,0 +1,973 @@ + + + + +Report Designer · Reader-view support — CSharpDB Studio Mockup + + + + + +
+ +
CSharpDB.Admin.Reports · Designer-side proposal
+

Add Reader-view support to the report designer

+

+ The previous mockup (reports-mobile.html) added a Reader + view at runtime so phones aren't stuck side-scrolling an 816 px paper page. But if the + designer can't see or shape the Reader view, it's effectively invisible to the people + building reports — so the runtime view drifts from intent over time. +

+

+ This proposal adds three things to Pages/Designer.razor: a Paper / Reader / + Split view toggle in the toolbar, a Reader-view subsection in the Selection inspector, and + a Reader-lint summary panel that catches problems before users hit them on a phone. The + existing 3-column layout (Toolbox / Bands / Page-settings + Selection) stays. +

+ + +

1 · Designer toolbar gains a view-mode switch (Paper / Reader / Split)

+ +
+
+ + + + + + +
+ Designer view +
+ + + +
+
+ +
+ Reader device +
+ + +
+
+ +
+ 3 reader hints needed + Letter · Portrait +
+ +
+ + + + +
+
+ +
+
+ Page Header + +
+
+
+
+
+ Customer Sales — Q1 2026 +
+
+
+
+ + +
+
+ Group Header · tier + +
+
+
+
+
+ Tier · {tier} + title +
+
+
+ + +
+
+ Detail + +
+
+
+
+
+ {id} +
+
+ {customer_name} + title +
+
+ {orders} + field +
+
+ {avg_amount} + field +
+
+ {total_amount} + field +
+
+ {last_order_at} +
+
+
+ + +
+
+ Group Footer · tier + +
+
+
+
+
+ Subtotal · =SUM(orders) · =SUM(total_amount) +
+
+
+
+ + +
+
+ + Reader preview · phone · 380 px +
+ + +
+
+
+
+

Customer Sales — Q1 2026

+
vw_customer_sales · live preview
+
+
+ Tier · Gold + 3 +
+
+
Alice Smith
+
CUST-00001
+
+
Orders
12
+
Avg
$412.50
+
Total
$4,950.00
+
+
+
+
David Smith
+
CUST-00004
+
+
Orders
7
+
Avg
$389.00
+
Total
$2,723.00
+
+
+
+
+
+ + + +
+
+ + +

2 · Auto-derive Reader hints + lint warnings

+

+ For existing reports the designer can synthesize hints from the layout in one click. The + lint panel surfaces problems specific to Reader so they don't get discovered on a phone. +

+ +
+

+ Reader-view lint · Customer Sales — Q1 2026 + +

+ +
+ +
+ 3 BoundText controls have no Reader role assigned. +
Detail band · {orders}, {avg_amount}, {last_order_at}
+
+ +
+ +
+ +
+ No Card title set on Detail band. +
First BoundText {id} would auto-promote at runtime — set explicitly to avoid surprises.
+
+ +
+ +
+ +
+ 2 Line controls in Page Header will be hidden in Reader. +
Decorative controls (Line, Box) auto-hide. No action needed unless you want them shown.
+
+ +
+ +
+ +
+ Group Footer formula =SUM(orders) renders as a single line in Reader. +
Group Footer · tier · width 280 px on paper, full width on phone.
+
+ +
+
+ + +

3 · Implementation notes & non-goals

+ +
+
    +
  • + ● + Toolbar gains two segmented controls — Designer view (Paper / + Reader / Split) and, when Reader or Split is active, a + Reader-device toggle (Tablet / Phone). State persists per + report in localStorage; default is Split. +
    • +
  • + ● + Split mode reuses the runtime Reader projection. The reader pane on the + right is the same IReportReaderService output described in + reports-mobile.html, rendered into an iframe-like + container at the chosen device width. Edits on the paper canvas trigger a debounced + preview rebuild. +
    • +
  • + ● + Selection inspector grows a "Reader view" subsection — Role dropdown + (title / subtitle / field / hidden), custom label override, "show on + phone" toggle, and three quick-action buttons (Title / Field / Hide) for one-click + changes. Stored in ReportControlDefinition.Props["readerHint"] via the + existing PropertyBag — no schema migration. +
    • +
  • + ● + Controls on the canvas show a tiny role badge (title, + field, hidden) when a Reader hint is set — quick visual + map of what the Reader view will pick up. Toggle-able from a View menu. +
    • +
  • + ● + "Auto-derive all hints" runs the inference rules on the whole report — first + BoundText in Detail → title; remaining BoundText → fields with BoundFieldName as + the label; Line / Box → hidden; GroupHeader BoundText → section title; GroupFooter + CalculatedText → summary row. Pre-populates everything; users only refine. +
    • +
  • + ● + Lint panel appears below the designer (or as a collapsible footer). Each + item has a one-click fix. The "3 reader hints needed" pill in the toolbar is the + count summary; clicking it scrolls the lint panel into view. +
    • +
  • + ● + Page Settings stays paper-only. Reader has no concept of paper size / + margins — it's a continuous list. Margin/orientation controls don't move; they just + don't affect the reader pane. +
    • +
  • + ● + Non-goal · mobile-friendly designer. The 3-column desktop layout + (220 / flex / 320) and pointer-driven drag-resize are the right tools for building + a paper report. Making the designer touch-friendly would compromise it for a use + case that doesn't really exist (nobody designs a report on a phone). The Reader + preview pane goes mobile-sized, but the surrounding designer chrome stays + desktop-only. +
    • +
  • + ● + Out of scope — drag-and-drop within the Reader preview (Reader is read-only), + a separate "mobile report definition" (whole point is one definition / two views), + Reader-specific charts (current renderer only supports text / line / box anyway). +
    • +
+
+ +

+ Pairs with: reports-mobile.html (the runtime Reader view). + Back to all mockups. +

+
+ + diff --git a/www/admin-ui-mockups/reports-mobile.html b/www/admin-ui-mockups/reports-mobile.html new file mode 100644 index 00000000..9dac6e6b --- /dev/null +++ b/www/admin-ui-mockups/reports-mobile.html @@ -0,0 +1,817 @@ + + + + +Mobile-friendly Reports — CSharpDB Studio Mockup + + + + + +
+ +
CSharpDB.Admin.Reports · UI proposal
+

Mobile-friendly reports — keep paper, add a Reader view

+

+ Reports are different from forms: a report is supposed to look like paper. The current + renderer fixes an 816 × 1056 page surface and absolutely-positions every band's controls in + pixels (Pages/Preview.razor, reports.css). On a phone the page + side-scrolls. That's painful to read but right for print fidelity. +

+

+ The proposal: do not reflow the paper. Add a Reader view that the same + ReportDefinition produces — bands become sections, repeated detail rows become + cards, group footers become summary lines. Phone visitors get Reader by default, desktop + gets Paper, and the toolbar lets the user swap. Print & Export PDF always use Paper. +

+ + +

1 · Preview toolbar gains a View-mode switch

+ +
+
+
+
+ View +
+ + +
+
+
+ Page +
+ + + + +
+
+
+ Print uses Paper +
+
+
+ + + +
+
+ +
+
+
+

Customer Sales — Q1 2026

+
vw_customer_sales · 2026-04-26 · Letter · Portrait
+ +
+ IDCustomerOrdersAvg ($)Total ($) +
+ +
Tier · Gold
+
1Alice Smith12412.504,950.00
+
4David Smith7389.002,723.00
+
5Eve Smithers22221.004,862.00
+
Subtotal · 41 orders · $12,535.00
+ +
Tier · Silver
+
2Bob Lee3102.00306.00
+
7Grace Smith588.00440.00
+
Subtotal · 8 orders · $746.00
+ +
Customer SalesPage 1 of 4
+
+
tablet preview · Paper view · Letter portrait
+
+
+
+ + +
+

Reader-view settings

+ +
+ Reader enabled +
+
+
+ Auto on phone +
+
+
+ Group cards + Collapsible +
+
+ Card sort + Match Paper order +
+ +

Selected control

+ +
+ Field + customer_name +
+
+ Reader role + + + +
+
+ Reader label + +
+
+ Show on phone +
+
+ +

+ Stored in ControlDefinition.Props["readerHint"]. Decorative + Line / Box controls auto-hide in Reader. BoundText controls auto-promote: first + in the Detail band → card title, others → labelled fields. +

+
+
+ + +

2 · Same report on desktop, tablet, and phone

+

+ Desktop & tablet stay on Paper for fidelity. Phone defaults to Reader. +

+ +
+ +
+
+

Customer Sales — Q1 2026

+
vw_customer_sales · Letter · Portrait
+
+ IDCustomerOrdersAvg ($)Total ($) +
+
Tier · Gold
+
1Alice Smith12412.504,950.00
+
4David Smith7389.002,723.00
+
5Eve Smithers22221.004,862.00
+
Subtotal · 41 · $12,535.00
+
Tier · Silver
+
2Bob Lee3102.00306.00
+
7Grace S.588.00440.00
+
Subtotal · 8 · $746.00
+
+
desktop · Paper · ≥ 1024 px
+
+ + +
+
+

Customer Sales — Q1 2026

+
Fit-width · 75%
+
+ IDCustomerOrdersAvgTotal +
+
Tier · Gold
+
1Alice Smith124124,950
+
4David Smith73892,723
+
5Eve Smithers222214,862
+
Subtotal · $12,535
+
Tier · Silver
+
2Bob Lee3102306
+
7Grace S.588440
+
Subtotal · $746
+
+
tablet · Paper @ Fit-width · ≥ 640 px
+
+ + +
+
+
+
+

Customer Sales — Q1 2026

+
vw_customer_sales · 41 rows · 2 groups
+
+ +
+ Tier · Gold + 3 customers +
+ +
+
Alice Smith
+
CUST-00001
+
+
Orders
12
+
Avg
$412.50
+
Total
$4,950.00
+
+
+
+
David Smith
+
CUST-00004
+
+
Orders
7
+
Avg
$389.00
+
Total
$2,723.00
+
+
+ +
+ Subtotal · 41 orders + $12,535.00 +
+ +
+ Tier · Silver + 2 customers +
+ +
Customer Sales · 1–5 of 41 · scroll for more
+
+
phone · Reader (default) · < 640 px
+
+
+ + +

3 · Today vs. Reader on a 380 px phone

+ +
+ +
+
+
+
+

Customer Sales — Q1 2026

+
vw_customer_sales · 2026-04-26 · Letter
+
+ IDCustomerOrdersAvg ($)Total ($) +
+
Tier · Gold
+
1Alice Smith12412.504,950.00
+
4David Smith7389.002,723.00
+
5Eve Smithers22221.004,862.00
+
Subtotal · 41 orders · $12,535.00
+
+
+
+ + Page 1 of 4 · pinch to zoom + +
+
TODAY · Paper side-scrolls · text tiny · headers off-screen
+
+ + +
+
+
+
+

Customer Sales — Q1 2026

+
vw_customer_sales · 41 rows · 2 groups
+
+ +
+ Tier · Gold + 3 customers +
+ +
+
Alice Smith
+
CUST-00001
+
+
Orders
12
+
Avg
$412.50
+
Total
$4,950.00
+
+
+
+
David Smith
+
CUST-00004
+
+
Orders
7
+
Avg
$389.00
+
Total
$2,723.00
+
+
+
+
Eve Smithers
+
CUST-00005
+
+
Orders
22
+
Avg
$221.00
+
Total
$4,862.00
+
+
+ +
+ Gold subtotal · 41 + $12,535.00 +
+ +
+ + +
+
+
READER · readable · cards · footer summary · paper still one tap away
+
+
+ + +

4 · Implementation notes (no code changed)

+ +
+
    +
  • + ● + Don't touch Paper. The existing band-based, pixel-positioned layout from + Pages/Preview.razor stays exactly as-is for fidelity. Print and Export + PDF route through Paper unconditionally. +
    • +
  • + ● + Add a ViewMode selector to the preview toolbar: + Paper ↔ Reader. Auto-pick on first render based on + viewport (Reader below ~640 px, Paper above) — a single + matchMedia probe in JS interop is enough. User toggle persists per + report in localStorage. +
    • +
  • + ● + Reader is a derived view, not a second layout. A new + IReportReaderService takes the same ReportPreviewResult + that DefaultReportPreviewService already produces and walks the bands + semantically: +
    — ReportHeader / PageHeader → page intro (rendered once). +
    — GroupHeader → collapsible <section>. +
    — Detail band repeated → list of cards. First + BoundText in the band becomes the card title; others become + <dt>/<dd> pairs labelled with BoundFieldName. +
    — GroupFooter → summary row at the end of the section. +
    — Line / Box controls → hidden in reader. +
    • +
  • + ● + Per-control overrides via Props["readerHint"] — + { "role": "title" | "subtitle" | "field" | "hidden", "label": "Customer", + "showOnPhone": true }. Property inspector grows a Reader-view sub-section. + Schema unchanged; this is just data inside the existing PropertyBag. +
    • +
  • + ● + Auto-reader for legacy reports — the inference rules above mean any existing + ReportDefinition gets a usable Reader view with zero edits. Hints only + exist to refine it. +
    • +
  • + ● + Paper view on phone gets quality-of-life fixes too — toolbar gains + Fit-width / 100% / 150% / 200% zoom, and a sticky page pager (today the preview + stacks all pages vertically and lets each page surface side-scroll, see + reports.css .report-page-surface { overflow-x: auto; }). +
    • +
  • + ● + Out of scope: rich charts in Reader (current renderer only supports text / + line / box), interactive drill-down (Reader is read-only), and a separate "mobile + report definition" — the whole point is one definition, two views. +
    • +
+
+ +

+ Compare with the forms equivalent: forms-mobile.html. Back + to all mockups. +

+
+ + diff --git a/www/admin-ui-mockups/sidebar.html b/www/admin-ui-mockups/sidebar.html new file mode 100644 index 00000000..c731a489 --- /dev/null +++ b/www/admin-ui-mockups/sidebar.html @@ -0,0 +1,474 @@ + + + + +Object Explorer — CSharpDB Studio Mockup + + + + + +
+ +
+
+ + CSharpDB Studio + — relational.db +
+ + +
+ Connected + + +
+ +
+ + +
+
+
Customers
+
+
+
+ +

Hover items in the sidebar to see star & "more" actions.

+

Click a chevron next to a table to drill into its columns, indexes, and triggers.

+
+
+
+
+ +
+ Connected + relational.db + 24 tables · 6 views · 11 procs +
+ 2.4 GB / 3.5 GB +
+
+ + +
+
Reality check vs. the live screen
+
    +
  • ALREADY EXISTS — Object Explorer header with Collapse-All button, search/filter input, expandable groups (User Tables, System Tables, Forms, Reports, System Catalog, Views, Indexes, Triggers, Procedures), per-item count badges, active-tab highlight, right-click context menus for tables / forms / reports / views / indexes / triggers / procedures, drag-resize handle.
    • +
  • NEW: Pinned section — any item can be starred and persists at the top across sessions. Today there's no concept of pinning.
    • +
  • NEW: Recent section — last few opened objects with relative timestamps.
    • +
  • NEW: Type filter chips (All / Tables / Forms / Reports / More) for quick narrowing.
    • +
  • NEW: Drill-down under each table — a chevron expands columns (with type icons), indexes, and triggers as nested rows. Today indexes/triggers are scattered across separate top-level groups; columns require switching to the Schema view.
    • +
  • RESTYLE: Per-item hover actions — star + "⋯" appear on hover. The same operations are available today but only via right-click, which is undiscoverable for new users.
    • +
  • RESTYLE: Per-group "+" quick-add on hover (New Table, New Form, etc.) — same as the existing right-click "New …" items.
    • +
  • RESTYLE: Compact sort & refresh icons next to Collapse-All in the header. Refresh exists today but only on context menus.
    • +
  • NEW: Match highlighting in filter results (bold "Customers"). Today filter is text-substring match without highlighting.
    • +
+
+ + diff --git a/www/admin-ui-mockups/styles.css b/www/admin-ui-mockups/styles.css new file mode 100644 index 00000000..d5707d98 --- /dev/null +++ b/www/admin-ui-mockups/styles.css @@ -0,0 +1,355 @@ +/* ───────────────────────────────────────────────────────── + Shared design tokens & frame styles for mockups. + Tokens mirror src/CSharpDB.Admin/wwwroot/css/app.css so + the mockups feel like real previews of the admin shell. + ───────────────────────────────────────────────────────── */ + +:root { + /* Tokyo-Night-inspired dark palette (matches the live admin) */ + --bg-primary: #1a1b26; + --bg-secondary: #16171f; + --bg-tertiary: #1f2029; + --bg-elevated: #24253a; + --bg-hover: #292a3e; + --bg-active: #33345a; + --border-color: #2a2b3d; + --border-light: #363750; + --text-primary: #c0caf5; + --text-secondary: #7982a9; + --text-muted: #565f89; + --accent-blue: #7aa2f7; + --accent-cyan: #7dcfff; + --accent-green: #9ece6a; + --accent-yellow: #e0af68; + --accent-orange: #ff9e64; + --accent-red: #f7768e; + --accent-magenta: #bb9af7; + --accent-teal: #2ac3de; + --shadow-popup: 0 16px 48px rgba(0,0,0,0.55); + + --radius-sm: 4px; + --radius-md: 6px; + --radius-lg: 10px; + + --font-ui: 'Inter', -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; + --font-mono: 'JetBrains Mono', 'Fira Code', Consolas, monospace; +} + +* { box-sizing: border-box; margin: 0; padding: 0; } + +html, body { height: 100%; } + +body { + font-family: var(--font-ui); + background: var(--bg-primary); + color: var(--text-primary); + font-size: 13px; + line-height: 1.5; + -webkit-font-smoothing: antialiased; +} + +a { color: var(--accent-blue); text-decoration: none; } +a:hover { text-decoration: underline; } + +/* ─── App frame ─── */ +.app { + display: grid; + grid-template-rows: 38px 1fr 26px; + height: 100vh; +} + +.titlebar { + background: var(--bg-secondary); + border-bottom: 1px solid var(--border-color); + display: flex; + align-items: center; + gap: 12px; + padding: 0 12px; + user-select: none; +} + +.brand { + display: flex; + align-items: center; + gap: 8px; + font-weight: 600; +} + +.brand .logo { + width: 24px; height: 24px; + background: linear-gradient(135deg, var(--accent-blue), var(--accent-magenta)); + border-radius: 5px; + display: grid; + place-items: center; + font-size: 12px; + color: white; + font-weight: 700; +} + +.brand .dbname { color: var(--text-muted); font-weight: 400; font-size: 12px; } + +.tb-actions { + display: flex; + align-items: center; + gap: 4px; + padding-left: 12px; + margin-left: 0; + border-left: 1px solid var(--border-color); +} + +.tb-btn { + display: inline-flex; + align-items: center; + gap: 5px; + padding: 4px 10px; + background: transparent; + border: 1px solid transparent; + border-radius: var(--radius-sm); + color: var(--text-secondary); + font-size: 11px; + cursor: pointer; + white-space: nowrap; +} + +.tb-btn:hover { color: var(--text-primary); background: var(--bg-hover); } + +.tb-btn.icon-only { + width: 30px; height: 26px; + padding: 0; + justify-content: center; +} + +.spacer { flex: 1; } + +/* Database switcher dropdown trigger */ +.db-switcher { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 4px 10px; + background: var(--bg-tertiary); + border: 1px solid var(--border-color); + border-radius: var(--radius-sm); + color: var(--text-primary); + font-size: 12px; + cursor: pointer; +} + +.db-switcher:hover { border-color: var(--border-light); } + +.db-switcher .bi-caret-down-fill { font-size: 9px; color: var(--text-muted); } + +/* Command palette launcher in titlebar */ +.palette-launcher { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 4px 10px; + background: var(--bg-tertiary); + border: 1px solid var(--border-color); + border-radius: var(--radius-md); + color: var(--text-muted); + font-size: 12px; + min-width: 240px; + cursor: pointer; +} + +.palette-launcher:hover { border-color: var(--border-light); color: var(--text-secondary); } + +.palette-launcher .kbd { margin-left: auto; } + +.titlebar .conn-pill { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 3px 9px; + background: rgba(158,206,106,0.1); + border: 1px solid rgba(158,206,106,0.3); + color: var(--accent-green); + border-radius: 999px; + font-size: 11px; +} + +.dot { width: 6px; height: 6px; border-radius: 50%; background: var(--accent-green); } + +/* ─── Main split ─── */ +.body { + display: grid; + grid-template-columns: 280px 1fr; + overflow: hidden; + min-height: 0; +} + +.sidebar { + background: var(--bg-secondary); + border-right: 1px solid var(--border-color); + overflow: hidden; + display: flex; + flex-direction: column; +} + +.content { + background: var(--bg-primary); + overflow: auto; + display: flex; + flex-direction: column; +} + +/* ─── Tab strip (lightweight) ─── */ +.tabstrip { + display: flex; + background: var(--bg-secondary); + border-bottom: 1px solid var(--border-color); + height: 36px; + flex-shrink: 0; +} + +.tabstrip .tab { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 0 14px; + color: var(--text-muted); + font-size: 12px; + border-right: 1px solid var(--border-color); + border-top: 2px solid transparent; + cursor: pointer; +} + +.tabstrip .tab.active { + color: var(--text-primary); + background: var(--bg-primary); + border-top-color: var(--accent-blue); +} + +.tabstrip .tab .bi-x { color: var(--text-muted); margin-left: 4px; opacity: 0; } +.tabstrip .tab:hover .bi-x { opacity: 0.6; } +.tabstrip .tab.dirty::after { content: ""; width: 6px; height: 6px; background: var(--accent-blue); border-radius: 50%; margin-left: 4px; } + +.tabstrip .tab-new { + width: 32px; + border: none; + background: transparent; + color: var(--text-muted); + cursor: pointer; +} + +.tabstrip .tab-new:hover { color: var(--text-primary); background: var(--bg-hover); } + +/* ─── Status bar ─── */ +.statusbar { + background: var(--bg-secondary); + border-top: 1px solid var(--border-color); + display: flex; + align-items: center; + gap: 18px; + padding: 0 12px; + color: var(--text-muted); + font-size: 11px; +} + +.statusbar .pill { + display: inline-flex; + align-items: center; + gap: 5px; +} + +.statusbar .pill.warn { color: var(--accent-yellow); } +.statusbar .pill.ok { color: var(--accent-green); } +.statusbar .pill.action { color: var(--accent-blue); cursor: pointer; } +.statusbar .pill.action:hover { color: var(--accent-cyan); } + +/* ─── Reusable bits ─── */ +.kbd { + display: inline-flex; + align-items: center; + padding: 1px 6px; + font-family: var(--font-mono); + font-size: 10px; + background: rgba(255,255,255,0.06); + border: 1px solid rgba(255,255,255,0.08); + border-bottom-color: rgba(255,255,255,0.04); + border-radius: 4px; + color: var(--text-secondary); +} + +.btn { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 6px 12px; + border-radius: var(--radius-sm); + border: 1px solid var(--border-color); + background: var(--bg-tertiary); + color: var(--text-primary); + font-size: 12px; + cursor: pointer; +} + +.btn:hover { background: var(--bg-hover); } + +.btn.primary { background: var(--accent-blue); color: #0a0b13; border-color: var(--accent-blue); font-weight: 600; } +.btn.primary:hover { filter: brightness(1.08); } +.btn.ghost { background: transparent; } + +.icon-table { color: var(--accent-blue); } +.icon-system { color: var(--accent-cyan); } +.icon-view { color: var(--accent-magenta); } +.icon-trigger { color: var(--accent-orange); } +.icon-index { color: var(--accent-green); } +.icon-form { color: var(--accent-magenta); } +.icon-report { color: var(--accent-cyan); } + +/* ─── Annotations panel ─── */ +.notes { + position: fixed; + right: 18px; + bottom: 44px; + width: 320px; + max-height: 60vh; + overflow: auto; + background: var(--bg-elevated); + border: 1px solid var(--border-light); + border-radius: var(--radius-lg); + box-shadow: var(--shadow-popup); + z-index: 50; +} + +.notes header { + padding: 10px 14px; + border-bottom: 1px solid var(--border-color); + display: flex; + align-items: center; + gap: 8px; + font-size: 12px; + font-weight: 600; + color: var(--text-primary); +} + +.notes header .bi { color: var(--accent-yellow); } + +.notes ul { + list-style: none; + padding: 8px 0; +} + +.notes li { + padding: 8px 14px; + font-size: 12px; + color: var(--text-secondary); + border-bottom: 1px solid var(--border-color); +} + +.notes li:last-child { border-bottom: none; } + +.notes li b { color: var(--text-primary); font-weight: 600; } + +.notes .pin { + display: inline-block; + width: 6px; + height: 6px; + border-radius: 50%; + background: var(--accent-blue); + margin-right: 6px; + vertical-align: middle; +} From fa273001f692f09341a9faadb8371af55cd5ea13 Mon Sep 17 00:00:00 2001 From: Maximum Code Date: Mon, 27 Apr 2026 18:05:01 -0700 Subject: [PATCH 3/4] Add v3.5.0 PR notes --- docs/releases/v3.5.0-pr-notes.md | 147 +++++++++++++++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 docs/releases/v3.5.0-pr-notes.md diff --git a/docs/releases/v3.5.0-pr-notes.md b/docs/releases/v3.5.0-pr-notes.md new file mode 100644 index 00000000..7580cf09 --- /dev/null +++ b/docs/releases/v3.5.0-pr-notes.md @@ -0,0 +1,147 @@ +## Summary + +This `v3.5.0` release prepares the collection binary payload fast path, the +release benchmark refresh, and the confirmed CSharpDB Studio admin UI mockup +docs that are included on the branch. + +The collection work completes the opt-in source-generated collection fast path +around fixed generated field order, compact type/null metadata, and raw value +payloads. Existing non-generated collection paths continue to use their current +JSON and binary document behavior. The runtime path now avoids duplicated direct +payload header parsing, adds single-segment `ReadOnlySpan` top-level field +lookups, and uses targeted UTF-8 span plumbing for text index/read/compare +paths. + +The benchmark work adds generated collection codec coverage, records the +collection payload investigation, and refreshes the published benchmark README +from the April 26, 2026 release-core artifacts. The final release guardrail +compare passed with `PASS=185, WARN=0, SKIP=0, FAIL=0`. + +The admin UI work adds forms/reports access-parity notes plus static CSharpDB +Studio admin UI mockups under `www/admin-ui-mockups`. + +## Type of Change + +- [ ] Bug fix +- [x] New feature +- [ ] Breaking change +- [x] Documentation update +- [x] Refactor / maintenance +- [ ] Tests only + +## Related Issues + +No issue numbers were linked for this branch. Included work in this release: + +- Completed the roadmap item for source-generated collection fast path + completion. +- Added generated collection binary payload encode/decode around fixed field + order, compact type/null metadata, and raw values. +- Kept non-generated collection paths unchanged. +- Added direct binary payload decode improvements and top-level span lookup + support. +- Added targeted UTF-8 span plumbing for collection text index/read/compare + paths. +- Added generated collection codec benchmarks and expanded collection binary + codec/generator tests. +- Refreshed release-core benchmark documentation and history. +- Added CSharpDB Studio admin UI access-parity docs and static mockups. + +## Testing + +- [x] `dotnet build CSharpDB.slnx` +- [x] Relevant tests executed +- [x] Failure-path tests executed (if applicable: cancellation, invalid/unsupported inputs, non-`DbException` paths) +- [x] Manual verification performed (if applicable) + +Validation performed: + +- `dotnet build CSharpDB.slnx -c Release --no-restore` +- `dotnet test CSharpDB.slnx -c Release --no-build -m:1 -- RunConfiguration.DisableParallelization=true` +- Non-parallel unit test run passed with `1,652` tests. +- `dotnet run -c Release --project .\tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --release-core --repeat 3 --repro` +- `pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Run-Perf-Guardrails.ps1 -Mode release` +- `pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Compare-Baseline.ps1 -ThresholdsPath .\tests\CSharpDB.Benchmarks\perf-thresholds.json -CurrentMicroResultsDir .\tests\CSharpDB.Benchmarks\results\.tmp-current-micro-run -ReportPath .\tests\CSharpDB.Benchmarks\results\perf-guardrails-last.md` +- `pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Update-BenchmarkReadme.ps1 -RunManifest .\tests\CSharpDB.Benchmarks\release-core-manifest.json` +- `Get-Content -Raw .\tests\CSharpDB.Benchmarks\release-core-manifest.json | ConvertFrom-Json` +- `pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Update-BenchmarkReadme.ps1 -RunManifest .\tests\CSharpDB.Benchmarks\release-core-manifest.json -DryRun` +- `git diff --check -- tests\CSharpDB.Benchmarks\README.md tests\CSharpDB.Benchmarks\HISTORY.md tests\CSharpDB.Benchmarks\release-core-manifest.json` + +Benchmark validation details: + +- Initial release guardrail wrapper completed benchmark collection but failed + compare with volatile samples: + `PASS=174, WARN=0, SKIP=0, FAIL=11`. +- Focused retry refreshed the volatile `CollationIndexBenchmarks`, + `CollectionIndexBenchmarks`, and `CompositeIndexBenchmarks` micro CSVs. +- Final official compare passed: + `PASS=185, WARN=0, SKIP=0, FAIL=0`. + +Focused collection investigation: + +| Metric | Result | +|--------|-------:| +| Matched rows vs same-machine HEAD baseline | `60` | +| Faster matched rows | `50` | +| Slower matched rows | `10` | +| Median matched speedup | `+4.1%` | +| Mean matched speedup | `+4.8%` | + +Focused recovery highlights: + +| Benchmark | Before | After | Change | +|-----------|-------:|------:|-------:| +| Collection field read, missing field | `223.22 ns` | `136.73 ns` | `+38.7%` | +| Collection decode, direct payload | `333.80 ns` | `155.20 ns` | `+53.5%` | +| Collection field read, early field | `108.60 ns` | `49.77 ns` | `+54.2%` | +| Collection field compare, late text field | `159.55 ns` | `97.60 ns` | `+38.8%` | +| Collection field compare, bound accessor | `128.03 ns` | `92.83 ns` | `+27.5%` | + +Path-index follow-up highlights: + +| Benchmark | Final vs same-machine HEAD baseline | +|-----------|------------------------------------:| +| Nested path equality via `FindByIndex` | `+53.6%` | +| Nested path equality via `FindByPath` | `+55.9%` | +| Array path equality via `FindByIndex` | `+52.1%` | +| Text range path lookup | `+42.7%` | +| Guid equality path lookup | `+59.4%` | + +Published scorecard examples from the refreshed benchmark README: + +| Area | Result | +|------|-------:| +| SQL file-backed single insert | `450.4 ops/sec` | +| SQL file-backed batch x100 | `41.88K rows/sec` | +| Collection file-backed put | `447.3 ops/sec` | +| Collection file-backed batch x100 | `42.28K docs/sec` | +| Collection hot point get | `1.60M ops/sec` | +| CSharpDB InsertBatch B1000 | `233.06K rows/sec` | + +## Checklist + +- [x] I followed the project style and conventions. +- [x] I added or updated tests for behavior changes. +- [x] I covered both success and failure paths for changed behavior. +- [x] I updated docs for user-facing changes. +- [x] I verified no sensitive data was added. + +## Notes for Reviewers + +- The highest-risk runtime changes are in the generated collection model and + collection payload codec paths: + `CollectionModelGenerator`, `CollectionPayloadCodec`, + `CollectionBinaryDocumentCodec`, `CollectionDocumentCodec`, + `CollectionIndexedFieldReader`, and collection index binding. +- The generator diff is intentionally large because generated code now owns the + binary record encode/decode shape for opt-in generated models. +- Existing non-generated collection document behavior should remain on the + existing JSON and binary document paths. +- The benchmark README generated region should be edited through + `release-core-manifest.json` plus `scripts/Update-BenchmarkReadme.ps1`, not + manually. +- The release-core source artifacts were not replaced by targeted micro runs. + The targeted retry only refreshed volatile guardrail micro CSVs before + rerunning the official compare. +- The admin UI mockup commit is included because the branch owner confirmed it + should be part of this PR. From 765c208b331f3ce61bae56e07949e7d6549b259e Mon Sep 17 00:00:00 2001 From: Maximum Code Date: Mon, 27 Apr 2026 18:10:13 -0700 Subject: [PATCH 4/4] Update root release notes for v3.5.0 --- RELEASE_NOTES.md | 290 ++++++++++++++++++++--------------------------- 1 file changed, 122 insertions(+), 168 deletions(-) diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index cd51a756..dd327245 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,173 +1,127 @@ # What's New -## v3.4.0 - -v3.4.0 focuses on daemon packaging, cross-platform deployment, and remote host -consolidation without changing SQL, storage, WAL, query planning, or the gRPC -client contract. - -### Remote Host Consolidation - -- `CSharpDB.Daemon` now hosts both the existing REST `/api` surface and gRPC - from one long-running process. -- REST and gRPC requests share the same warm daemon-hosted database client, so - remote users no longer need separate REST and gRPC host processes for the - same database. -- Plain `http://` daemon endpoints support `Transport = Grpc` through - gRPC-Web compatibility so gRPC and REST can share the default service URL. - HTTPS and custom test clients can continue using native gRPC. -- REST is enabled in the daemon by default and can be disabled with - `CSharpDB__Daemon__EnableRestApi=false`. -- The standalone `CSharpDB.Api` REST host remains supported for REST-only - deployments. -- Release archive smoke validation now calls the daemon REST `/api/info` - endpoint and a gRPC `GetInfoAsync` client after extracting and starting each - daemon binary. - -### Admin Warm Local Database Hosting - -- `CSharpDB.Admin` direct local mode keeps a warm in-process database instance - and now opens it through hybrid incremental-durable database options by - default. -- Admin startup and database switching both use the same host database option - builder, so opening a different database from the UI keeps the same warm - local database behavior. -- Admin remote mode still uses `CSharpDB:Transport` plus `CSharpDB:Endpoint` - without attaching local direct/hybrid options. -- Set `CSharpDB__HostDatabase__OpenMode=Direct` to opt back into the older - plain direct open path for local Admin runs. -- Admin table data filters now support contains, starts-with, and ends-with - `LIKE` placement plus exact `=` match mode per column. -- The Admin SQL query editor now has homegrown guided completions for SQL - keywords, table/view selection, select-list columns, qualified alias columns, - and stored procedure names without adding a third-party editor dependency. -- The Admin SQL query tab now exposes a visible vertical splitter so the SQL - editor and results pane can be resized when longer queries need more working - space. -- The visual query designer now exposes its own splitter so the generated SQL - preview can be expanded against the results section instead of staying pinned - to a fixed preview height. -- The form designer property inspector now renders the selected control ID with - theme-aware display styling instead of inheriting browser-native readonly - input colors that were hard to read in the dark theme. - -### Daemon Service Packaging - -- Added Windows Service and systemd host integration to `CSharpDB.Daemon`. - These hooks are no-ops for normal console and `dotnet run` execution. -- Added Windows service install/uninstall scripts with defaults for - `CSharpDBDaemon`, `C:\Program Files\CSharpDB\Daemon`, - `C:\ProgramData\CSharpDB`, and `http://127.0.0.1:5820`. -- Added Linux systemd service template plus install/uninstall scripts with - defaults for `/opt/csharpdb-daemon`, `/var/lib/csharpdb`, service user - `csharpdb`, and `http://127.0.0.1:5820`. -- Added macOS launchd plist template plus install/uninstall scripts with - defaults for `/usr/local/lib/csharpdb-daemon`, - `/usr/local/var/csharpdb`, and `http://127.0.0.1:5820`. - -### Release Archives - -- Added `scripts/Publish-CSharpDbDaemonRelease.ps1` for self-contained, - single-file, non-trimmed daemon release archives. -- Added release archive coverage for `win-x64`, `linux-x64`, and `osx-arm64`. -- Added checksum generation through `SHA256SUMS.txt`. -- Updated the GitHub Release workflow to build daemon archives on native - Windows, Linux, and macOS runners, verify each archive, smoke-start the - extracted daemon, and attach the archives plus combined checksums to the - GitHub Release. - -### Docs - -- Updated the daemon README with archive installation, service installation, - upgrade, uninstall, and configuration override guidance. -- Updated the Admin README with warm in-process local database behavior, hybrid - local hosting defaults, and the direct open-mode opt-out. -- Updated the scripts README with daemon packaging and service installer - references. -- Updated the roadmap to mark daemon service packaging done and scoped - cross-platform daemon archive deployment in progress. -- Added a new blog post covering the C# launcher pattern for - `CSharpDB.Admin`, including syntax-highlighted examples for the launcher - executable flow. -- Migrated the remaining source-heavy markdown docs into companion reference - pages under `www` for architecture, getting started, performance, SQL query - execution pipeline, SQL reference, storage engine, roadmap, and the - CSharpDB-versus-SQLite benchmarking article so the full original content now - stays published on the website. -- Updated the curated docs/blog pages and sitemap to point at the new source - reference routes when users need the full original long-form content. -- Removed the duplicated markdown copies of the CLI, REST API, MCP server, - internals, and storage inspector guides after their website versions were - audited and verified. - -### Samples And Forms Runtime - -- Added `samples/fulfillment-hub`, a runnable end-to-end warehouse and order - fulfillment sample that seeds tables, indexes, views, triggers, procedures, - saved queries, Admin forms, Admin reports, stored pipelines, typed - collections, and a full-text index into one database. -- The Fulfillment Hub sample includes an operational workbook plus a narrative - README walkthrough that teaches the platform through receiving, allocation, - shipment, returns, audit, pipeline, collection, and full-text flows instead - of a flat feature list. -- Added a forms-only runtime host at `src/CSharpDB.Admin.Forms.Web` that lists - stored forms and runs them without exposing the form designer. -- The forms runtime host points at any target CSharpDB database through - `CSharpDB:DataSource`, `ConnectionStrings:CSharpDB`, or `CSharpDB:Endpoint` - and reuses the existing `DataEntry` runtime component from - `CSharpDB.Admin.Forms`. -- `CSharpDB.Admin.Forms` now supports runtime-only hosting through optional - `ShowDesignerButton`, `BackHref`, and `BackLabel` parameters on - `DataEntry.razor`, while keeping the existing Admin studio behavior unchanged - by default. +## v3.5.0 + +v3.5.0 focuses on the collection binary payload fast path, generated +collection codec performance, targeted UTF-8 span plumbing, and refreshed +release benchmark publishing. It also includes the confirmed CSharpDB Studio +admin UI access-parity notes and static mockups that are part of this branch. + +### Collection Binary Payload Fast Path + +- Added the opt-in source-generated collection fast path for fixed generated + field order, compact type/null metadata, and raw value payloads. +- Existing non-generated collection paths continue to use their current JSON + and binary document behavior. +- Generated collection models can now use direct binary record encode/decode + instead of routing through the slower document-shaped path. +- `CollectionDocumentCodec` now parses direct binary payload headers once + and decodes the key/document from that parsed header. +- `CollectionBinaryDocumentCodec` now has single-segment + `ReadOnlySpan` lookup overloads for top-level binary document fields. +- `CollectionPayloadCodec` fast header parsing now favors the common binary + payload marker path. +- Collection field and index bindings now expose generated accessors to faster + direct field-reader paths where available. + +### UTF-8 Text Index And Compare Paths + +- Added targeted UTF-8 span plumbing for collection text index/read/compare + paths. +- Reduced transient allocations in top-level string property reads by avoiding + per-call `byte[]` and `byte[][]` path materialization where the + single-segment path applies. +- Updated ordered text index key comparison coverage. + +### Tests And Benchmarks + +- Added `GeneratedCollectionCodecBenchmarks`. +- Expanded generated collection model tests around binary payload support. +- Expanded binary document codec tests for direct field access. +- Added ordered text index key codec coverage. +- Refreshed `tests/CSharpDB.Benchmarks/README.md` and + `release-core-manifest.json` from the April 26, 2026 release-core artifacts. +- Recorded the collection binary payload investigation, noisy initial guardrail + compare, focused retry, and final passing release guardrail in + `tests/CSharpDB.Benchmarks/HISTORY.md`. + +Focused collection investigation: + +| Metric | Result | +|--------|-------:| +| Matched rows vs same-machine HEAD baseline | `60` | +| Faster matched rows | `50` | +| Slower matched rows | `10` | +| Median matched speedup | `+4.1%` | +| Mean matched speedup | `+4.8%` | + +Focused recovery highlights: + +| Benchmark | Before | After | Change | +|-----------|-------:|------:|-------:| +| Collection field read, missing field | `223.22 ns` | `136.73 ns` | `+38.7%` | +| Collection decode, direct payload | `333.80 ns` | `155.20 ns` | `+53.5%` | +| Collection field read, early field | `108.60 ns` | `49.77 ns` | `+54.2%` | +| Collection field compare, late text field | `159.55 ns` | `97.60 ns` | `+38.8%` | +| Collection field compare, bound accessor | `128.03 ns` | `92.83 ns` | `+27.5%` | + +Path-index follow-up highlights: + +| Benchmark | Final vs same-machine HEAD baseline | +|-----------|------------------------------------:| +| Nested path equality via `FindByIndex` | `+53.6%` | +| Nested path equality via `FindByPath` | `+55.9%` | +| Array path equality via `FindByIndex` | `+52.1%` | +| Text range path lookup | `+42.7%` | +| Guid equality path lookup | `+59.4%` | + +Published scorecard examples from the refreshed benchmark README: + +| Area | Result | +|------|-------:| +| SQL file-backed single insert | `450.4 ops/sec` | +| SQL file-backed batch x100 | `41.88K rows/sec` | +| Collection file-backed put | `447.3 ops/sec` | +| Collection file-backed batch x100 | `42.28K docs/sec` | +| Collection hot point get | `1.60M ops/sec` | +| CSharpDB InsertBatch B1000 | `233.06K rows/sec` | + +### CSharpDB Studio Admin UI Notes + +- Added admin forms access-parity notes. +- Added admin reports access-parity notes. +- Added static CSharpDB Studio admin UI mockups under `www/admin-ui-mockups`. +- Included dashboard, data, query, heavy operations, reports designer, mobile + forms/reports, command palette, sidebar, and shared styling mockups. ### Validation -- PowerShell parser validation passed for the daemon release publisher and - Windows install/uninstall scripts. -- `bash -n` passed for Linux and macOS service install/uninstall scripts. -- `dotnet restore CSharpDB.slnx` completed successfully. -- `dotnet build CSharpDB.slnx -c Release` completed successfully. -- `dotnet build CSharpDB.slnx -c Release --no-restore` completed successfully. -- `dotnet test tests\CSharpDB.Api.Tests\CSharpDB.Api.Tests.csproj -c Release --no-build` - passed with `15` tests. -- `dotnet test tests\CSharpDB.Daemon.Tests\CSharpDB.Daemon.Tests.csproj -c Release --no-build` - passed with `18` tests. -- `dotnet test tests\CSharpDB.Admin.Forms.Tests\CSharpDB.Admin.Forms.Tests.csproj -c Release --no-build` - passed with `211` tests. -- Content audit confirmed the new source-reference pages preserve the migrated - markdown coverage (100% heading coverage and 99.7-100% token overlap across - the migrated set). -- `python -c "import xml.etree.ElementTree as ET; ET.parse('www/sitemap.xml')"` - passed for the updated site map. -- Repo scan found no remaining references to the deleted duplicated markdown - docs. -- `dotnet run --project samples\fulfillment-hub\FulfillmentHubSample.csproj` - completed successfully and seeded `3` forms, `3` reports, `5` procedures, - `5` saved queries, `3` stored pipelines, `2` collections, and the - `fts_ops_playbooks` full-text index into the sample database. -- `dotnet build src\CSharpDB.Admin.Forms.Web\CSharpDB.Admin.Forms.Web.csproj` - completed successfully. -- `dotnet run --project src\CSharpDB.Admin.Forms.Web\CSharpDB.Admin.Forms.Web.csproj -- --urls http://127.0.0.1:5095 --CSharpDB:DataSource=` - started successfully, listed the seeded sample forms at `/`, and served the - runtime-only `orders-workbench` form route at `/forms/orders-workbench` - without the designer action. -- `.\scripts\Publish-CSharpDbDaemonRelease.ps1 -Version 3.4.0 -Runtime win-x64 -OutputRoot artifacts\daemon-release-local` - created `csharpdb-daemon-v3.4.0-win-x64.zip` and `SHA256SUMS.txt`. -- The extracted `win-x64` daemon archive smoke-started successfully, served - `/api/info`, and accepted a gRPC `GetInfoAsync` client call on the same base - URL with a temporary database. -- `dotnet run --project src\CSharpDB.Api\CSharpDB.Api.csproj --configuration Release --no-build --no-launch-profile` - smoke-started successfully and served `/api/info` with a temporary database. -- `dotnet run --project src\CSharpDB.Daemon\CSharpDB.Daemon.csproj --configuration Release --no-build --no-launch-profile` - smoke-started successfully, served `/api/info`, and accepted a gRPC - `GetInfoAsync` client call on the same base URL with a temporary database. -- `dotnet run --project src\CSharpDB.Admin\CSharpDB.Admin.csproj --configuration Release --no-build --no-launch-profile` - smoke-started successfully in direct hybrid incremental-durable mode with a - temporary database. -- `dotnet build src\CSharpDB.Admin.Forms\CSharpDB.Admin.Forms.csproj` - completed successfully after the form designer property inspector styling - changes. -- `dotnet build src\CSharpDB.Admin\CSharpDB.Admin.csproj -p:BaseOutputPath=artifacts\verify\` - completed successfully after the query tab and visual designer splitter - changes. +- `dotnet build CSharpDB.slnx -c Release --no-restore` +- `dotnet test CSharpDB.slnx -c Release --no-build -m:1 -- RunConfiguration.DisableParallelization=true` +- Non-parallel unit test run passed with `1,652` tests. +- `dotnet run -c Release --project .\tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --release-core --repeat 3 --repro` +- `pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Run-Perf-Guardrails.ps1 -Mode release` +- `pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Compare-Baseline.ps1 -ThresholdsPath .\tests\CSharpDB.Benchmarks\perf-thresholds.json -CurrentMicroResultsDir .\tests\CSharpDB.Benchmarks\results\.tmp-current-micro-run -ReportPath .\tests\CSharpDB.Benchmarks\results\perf-guardrails-last.md` +- `pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Update-BenchmarkReadme.ps1 -RunManifest .\tests\CSharpDB.Benchmarks\release-core-manifest.json` +- `Get-Content -Raw .\tests\CSharpDB.Benchmarks\release-core-manifest.json | ConvertFrom-Json` +- `pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Update-BenchmarkReadme.ps1 -RunManifest .\tests\CSharpDB.Benchmarks\release-core-manifest.json -DryRun` +- `git diff --check -- tests\CSharpDB.Benchmarks\README.md tests\CSharpDB.Benchmarks\HISTORY.md tests\CSharpDB.Benchmarks\release-core-manifest.json` + +Release benchmark guardrail result: + +```text +Compared 185 rows against baseline. PASS=185, WARN=0, SKIP=0, FAIL=0 +``` + +### Review Notes + +- The highest-risk runtime changes are in the generated collection model and + collection payload codec paths: `CollectionModelGenerator`, + `CollectionPayloadCodec`, `CollectionBinaryDocumentCodec`, + `CollectionDocumentCodec`, `CollectionIndexedFieldReader`, and collection + index binding. +- The generator diff is intentionally large because generated code now owns the + binary record encode/decode shape for opt-in generated models. +- The benchmark README generated region should be edited through + `release-core-manifest.json` plus `scripts/Update-BenchmarkReadme.ps1`, not + manually.