version3.7.0 release

[MaxAkbar]

Summary

This version3.7.0 release completes the current optimizer and async I/O
close-out work, adds public planner observability, introduces opt-in adaptive
join reoptimization, and fixes the view/query paths that motivated the
customer-filtered fulfillment investigation.

The query-planning work adds EXPLAIN ESTIMATE and sys.planner_* diagnostic
catalogs, then exposes those diagnostics in the Admin Query tab. It also adds
phase-one adaptive join reoptimization behind explicit opt-in settings so
eligible index nested-loop and hash joins can correct bad cardinality
assumptions before rows are emitted while default query behavior remains
unchanged.

The view/admin work makes paged view browsing use bounded LIMIT/OFFSET
plans, lets simple view row-goal planning reorder eligible join chains before
the view operator tree is built, fixes Query tab grid scrolling, and preserves
SQL column type metadata across engine, API, client, and Admin result surfaces.

The release also closes the current generated collection fast-path and DataGen
direct-load phase, refreshes release benchmark docs and scorecards, adds
optimizer/async I/O close-out diagnostics, and adds a dedicated WAL point-read
benchmark plus configurable guardrail metric comparison for sub-microsecond
latency rows.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Refactor / maintenance
• Tests only

Related Issues

No issue numbers were linked for this release branch. Included work:

• Added public planner diagnostics through EXPLAIN ESTIMATE FOR SELECT, WITH,
  and compound queries.
• Added sys.planner_* virtual catalogs for histograms, heavy hitters, and
  composite index prefix statistics.
• Added Admin Query tab estimate execution and Plan tab rendering.
• Added opt-in adaptive query reoptimization through DatabaseOptions,
  EnableAdaptiveQueryReoptimization(...), and direct embedded ADO.NET
  connection strings.
• Added adaptive join diagnostics and fail-closed handling for unsupported or
  risky shapes.
• Added row-goal reordering for eligible simple view join chains.
• Updated Admin DataGrid view paging to issue bounded LIMIT/OFFSET SQL.
• Fixed Query tab grid scrolling and pager layout.
• Improved lookup-join planning for indexed local predicates, residual
  right-side filters, and index scan capacity hints.
• Propagated SQL result ColumnTypes through local engine transport, HTTP API,
  HTTP client, and Admin DataGrid.
• Added a fulfillment sample orders(customer_id) lookup index.
• Closed the current generated collection fast-path phase and refreshed related
  roadmap/static pages.
• Moved DataGen direct loads onto the write-optimized storage preset and
  reduced direct-load row-buffer overhead.
• Added optimizer close-out and async I/O close-out diagnostic benchmark suites.
• Refreshed release benchmark README tables, benchmark catalog entries, and the
  release-core manifest with the May 6 baseline.
• Added a WAL point-read BenchmarkDotNet suite and guardrail support for
  comparing latency columns such as P95 instead of only Mean.

Testing

• dotnet build CSharpDB.slnx
• Relevant tests executed
• Failure-path tests executed (if applicable: cancellation, invalid/unsupported inputs, non-DbException paths)
• Manual verification performed (if applicable)

Validation reported across the commit range:

• dotnet test .\CSharpDB.slnx -c Release
• dotnet build .\CSharpDB.slnx -c Release --no-restore
• dotnet test .\CSharpDB.slnx -c Release --no-build
• dotnet run -c Release --project .\tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --optimizer-closeout --repro
• dotnet run -c Release --project .\tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --async-io-closeout --repro
• pwsh -NoProfile .\tests\CSharpDB.Benchmarks\scripts\Run-Perf-Guardrails.ps1 -Mode release
  • Reported PASS=187, WARN=0, SKIP=0, FAIL=0.
• dotnet build .\src\CSharpDB.Admin\CSharpDB.Admin.csproj -c Release
• dotnet test .\tests\CSharpDB.Admin.Forms.Tests\CSharpDB.Admin.Forms.Tests.csproj -c Release --filter FullyQualifiedName~DataGridTests
• dotnet test .\tests\CSharpDB.Tests\CSharpDB.Tests.csproj -c Release --filter FullyQualifiedName~SimpleViewLateUnindexedDetailJoinWithLimit|FullyQualifiedName~JoinChainWithLimit|FullyQualifiedName~PurchaseOrderLineJoinChainWithLimit
• Targeted generated collection tests, trim smoke publish, DataGen Release
  build, relational direct-load smoke, and document direct-load smoke.
• dotnet build tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -c Release --no-restore
• dotnet run -c Release --project tests\CSharpDB.Benchmarks\CSharpDB.Benchmarks.csproj -- --micro --filter *WalPointReadBenchmarks* --job Dry
• Focused lookup-join and SQL column type tests for:
  Join_WithWhereOnRightPrimaryKeyLookupSide_AppliesPredicate,
  Join_WithUniqueTextFilterAndIndexedDependentSide_UsesLookupJoins, and
  ExecuteSqlAsync_ReturnsQueryColumnTypes.

Checklist

• I followed the project style and conventions.
• I added or updated tests for behavior changes.
• I covered both success and failure paths for changed behavior.
• I updated docs for user-facing changes.
• I verified no sensitive data was added.

Notes for Reviewers

• Adaptive reoptimization is opt-in. Default query planning and execution remain
  unchanged unless a host enables the feature.
• Remote endpoint connection strings reject the adaptive reoptimization key;
  remote hosts must enable the feature server-side.
• EXPLAIN ESTIMATE and sys.planner_* are public diagnostic surfaces and
  should stay isolated from normal select execution side effects.
• Simple view row-goal planning is intentionally scoped to eligible join chains
  and bounded browsing shapes.
• The SQL result ColumnTypes DTO addition is additive, but client/UI consumers
  should handle missing values from older servers.
• The WAL point-read benchmark gives future releases a cleaner signal, but it
  does not yet have a historical baseline captured as a guardrail.