Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<CentralPackageTransitivePinningEnabled>true</CentralPackageTransitivePinningEnabled>
<SharpCoreDBVersion>2.0.0.1</SharpCoreDBVersion>
<SharpCoreDBVersion>2.0.0.2</SharpCoreDBVersion>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="SharpCoreDB" Version="$(SharpCoreDBVersion)" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<LangVersion>14.0</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<IsPackable>false</IsPackable>
<UseLocalSharpCoreDbSources Condition="'$(UseLocalSharpCoreDbSources)' == '' and Exists('..\..\..\src\SharpCoreDB.CQRS\SharpCoreDB.CQRS.csproj')">true</UseLocalSharpCoreDbSources>
<UseLocalSharpCoreDbSources Condition="'$(UseLocalSharpCoreDbSources)' == ''">false</UseLocalSharpCoreDbSources>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<LangVersion>14.0</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<IsPackable>false</IsPackable>
<UseLocalSharpCoreDbSources Condition="'$(UseLocalSharpCoreDbSources)' == '' and Exists('..\..\..\src\SharpCoreDB\SharpCoreDB.csproj')">true</UseLocalSharpCoreDbSources>
<UseLocalSharpCoreDbSources Condition="'$(UseLocalSharpCoreDbSources)' == ''">false</UseLocalSharpCoreDbSources>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<LangVersion>14.0</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<IsPackable>false</IsPackable>
<UseLocalSharpCoreDbSources Condition="'$(UseLocalSharpCoreDbSources)' == '' and Exists('..\..\..\src\SharpCoreDB.EventSourcing\SharpCoreDB.EventSourcing.csproj')">true</UseLocalSharpCoreDbSources>
<UseLocalSharpCoreDbSources Condition="'$(UseLocalSharpCoreDbSources)' == ''">false</UseLocalSharpCoreDbSources>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<LangVersion>14.0</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<IsPackable>false</IsPackable>
<!-- Set to true to use local source projects instead of NuGet packages -->
<UseLocalSharpCoreDbSources Condition="'$(UseLocalSharpCoreDbSources)' == '' and Exists('..\..\..\src\SharpCoreDB\SharpCoreDB.csproj')">true</UseLocalSharpCoreDbSources>
Expand Down
72 changes: 72 additions & 0 deletions docs/2.0.0.2_WHAT_CHANGED.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# SharpCoreDB 2.0.0.2 — What changed & the 2.x engine vs the 1.9.x line

- **Release:** `2.0.0.2` (2026-09-04) — 4-part patch release on the 2.0.0.x line.
- **Quality gate:** full suite **1770 tests / 0 failed**, CI green on Windows/Ubuntu/macOS +
CodeQL + SonarCloud + Server Compatibility Smoke for every merged PR.
- **Contents:** everything merged since 2.0.0.1 — the perf/hardening batch of PRs #376-#388 —
plus the consolidated 2.x-vs-1.9.x overview below.
- See `docs/CHANGELOG.md` → `[2.0.0.2]` for the detailed change list and
`docs/2.0.0.0_WHAT_CHANGED.md` for the full 2.0.0.0 report (this document is the short,
forward-looking summary).

## The major steps: 2.x engine vs the 1.9.x engine

### 1. Storage & on-disk format (directory + single-file)
| Step | 1.9.x | 2.x |
|---|---|---|
| Record layout | variable-length length-prefixed rows only | **Fixed-width record layout** (constant stride + out-of-line `.ovf` overflow arena) for columnar PK tables; **default for new PK tables** since B7; per-table flag **persisted in metadata** (authoritative on reopen) with **1.x → 2.0 auto-migration** (opt-in via `FixedWidthRecordLayout`) |
| DELETE durability | logical delete + full-file rewrite at flush | **Commit-time tombstone markers** (in-place negative length-prefix) → durable in O(delete), no full-file rewrite |
| In-place writes | only same-length patches, buffered later | **Buffered in-place overwrites flushed per storage page** (C6); markers also batched per page (C5) |
| Single-file (`.scdb`) | fixed-layout registry, compression bugs | **Format v2** (block registry/FSM/table directory), crash-safe auto-migration, block-level Brotli/GZip/Zstd, envelope encryption, concurrent-write corruption fix (2.0.0.1) |

### 2. Performance engine (the “performance-first” rewrite)
- **Single-pass contiguous UPDATE/DELETE (B8/B9):** strictly ascending `pk = literal` batches on
plaintext fixed-width PK tables are resolved and applied as ONE contiguous range — and since M3
the probe verifies the span by decoding each record’s PK slot instead of doing ~10K B-tree
searches per batch; PK entries are removed with one descending `DeleteBulk`.
- **Whole-file DML resolution & key-only decode (B1/W1):** batch UPDATE/DELETE resolve all targets
from a single in-memory snapshot decoding only PK/hash columns (no per-row preads).
- **Legacy variable-length fast path (#380):** strictly-ascending INTEGER-PK batches resolve with a
sequential decode pass + early exit (unordered files fall back safely).
- **Index maintenance:** PK deletes via `IIndex.DeleteBulk` (descending); duplicate-key hash removal
no longer quadratic (P5).
- **Batch INSERT (WP14, 2.0.0.1):** column-ordered `object[]` inserts — no per-row dictionaries.
- **Structured canonical batch parsing (B3/B9)** and, from 2.0.0.1, in-place field patches (WP11).

### 3. Measurement discipline
- Fair-PK harness reports **median of 3 per phase** (D1) and gained a **same-window interleaved
A/B mode** (`--pk-ab`) so config comparisons are attributable; a **pure-default-config arm**
(`--pk-default`) exists to guard the out-of-the-box story.

### 4. Reliability, compatibility & hardening
- **1.x reads, 2.x writes forward-compat guarantee documented**, including the **downgrade policy**
(files with commit-time tombstone markers are not readable by pre-marker binaries) and the
recommended read-only-first upgrade order — see `docs/manual/upgrade-and-downgrade.md`.
- **Default engine selection hardened:** `StorageEngineType.Auto` never silently lands on
PageBased (not yet OLTP-ready: ~26K UPDATE ops/s vs ~245K on the fixed-width Columnar path);
PageBased remains explicit opt-in.
- 4-part patch versioning (`n.n.n.n`) since 2.0.0.0.

## Benchmark snapshot (fair-PK, tuned harness config, median-of-3, same machine)

| | SharpCoreDB (Columnar fixed-width) | SQLite | gap |
|---|---:|---:|---:|
| UPDATE | ~163-245K ops/s | ~270-315K | ~0.8-1.7x of SQLite (layout/run dependent) |
| DELETE | ~106-172K ops/s | ~353-420K | ~2.1-3.5x |
| INSERT | ~125-152K ops/s | ~186-190K | competitive |
| READ | ~72-110K ops/s | ~95-107K | competitive |

**Honest caveats for this release:**
- The **default config** (`NoEncryptMode=false`) runs ~1.3-1.6x slower than `NoEncryptMode=true`
on ALL phases: `Storage` keeps two encryption layers and the default still pays AES work in the
file-level read/page-cache wrappers even though per-record at-rest encryption
(`EnableAtRestRecordEncryption`) is off. Root cause quantified with same-window A/B (P3d) and
tracked as the next optimization; **FullSync was ruled out** as a cause.
- These numbers are synthetic ascending-PK batches; other shapes fall back to correct but slower
generic paths.

## Upgrade & downgrade

Follow `docs/manual/upgrade-and-downgrade.md`: back up → read-only open → read-write open →
verify. **Downgrading below the marker-introducing 2.x release is not supported** for databases
that already contain commit-time tombstone markers.
5 changes: 5 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]


## [2.0.0.2] - 2026-09-04

> Full release notes incl. the 2.x-vs-1.9.x major steps: [`docs/2.0.0.2_WHAT_CHANGED.md`](2.0.0.2_WHAT_CHANGED.md).

### Hardening

- **NoEncryptMode root cause quantified + explained (P3d)** - definitive 3-rep same-window `--pk-ab`
Expand Down
5 changes: 5 additions & 0 deletions docs/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,11 @@ guide explaining when SharpCoreDB is fastest.

## 10. Benchmarks & Performance

- `2.0.0.2_WHAT_CHANGED.md` — ⭐ **release notes (2.0.0.2)**: the 2.x-vs-1.9.x major steps + the
2.0.0.2 hardening/perf batch
- `CHANGELOG.md` — full per-version change log (`[2.0.0.2]`, new `[Unreleased]` on top)
- `manual/upgrade-and-downgrade.md` — compatibility matrix & downgrade policy
- `benchmarks/default-config-pk.md` — default-config fair-PK benchmark + NoEncryptMode root cause
- `manual/performance.md` — ⭐ **Performance Guide**: when SharpCoreDB is fastest + v2.0 results
- `performance/V2_PERFORMANCE_PLAN.md` — v2.x performance roadmap, root-cause analysis, .NET 11 plan
- `benchmarks/SHARPCOREDB_COMPARATIVE_BENCHMARKS.md` — comparative report vs SQLite/LiteDB/BLite
Expand Down
9 changes: 6 additions & 3 deletions docs/manual/upgrade-and-downgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,10 @@ Recommended upgrade order:
- Planned (CI): a true **cross-version** job that writes a database with a pinned older commit and
reads it with `master` (requires an old-binary generator; tracked as follow-up).

## Changelog
## Changelog & release notes

See `docs/CHANGELOG.md` → `[Unreleased]` → **Hardening** for the marker/downgrade notes that
accompanied the tombstone work (PRs #367/#368) and this policy document.
- `docs/CHANGELOG.md` → `[2.0.0.2]` → **Hardening** for the marker/downgrade notes that accompanied
the tombstone work (PRs #367/#368) and the 2.0.0.2 hardening batch.
- `docs/2.0.0.2_WHAT_CHANGED.md` — full release notes incl. the consolidated
**2.x-vs-1.9.x major steps** table.
- `docs/2.0.0.0_WHAT_CHANGED.md` — the original 2.0.0.0 report.
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Analytics/SharpCoreDB.Analytics.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>

<!-- NuGet Package Properties -->
<PackageId>SharpCoreDB.Analytics</PackageId>
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.CQRS/SharpCoreDB.CQRS.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<NoWarn>$(NoWarn);1591;SA1101;SA1633;SA1309</NoWarn>

<PackageId>SharpCoreDB.CQRS</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.CQRS</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<!-- Assembly info -->
<AssemblyName>SharpCoreDB.Client.Protocol</AssemblyName>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Description>Client-side protocol implementation for SharpCoreDB (gRPC, binary)</Description>

<!-- NuGet Package Properties -->
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Client/SharpCoreDB.Client.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<!-- Assembly info -->
<AssemblyName>SharpCoreDB.Client</AssemblyName>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Description>.NET client library for SharpCoreDB network server (ADO.NET-like API)</Description>

<!-- NuGet packaging -->
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
<PackageProjectUrl>https://github.com/MPCoreDeveloper/SharpCoreDB</PackageProjectUrl>
<RepositoryUrl>https://github.com/MPCoreDeveloper/SharpCoreDB</RepositoryUrl>
<PackageTags>ado.net;database;data-provider;sharpcoredb</PackageTags>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<RepositoryType>git</RepositoryType>
<PackageReadmeFile>NuGet.README.md</PackageReadmeFile>
<PackageIcon>SharpCoreDB.jpg</PackageIcon>
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Distributed/SharpCoreDB.Distributed.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

<!-- NuGet Package Properties -->
<PackageId>SharpCoreDB.Distributed</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Distributed</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

<!-- NuGet Package Properties -->
<PackageId>SharpCoreDB.EntityFrameworkCore</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.EntityFrameworkCore</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<NoWarn>$(NoWarn);1591;SA1101;SA1633;SA1309</NoWarn>

<PackageId>SharpCoreDB.EventSourcing</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.EventSourcing</Product>
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Extensions/SharpCoreDB.Extensions.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

<!-- NuGet Package Properties -->
<PackageId>SharpCoreDB.Extensions</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Extensions</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<Nullable>enable</Nullable>

<PackageId>SharpCoreDB.Functional.Dapper</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Functional.Dapper</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<Nullable>enable</Nullable>

<PackageId>SharpCoreDB.Functional.EntityFrameworkCore</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Functional.EntityFrameworkCore</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<Nullable>enable</Nullable>

<PackageId>SharpCoreDB.Functional.Linq2DB</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Functional.Linq2DB</Product>
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Functional/SharpCoreDB.Functional.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<Nullable>enable</Nullable>

<PackageId>SharpCoreDB.Functional</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Functional</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<LangVersion>latest</LangVersion>
<AssemblyName>SharpCoreDB.Graph.Advanced</AssemblyName>
<RootNamespace>SharpCoreDB.Graph.Advanced</RootNamespace>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<PackageId>SharpCoreDB.Graph.Advanced</PackageId>
<Authors>MPCoreDeveloper</Authors>
<Company>MPCoreDeveloper</Company>
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Graph/SharpCoreDB.Graph.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
<NoWarn>$(NoWarn);1591;SA1101;SA1633;SA1309</NoWarn>

<PackageId>SharpCoreDB.Graph</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Graph</Product>
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Identity/SharpCoreDB.Identity.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
<GenerateDocumentationFile>true</GenerateDocumentationFile>

<PackageId>SharpCoreDB.Identity</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Identity</Product>
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Projections/SharpCoreDB.Projections.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<NoWarn>$(NoWarn);1591;SA1101;SA1633;SA1309</NoWarn>

<PackageId>SharpCoreDB.Projections</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Projections</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<!-- NuGet Package Properties -->
<PackageId>SharpCoreDB.Provider.Sync</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Provider.Sync</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

<!-- NuGet Package Properties -->
<PackageId>SharpCoreDB.Provider.YesSql</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Provider.YesSql</Product>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

<!-- NuGet Package Metadata -->
<PackageId>SharpCoreDB.Serilog.Sinks</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.Serilog.Sinks</Product>
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Server.Core/SharpCoreDB.Server.Core.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<!-- Assembly info -->
<AssemblyName>SharpCoreDB.Server.Core</AssemblyName>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Description>Core server infrastructure for SharpCoreDB network server</Description>

<!-- NuGet Package Properties -->
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<!-- Assembly info -->
<AssemblyName>SharpCoreDB.Server.Protocol</AssemblyName>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Description>Network protocol definitions for SharpCoreDB server (gRPC, binary, HTTP)</Description>

<!-- NuGet Package Properties -->
Expand Down
2 changes: 1 addition & 1 deletion src/SharpCoreDB.Server/SharpCoreDB.Server.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<AssemblyName>sharpcoredb-server</AssemblyName>
<Product>SharpCoreDB Network Server</Product>
<Description>SharpCoreDB network database server with gRPC and HTTP support</Description>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Copyright>Copyright (c) 2026 MPCoreDeveloper</Copyright>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

<!-- NuGet Package Properties -->
<PackageId>SharpCoreDB.VectorSearch</PackageId>
<Version>2.0.0.1</Version>
<Version>2.0.0.2</Version>
<Authors>MPCoreDeveloper</Authors>
<Company>SharpCoreDB</Company>
<Product>SharpCoreDB.VectorSearch</Product>
Expand Down
Loading
Loading