Skip to content

feat: sqlite benchmark#12

Merged
cungminh2710 merged 2 commits into
mainfrom
minhc/sqlite-bench
Mar 29, 2026
Merged

feat: sqlite benchmark#12
cungminh2710 merged 2 commits into
mainfrom
minhc/sqlite-bench

Conversation

@cungminh2710
Copy link
Copy Markdown
Contributor

What this PR for?

What did you do for validating the changes?

Anything else you want to add? (Optional)

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 29, 2026

Greptile Summary

This PR introduces a SQLite-first benchmark suite for Rain ORM, covering six representative CRUD and join workloads (BenchmarkSQLiteInsertModel, BenchmarkSQLiteInsertSet, BenchmarkSQLiteSelectPointLookup, BenchmarkSQLiteSelectFilteredSlice, BenchmarkSQLiteSelectBulkScan, BenchmarkSQLiteSelectJoinScan) across three deterministic dataset sizes (small, medium, large). It also refactors the shared test helpers in sqlite_integration_test.go to accept testing.TB instead of *testing.T, enabling benchmarks to reuse the same schema-creation and DB-open utilities.

  • Benchmark harness: Each sub-benchmark creates an isolated temp-file SQLite DB, seeds it in a single batched transaction (500 rows/batch), validates row counts, then calls b.ResetTimer() — keeping setup cost out of the measured path.
  • tx = nil guard: The deferred rollback in seedSQLiteBenchmarkData correctly skips cleanup after a successful commit, avoiding a spurious double-rollback error.
  • testing.TB refactor: Widening openSQLiteTestDB and createSQLiteSchema from *testing.T to testing.TB is a clean, non-breaking change; all existing integration tests continue to work unchanged.
  • Documentation: The ADR and README clearly document workloads, dataset defaults, known limitations (e.g. insert benchmarks grow the table during measurement, results are for trend analysis rather than absolute comparisons), and deferred work items.
  • Minor suggestion: The make bench target (line 24 of Makefile) does not specify -count=N. Since the README recommends comparing saved benchmark outputs over time, adding -count=3 would give each metric multiple samples and reduce run-to-run noise.

Confidence Score: 5/5

Safe to merge — all remaining feedback is P2 style suggestions with no correctness or data-integrity concerns.

The benchmark harness is structurally sound: setup is correctly isolated before b.ResetTimer(), the tx-nil guard prevents double-rollback, helper functions are properly generalised to testing.TB, and known limitations are documented in the ADR. The single P2 comment (missing -count flag in Makefile) does not affect correctness.

No files require special attention.

Important Files Changed

Filename Overview
pkg/rain/sqlite_benchmark_test.go New benchmark suite covering insert (Model/Set), point lookup, filtered slice, bulk scan, and join scan across three dataset sizes; setup correctly precedes b.ResetTimer(), tx-nil guard avoids double-rollback after commit, all patterns are idiomatic.
pkg/rain/sqlite_integration_test.go openSQLiteTestDB and createSQLiteSchema generalised from *testing.T to testing.TB so benchmarks can share them; all existing integration tests remain unchanged.
Makefile Adds bench target that runs benchmarks with -benchmem; minor suggestion to add -count for reproducible comparison across runs.
README.md Documents benchmark usage, available metrics, and dataset size semantics; consistent with the ADR and the Makefile target.
docs/adr/2026-03-29-sqlite-performance-benchmark-suite.md ADR clearly records scope, workloads, dataset defaults, harness rules, and deferred work; known limitations (growing table during insert benchmarks, single-sample defaults) are explicitly called out.

Sequence Diagram

sequenceDiagram
    participant Runner as Go Benchmark Runner
    participant BF as BenchmarkFunc (e.g. BenchmarkSQLiteInsertModel)
    participant RSBD as runSQLiteBenchmarkDatasets
    participant NSBF as newSQLiteBenchmarkFixture
    participant DB as SQLite (temp file)

    Runner->>BF: call with b *testing.B
    BF->>RSBD: pass run closure
    loop for each dataset (small / medium / large)
        RSBD->>RSBD: b.Run(dataset.name, ...)
        RSBD->>NSBF: newSQLiteBenchmarkFixture(b, dataset)
        NSBF->>DB: openSQLiteTestDB → temp file DB
        NSBF->>DB: createSQLiteSchema (CREATE TABLE users, posts)
        NSBF->>DB: seedSQLiteBenchmarkData (batched INSERT in single tx)
        NSBF->>DB: validateSQLiteBenchmarkData (COUNT users, COUNT posts)
        NSBF-->>RSBD: *benchmarkFixture
        RSBD->>BF: invoke run closure (b, fixture, dataset)
        BF->>BF: b.ReportAllocs() + b.ResetTimer()
        loop for i := range b.N
            BF->>DB: ORM operation (insert / select / join)
            DB-->>BF: result
        end
    end
    Runner-->>Runner: report ns/op, B/op, allocs/op
Loading
Prompt To Fix All With AI
This is a comment left during a code review.
Path: Makefile
Line: 24

Comment:
**Consider adding `-count=N` for more reproducible comparisons**

The README explicitly recommends comparing saved benchmark outputs over time. Without `-count`, each `make bench` invocation produces a single sample per sub-benchmark, which means run-to-run noise can look like a real regression or improvement. Adding `-count=3` (or `-count=5`) gives the Go benchmark runner multiple samples to average over, making trend comparisons more reliable.

```suggestion
	go test -run '^$$' -bench . -benchmem -count=3 ./pkg/rain
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "." | Re-trigger Greptile

Comment thread Makefile Outdated
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@cungminh2710 cungminh2710 merged commit ac5d01b into main Mar 29, 2026
2 checks passed
@cungminh2710 cungminh2710 deleted the minhc/sqlite-bench branch March 29, 2026 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant