Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ test: ## run tests with coverage
go test -race -coverprofile=coverage.out $$(go list ./... | grep -v '/examples/')
go tool cover -func=coverage.out | sort -rnk3

.PHONY: bench
bench: ## run sqlite benchmark suite with allocation metrics
go test -run '^$$' -bench . -benchmem -count=3 ./pkg/rain

.PHONY: test-json
test-json: ## run tests with JSON output (for CI)
go test -json -race -coverprofile=coverage.out $$(go list ./... | grep -v '/examples/') > test-report.jsonl
Expand Down
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ A type-safe, SQL-like ORM for Go inspired by DrizzleORM — lightweight, fast, a
* [Project Layout](#project-layout)
* [Quick Start](#quick-start)
* [Examples](#examples)
* [Performance Benchmarks](#performance-benchmarks)
* [Makefile Targets](#makefile-targets)
* [Contribute](#contribute)

Expand Down Expand Up @@ -227,10 +228,41 @@ sqlType := d.DataType(schema.ColumnType{DataType: "string", Size: 255}) // VARC

See the [examples/](examples/) directory for complete, runnable examples.

# Performance Benchmarks

Rain includes a SQLite-first benchmark suite for measuring end-to-end ORM performance and memory usage across representative CRUD and join workloads.

Run the full suite:

```sh
make bench
```

Run a single workload:

```sh
go test -run '^$' -bench 'BenchmarkSQLiteSelectJoinScan' -benchmem ./pkg/rain
```

Run one workload for one dataset size:

```sh
go test -run '^$' -bench 'BenchmarkSQLiteSelectJoinScan/medium$' -benchmem ./pkg/rain
```

Compare two runs over time by saving the benchmark output and diffing the benchmark lines from the same machine and environment. Use the built-in Go metrics as the primary signals:

- `ns/op` shows the average execution time per benchmark iteration.
- `B/op` shows the average bytes allocated per iteration.
- `allocs/op` shows the average number of heap allocations per iteration.

The suite seeds deterministic `small`, `medium`, and `large` SQLite datasets before measurements start so setup cost does not pollute the reported ORM metrics.

# Makefile Targets

```sh
$> make
bench run sqlite benchmark suite with allocation metrics
bootstrap download tool and module dependencies
build build the library (verifies compilation)
clean clean up test artifacts
Expand Down
60 changes: 60 additions & 0 deletions docs/adr/2026-03-29-sqlite-performance-benchmark-suite.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# ADR: SQLite-first ORM performance benchmark suite

## Status
Accepted on 2026-03-29.

## Context
Rain had integration coverage for SQLite-backed ORM flows, but no benchmark suite for measuring end-to-end latency and allocation behavior. That left the project without a repeatable way to inspect how query construction, SQL compilation, execution, and scan paths behave under realistic ORM workloads.

The first benchmark suite needs to be practical for engineers running locally, deterministic enough to compare runs over time, and narrow enough to ship without introducing multi-dialect infrastructure or profiling workflows that are not yet required.

## Decision
Add a SQLite-first benchmark suite in `pkg/rain` using Go's native benchmark runner.

### Scope

- Measure end-to-end ORM execution rather than builder-only compilation.
- Focus on developer diagnostics instead of pass/fail CI regression thresholds.
- Use Go benchmark metrics as the memory signal: `ns/op`, `B/op`, and `allocs/op`.

### Workloads

- Single-row insert via `.Model(...)`
- Single-row insert via `.Set(...)`
- Point lookup select and struct scan
- Filtered select into a slice
- Bulk scan into a slice
- Join scan across aliased `users` and `posts` tables

### Dataset defaults

- `small`: 100 users / 1,000 posts
- `medium`: 1,000 users / 10,000 posts
- `large`: 10,000 users / 100,000 posts

### Harness rules

- Each benchmark dataset runs against an isolated SQLite database.
- Schema creation and deterministic data seeding happen before `b.ResetTimer()`.
- Seeded row counts are validated before measurements begin.
- Benchmarks stay on the public ORM API and reuse the same table definitions as integration tests.

## Deferred work

- Raw `database/sql` baseline comparisons
- `pprof` heap and CPU profile capture
- Postgres and MySQL benchmark backends
- CI thresholds for time or allocation regressions

## Consequences
### Positive

- Gives Rain a stable local benchmark entrypoint with realistic ORM workloads.
- Makes allocation behavior visible without adding extra tooling.
- Keeps extension paths open for future dialect backends and baseline comparisons.

### Negative

- Benchmark coverage is limited to SQLite in v1.
- Insert benchmarks still include normal table growth during the measured run.
- Results are intended for trend analysis, not absolute cross-machine comparisons.
Loading
Loading