perf: batch changelog insertion and deletion pg queries

[adityachoudhari26]

Summary by CodeRabbit

• New Features
  • Added changelog tracking to record and manage entity changes within workspaces.
  • Enabled batch processing of changelog operations (up to 500 items per batch) for improved performance and efficiency.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces a batch-based changelog persistence system for the workspace-engine database. It adds a new changelog_entry table, defines SQL queries for upserting, deleting, and listing changelog entries, generates corresponding Go types and batch operation methods, and refactors the Store persistence layer to execute changelog operations in batches of 500 items rather than individually.

Changes

Cohort / File(s)	Summary
Database Schema & Configuration apps/workspace-engine/pkg/db/queries/schema.sql, apps/workspace-engine/pkg/db/sqlc.yaml	Adds new changelog_entry table with composite primary key (workspace_id, entity_type, entity_id) and JSONB entity_data field. Registers changelog.sql in SQLC configuration for code generation.
SQL Queries apps/workspace-engine/pkg/db/queries/changelog.sql	Introduces three SQL query templates: UpsertChangelogEntry (INSERT...ON CONFLICT UPDATE), DeleteChangelogEntry, and ListChangelogEntriesByWorkspace (SELECT ordered by created_at).
Generated Go Code apps/workspace-engine/pkg/db/changelog.sql.go, apps/workspace-engine/pkg/db/models.go, apps/workspace-engine/pkg/db/db.go	Generates ListChangelogEntriesByWorkspaceRow struct and corresponding query method. Adds ChangelogEntry model struct. Extends DBTX interface with SendBatch method for batch execution support.
Batch Operations Implementation apps/workspace-engine/pkg/db/batch.go	Implements batch operation abstractions with DeleteChangelogEntryBatchResults/UpsertChangelogEntryBatchResults types, corresponding Params types, and Exec/Close methods. Adds ErrBatchAlreadyClosed error sentinel.
Persistence Layer apps/workspace-engine/pkg/db/persistence/store.go	Refactors Save to execute upserts and deletes in 500-item batches using new batch APIs. Refactors Load to query changelog via ListChangelogEntriesByWorkspace and iterate over typed rows. Adds workspace ID validation and batch-level error tracing.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• chore: psql changelog #702: Implements the same PostgreSQL changelog feature with identical schema, SQL queries, and Store/Save/Load persistence logic.

Suggested reviewers

• jsbroks

Poem

A rabbit bundles changelog cheer,
Batching entries, five hundred per sphere,
Upsert and delete in chunks so tight,
Persistence dances through database night! 🐰✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The pull request title clearly and specifically describes the main change: implementing batch-based SQL operations for changelog insertion and deletion to improve performance.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch batch-changelog-queries

---

No actionable comments were generated in the recent review. 🎉

🧹 Recent nitpick comments

apps/workspace-engine/pkg/db/persistence/store.go (3)

144-145: Prefer var declaration for initially-empty slices.

make([]persistence.Change, 0) allocates a backing array that may never be used (e.g., if all changes are of one type). A nil slice behaves identically for append and len checks.

Suggested fix

-	setEntries := make([]persistence.Change, 0)
-	unsetEntries := make([]persistence.Change, 0)
+	var setEntries []persistence.Change
+	var unsetEntries []persistence.Change

---

162-176: Double-wrapped error messages will produce verbose output.

upsertChangelogEntries already wraps with "failed to upsert changelog entry: ...", and then Save wraps again with "failed to upsert changes: ...". This yields messages like:

failed to upsert changes: failed to upsert changelog entry: <pg error>

Consider either removing the inner wrap (returning the raw error with span recording only) or removing the outer wrap.

---

207-211: Silently skipping unmarshal failures could hide data issues.

If Unmarshal fails for a registered entity type (as opposed to a genuinely unknown type), this continue will silently drop the entry with no trace. Consider logging or recording a span event for observability.

Suggested improvement

 	for _, row := range rows {
 		entity, err := jsonEntityRegistry.Unmarshal(row.EntityType, row.EntityData)
 		if err != nil {
+			span.AddEvent("skipped_changelog_entry", trace.WithAttributes(
+				attribute.String("entity_type", row.EntityType),
+				attribute.String("entity_id", row.EntityID),
+				attribute.String("error", err.Error()),
+			))
 			continue
 		}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}