Skip to content

Comments

feat: compress sqlite db#2278

Closed
MasterPtato wants to merge 1 commit into03-26-feat_graceful_worker_shutdownfrom
03-27-feat_compress_sqlite_db
Closed

feat: compress sqlite db#2278
MasterPtato wants to merge 1 commit into03-26-feat_graceful_worker_shutdownfrom
03-27-feat_compress_sqlite_db

Conversation

@MasterPtato
Copy link
Contributor

@MasterPtato MasterPtato commented Mar 27, 2025

Changes

This was referenced Mar 27, 2025
Closed
Closed
@MasterPtato Graphite App
Copy link
Contributor Author

MasterPtato commented Mar 27, 2025 •
edited
Loading

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

How to use the Graphite Merge Queue

Add the label merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

greptile-apps[bot]
greptile-apps bot reviewed Mar 27, 2025
View reviewed changes
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR adds SQLite compression using lz4_flex and updates snapshot, key, error, and metric handling in our pools module, while also refining logging in runtime shutdown and actor routes.

• Added lz4_flex dependency (v0.11.3) and updated snapshot logic in packages/common/pools/src/db/sqlite/mod.rs to compress and chunk database files.
• Introduced CompressedDbDataKey/CompressedDbDataChunkKey in packages/common/pools/src/db/sqlite/keys.rs and a new COMPRESSED_DATA constant in packages/common/fdb-util/src/keys.rs.
• Added new Lz4 error variant in packages/common/pools/src/error.rs and new upload/download metrics in packages/common/pools/src/metrics.rs (verify naming conflicts).
• Adjusted flush handling in chirp-workflow to log failures without forcing shutdown.
• Enhanced runtime shutdown observability in packages/common/runtime/src/lib.rs.

10 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 27, 2025 •
edited
Loading

Deploying rivet with  Cloudflare Pages  Cloudflare Pages

Latest commit: 910b7c0
Status: ✅  Deploy successful!
Preview URL: https://22e3b801.rivet.pages.dev
Branch Preview URL: https://03-27-feat-compress-sqlite-d.rivet.pages.dev

View logs

@MasterPtato MasterPtato force-pushed the 03-27-feat_compress_sqlite_db branch 2 times, most recently from dee60d0 to 2c0ca5c Compare March 27, 2025 20:53
NathanFlurry
NathanFlurry previously requested changes Mar 27, 2025
View reviewed changes
@MasterPtato MasterPtato force-pushed the 03-27-feat_compress_sqlite_db branch from 2c0ca5c to 910b7c0 Compare March 27, 2025 22:18
@graphite-app
Copy link
Contributor

graphite-app bot commented Mar 28, 2025 •
edited
Loading

Merge activity

  • Mar 28, 2:55 PM EDT: MasterPtato added this pull request to the Graphite merge queue.
  • Mar 28, 2:56 PM EDT: CI is running for this pull request on a draft pull request (#2280) due to your merge queue CI optimization settings.
  • Mar 28, 2:57 PM EDT: A user merged this pull request with the Graphite merge queue via draft PR: #2280.

graphite-app bot pushed a commit that referenced this pull request Mar 28, 2025
@MasterPtato
feat: compress sqlite db (#2278)
902bc29 
<!-- Please make sure there is an issue that this PR is correlated to. -->

## Changes

<!-- If there are frontend changes, please include screenshots. -->
@graphite-app graphite-app bot closed this Mar 28, 2025
@graphite-app graphite-app bot deleted the 03-27-feat_compress_sqlite_db branch March 28, 2025 18:57
@github-actions github-actions bot mentioned this pull request Mar 28, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@NathanFlurry NathanFlurry Awaiting requested review from NathanFlurry

1 more reviewer

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MasterPtato @NathanFlurry