Increase sqlite3_hard_heap_limit64 to 8 GiB behind an autogate

[joshthoward]

No description provided.

[ask-bonk]

I'm Bonk, and I've done a quick review of your PR.

This PR increases the SQLite sqlite3_hard_heap_limit64 from 512 MiB to 8 GiB behind the INCREASE_SQLITE_HARD_HEAP_LIMIT autogate.

Issues (ranked by severity):

1. (Low) Autogate naming mismatch: enum vs string name word ordering is inconsistent, breaking the convention every other autogate follows.
2. (Low) Minor grammar nit in the comment.

[ask-bonk]

Review posted successfully on PR #6570. I found two low-severity issues:

1. Autogate naming mismatch: The enum INCREASE_SQLITE_HARD_HEAP_LIMIT doesn't match the string sqlite-increase-hard-heap-limit (words are reordered). Every other autogate in the file follows the convention of matching the kebab-case of the enum name. Suggested fix provided.

2. Grammar nit: "a 8 GiB" should be "an 8 GiB" in the comment. Suggested fix provided.

The core change itself -- gating the heap limit increase behind an autogate with the static bool doOnce pattern -- is sound. The autogate is initialized before any SQLite database is created in normal server startup, and the process-wide limit only needs to be set once.

github run

[github-actions]

The generated output of @cloudflare/workers-types matches the snapshot in types/generated-snapshot 🎉

[a-robinson]

Does the higher limit have any negative effect on how much memory workerd uses locally? I’d imagine not since it’s just a limit and nothing gets preallocated?

[joshthoward]

@a-robinson

Does the higher limit have any negative effect on how much memory workerd uses locally? I’d imagine not since it’s just a limit and nothing gets preallocated?

Nothing about this should change the amount of memory allocated by SQLite other than the fact that queries which previously needed more than 512 MiB of memory would fail with SQLITE_NOMEM. SQLite will have a connection free memory in its page cache once the soft limit is exceeded. Raising this could allow for memory consumption to remain higher, but we aren't modifying that for now (we don't want cached DB pages to consume sandbox memory).