Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Size of scriptval database keeps growing #1879

Closed
ghost opened this issue Feb 23, 2014 · 8 comments
Closed

Size of scriptval database keeps growing #1879

ghost opened this issue Feb 23, 2014 · 8 comments
Milestone

Comments

@ghost
Copy link

ghost commented Feb 23, 2014

I've noticed that the .db-wal file of my script is significantly larger than it needed to be.

The script keeps roughly 20 KB of JSON data around (serialized array containing 300 often-changing objects) but the associated .db-wal file is now (after 400 GM_setValue calls or so) 30 MB large.

Please disable journaling mode or run VACUUM from time to time (if not already the case).

@Ventero
Copy link
Contributor

Ventero commented Feb 23, 2014

Mh, I guess this might be the problem (from the SQLite documentation):

It is not possible to enable or disable auto-vacuum after a table has been created.

In GM_ScriptStorage_getDb, the auto_vacuum however is set after the table has already been created, so the fix is fairly obvious. But the question is what to do with existing tables - we'll probably have to copy/dump them, delete the DB, and then re-create it in the proper order.

@Ventero
Copy link
Contributor

Ventero commented Feb 23, 2014

Ah, nevermind, running vacuum switches to the correct mode, even if it has been set after the table has already been created.

Ventero added a commit to Ventero/greasemonkey that referenced this issue Feb 23, 2014
@Ventero
Copy link
Contributor

Ventero commented Feb 23, 2014

Ventero@2517af4 fixes this issue. Note also the typo in auto_vacuum.

@arantius arantius added this to the 1.16 milestone Feb 24, 2014
@trespassersW
Copy link

And why do we need these rollback data ?
I patched my GM as follows:
this._db.executeSimpleSQL('PRAGMA journal_mode = OFF;')
and I didn't find any problem.

@kankanfirst
Copy link

Don't know how to patch the GM..
Same issue still remains in the most updated version.

@azev
Copy link

azev commented Sep 6, 2014

Mouseover_Popup_Image_Viewer.db-wal = 31.2Mb
WTF is that? a new OS?

@VisualKei
Copy link

Also stumbled upon this problem with 32Mb .db-wal files, and the suggested vacuum fix doesn't solve it completely (at least in my case).

I suggest adding this._db.executeSimpleSQL('PRAGMA wal_autocheckpoint = 10;'); line, since the root of the problem lies in the default value of wal_autocheckpoint=1000. It is multiplied by 32Kb (database page size) and gives this 32Mb max .db-wal file before auto-checkpoint kicks in and stops WAL growth. This completely solved the problem for me.

@arantius
Copy link
Collaborator

@VisualKei logged as #2036 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants