New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Size of scriptval database keeps growing #1879

Closed
ghost opened this Issue Feb 23, 2014 · 8 comments

Comments

Projects
None yet
6 participants
@ghost
Copy link

ghost commented Feb 23, 2014

I've noticed that the .db-wal file of my script is significantly larger than it needed to be.

The script keeps roughly 20 KB of JSON data around (serialized array containing 300 often-changing objects) but the associated .db-wal file is now (after 400 GM_setValue calls or so) 30 MB large.

Please disable journaling mode or run VACUUM from time to time (if not already the case).

@Ventero

This comment has been minimized.

Copy link
Contributor

Ventero commented Feb 23, 2014

Mh, I guess this might be the problem (from the SQLite documentation):

It is not possible to enable or disable auto-vacuum after a table has been created.

In GM_ScriptStorage_getDb, the auto_vacuum however is set after the table has already been created, so the fix is fairly obvious. But the question is what to do with existing tables - we'll probably have to copy/dump them, delete the DB, and then re-create it in the proper order.

@Ventero

This comment has been minimized.

Copy link
Contributor

Ventero commented Feb 23, 2014

Ah, nevermind, running vacuum switches to the correct mode, even if it has been set after the table has already been created.

Ventero added a commit to Ventero/greasemonkey that referenced this issue Feb 23, 2014

@Ventero

This comment has been minimized.

Copy link
Contributor

Ventero commented Feb 23, 2014

Ventero@2517af4 fixes this issue. Note also the typo in auto_vacuum.

@arantius arantius added this to the 1.16 milestone Feb 24, 2014

@arantius arantius closed this in 07bf354 Mar 21, 2014

@trespassersW

This comment has been minimized.

Copy link

trespassersW commented Jul 17, 2014

And why do we need these rollback data ?
I patched my GM as follows:
this._db.executeSimpleSQL('PRAGMA journal_mode = OFF;')
and I didn't find any problem.

@kankanfirst

This comment has been minimized.

Copy link

kankanfirst commented Aug 1, 2014

Don't know how to patch the GM..
Same issue still remains in the most updated version.

@azev

This comment has been minimized.

Copy link

azev commented Sep 6, 2014

Mouseover_Popup_Image_Viewer.db-wal = 31.2Mb
WTF is that? a new OS?

@VisualKei

This comment has been minimized.

Copy link

VisualKei commented Oct 14, 2014

Also stumbled upon this problem with 32Mb .db-wal files, and the suggested vacuum fix doesn't solve it completely (at least in my case).

I suggest adding this._db.executeSimpleSQL('PRAGMA wal_autocheckpoint = 10;'); line, since the root of the problem lies in the default value of wal_autocheckpoint=1000. It is multiplied by 32Kb (database page size) and gives this 32Mb max .db-wal file before auto-checkpoint kicks in and stops WAL growth. This completely solved the problem for me.

@arantius

This comment has been minimized.

Copy link
Collaborator

arantius commented Oct 14, 2014

@VisualKei logged as #2036 .

arantius added a commit to arantius/greasemonkey that referenced this issue Oct 16, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment