[QA] VFS consumes much disk space

[jnweiger]

Seen with testpilot 2.7.0 daily (post RC1) on Ubuntu 18.04

I have synced down a tree of ca 600.000 files -- all in virtual placeholder mode.

θ66° 1z [testy:/home/sync/testpilotcloud4] $ find . -type f | wc -l
585374
θ66° 1z [testy:/home/sync/testpilotcloud4] $ du -sh .s* .
1,1G	.sync_dd97b94dce89.db
32K	.sync_dd97b94dce89.db-shm
9,7M	.sync_dd97b94dce89.db-wal
2,5G	.

The database consumes ca 2kb per file.
The filesystem seems to consume ca 4k per file, although they are only placeholders.

Not sure, if we can do anyhting about that.

[jnweiger]

During syncing, the .sync*.db-wal file grows. After all 265000 additional files are there, we have:

$ du -sh .s* .
1,1G	.sync_dd97b94dce89.db
17M	.sync_dd97b94dce89.db-shm
8,6G	.sync_dd97b94dce89.db-wal
3,6G	.
$ find . -type f | wc -l
850200

The GUI is now very sluggish. Click on the three dots menu takes 25 seconds of 100% CPU before the menu opens.

[TheOneRing]

Obviously it has nothing to do with vfs but 265000 files.
Also the file should shrink after the sync is finished.

[TheOneRing]

Duplicate of #7646

[jnweiger]

850000 files by now. Not sure if the non-VFS behaviour is different. :-)

https://www.sqlite.org/wal.html suggests that sqlite checkpointing should be performed periodically. either automtically or manually. PRAGMA schema.wal_checkpoint;