Compact provides an API for regularly compressing the database.

[glycerine]

Without regular compaction, the bolt database will grow
without bound on disk.

Fixes #674

[glycerine]

@benbjohnson if you are too busy to review, is there somebody else you could suggest?

[vincent-petithory]

Most of the code seems to come from the existing compact cmd, so it should be fine. Le lun. 1 mai 2017 18:26, Jason E. Aten, Ph.D. <notifications@github.com> a écrit :

…

@benbjohnson <https://github.com/benbjohnson> if you are too busy to review, is there somebody else you could suggest? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#679 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGwMKCRYvL9IAFsld01rpm4rM5iXDTCks5r1gfHgaJpZM4NM7Ma> .

[benbjohnson]

@glycerine I'm hesitant to add this feature because I think it adds a lot of code and complexity for an issue that relatively few people have. Bolt doesn't typically work well with large values since it causes fragmentation.

This is also a feature that can largely be implemented outside of the core Bolt codebase. Why not make this a separate library that is layered on?

[glycerine]

This allows bolt to work with large values, because it allows the fragmentation they produce to be addressed quite effectively. I run regularly with a single value, 3MB blob.

We must close the old database file and re-open the newly compacted file, so all places where the database is in use must be told to stop in the middle of wherever they are, and to start using a new database handle. This is difficult and awkward, if it is possible at all, from the outside library. By making the change-over inside the DB structure itself, everyone starts using the newly compacted database in unison at once. Nobody crashes because they are trying to read/write the old closed and deleted file.

[benbjohnson]

@glycerine I thought it over for day and I decided that it's too much of an edge case feature to warrant the large amount of code required. I think the best case forward would be to leave it as a branch on your fork and we can add a link in the README for anyone who needs the functionality. It's easy enough to vendor in a different repo in a project and the bolt project doesn't change often enough that the fork would get out of date.

[adamluzsi]

Still the concept is awesome and as a package that wraps boltdb connection around and add this feature is a promising idea.

[glycerine]

@benbjohnson Thanks for thinking about it. I guess I just disagree about the importance of being able to store large values. But I understand that you want to keep the codebase small, and that is a valuable attribute.

@adamluzsi I don't see how to make a wrapper, since we need to add a check at the end of every transaction commit. If there was a hook for all transactions (not just on a tx-by-tx basis), then a wrapper might be doable.

I've made a fork to support big values, https://github.com/bigboltdb/bolt.