Skip to content

dwarfs-0.3.0
Compare
Choose a tag to compare
@mhx mhx released this 30 Dec 21:49
· 1541 commits to main since this release
v0.3.0
ae1e5c8

Even better compression than before

Mostly thanks to a new ordering algorithm that is now enabled by default, I've seen a 15% improvement in achievable compression ratio. In my standard test of packing 48 GiB of Perl installations, the resulting DwarFS image size reduced from 556 MiB to 472 MiB without any regression in compression speed.

More memory efficient FUSE driver

By switching to jemalloc, the FUSE driver has become much more memory efficient, using up to ten times less memory than with the standard glibc allocator.

Python scripting support

The Lua scripting interface has been fully replaced by a new Python interface. I've been looking for a luabind replacement, but none of the candidates seemed to be well maintained or reasonably easy to integrate. Python is much more approachable for most people and boost::python seems well maintained. The new interface also has a lot more features. You can find an example script in the distribution.

Fix for file system images created with versions before dwarfs-0.2.3

If you've created DwarFS images with the 0.2.0, 0.2.1 or 0.2.2 releases, symbolic links were stored in a way that the FUSE driver in the 0.2.x releases could not read them back correctly. With the new 0.3.0 release, these old images, including the symbolic links, can now be read again, so there's no need to rebuild your old images.

Improved file system format

The file system format has been updated with the 0.3.0 release to include integrity checking via SHA2-512/256 hashes as well as features that should make recovery easier in case of file system image corruption. In addition to the SHA hashes, the extremely fast xxHash library is used to store a second hash that is checked every time any part of the file system is used. While there are currently no recovery features implemented, having this data in the file system already should be really valuable. You can convert an old image to the new format using:

mkdwarfs -i old.dwarfs -o new.dwarfs --recompress none

Statically linked 64-bit Linux binaries available

Given the long list of dependencies, building DwarFS might not be an option for you. In that case, you can now download the binary distribution that should work fine on most 64-bit Linux distributions. FUSE drivers are included for both FUSE2 and FUSE3

Lots of smaller fixes & changes

See the Change Log for a full list of changes.

Assets 4