Skip to content

v0.35.0

Choose a tag to compare

@github-actions github-actions released this 12 Oct 16:09
· 537 commits to develop since this release
v0.35.0
aea1948

DittoFS v0.35.0

A modular virtual filesystem that decouples file interfaces from storage backends.

What's new

The main change in 0.35.0 is that BadgerDB is now the only metadata store: the PostgreSQL, SQLite and legacy memory metadata store backends have been removed. Read the migration guide below before upgrading a server that uses any of them.

Removed

  • Breaking: the PostgreSQL, SQLite and legacy memory metadata stores are gone; BadgerDB is the only metadata store. The postgres, sqlite and memory metadata store types no longer exist. A server whose configuration still names one refuses to start, with unsupported metadata store type, and changes nothing. There is no conversion tool: a share on a SQL store moves by copying its files to a share on a new instance. A memory store is removed before the upgrade and recreated as a Badger store kept in RAM, with dfsctl store metadata add --name <store> --in-memory. See the metadata store migration guide before upgrading. The control-plane database (database:, SQLite or PostgreSQL) is not affected.

Fixed

  • A write refused because the local store is full now reaches the client as "no space", not as an I/O error. SMB returns STATUS_DISK_FULL, NFSv3 NFS3ERR_NOSPC and NFSv4 NFS4ERR_NOSPC, where all three used to return their generic I/O error.

  • A request that arrives with no deadline gets 30 seconds at the block store. A cold read from a remote that stopped answering, or a write held while the local store was at capacity, used to wait out their own 60-second budgets, and the backpressure budget restarted whenever the remote drained a little.

  • Deleting a file no longer waits for a flush pass to finish uploading. The delete itself was already durable; only the best-effort reclaim at its end waited, and with the remote down it waited for upload slots that never freed. An SMB delete-on-close was measured held for 68 seconds, past the point where clients give up on the connection.

  • A write that lands while a CLONE replaces its destination is no longer read back as zeros. A write outside the cloned range could commit between the clone's metadata transaction and its journal discard and be lost, and a background carve or warm could republish the old content over the clone. CLONE now holds the source and destination exclusively for the whole replacement, and ordinary writes, carves and warms take shared ownership.

  • That ownership is no longer held across a clone's upload or a whole-share warm. A CLONE drains its source before taking the exclusive hold, and a warm holds one payload at a time, so reads and writes on other files no longer queue behind either.

  • Directory removal and rename are ordered against creates inside the directory. A directory could be removed after another operation had created or moved a child into it, a create could commit after its parent was gone, and a delayed removal could delete a name that a rename had already replaced. Creates of distinct names in one directory still run concurrently.

  • New files created over SMB get the user's own primary GID, as they do over NFS. A user with a GID and no group memberships used to write files owned by group 1000, and a user with groups got their first group's GID instead of their own.

  • dfsctl store block add --config no longer drops the flags given beside it. --compression, every --encryption-* flag, --parallel-uploads and the S3 field flags were silently discarded, so a store asked for compression or encryption was created without it. Flags are now merged into the JSON, and a flag that contradicts a key the JSON already sets is refused. --type memory now sends the compression, encryption and upload settings as well.

  • An S3 endpoint refused by the private-address guard fails at once. The SDK retried the refused dial up to ten times with backoff, turning a definite refusal into a 30-second timeout.

  • The startup warning for a missing config file names dfs init, not dittofs init.

Performance

  • Sequential reads of a file that is already local no longer re-download it. Prefetch ignored what the journal already held, so each warm 32 MiB pass fetched 24 MiB from the remote again; it now fetches only the ranges that are not resident. Demand fetches and integrity checks are unchanged.

Installation

Quick install (macOS / Linux):

curl -fsSL https://github.com/marmos91/dittofs/releases/download/v0.35.0/install.sh | sh

Homebrew (macOS/Linux):

brew tap marmos91/tap
brew install marmos91/tap/dfs      # Server daemon
brew install marmos91/tap/dfsctl   # Client CLI

Debian/Ubuntu (APT):

curl -fsSL https://s3.cubbit.eu/dittofs-binaries/apt/dittofs.gpg.key | gpg --dearmor --yes | sudo tee /usr/share/keyrings/dittofs.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/dittofs.gpg] https://s3.cubbit.eu/dittofs-binaries/apt stable main" | sudo tee /etc/apt/sources.list.d/dfs.list
sudo apt update && sudo apt install dfs
sudo systemctl enable --now dfs

RHEL/Fedora (YUM):

sudo curl -fsSLo /etc/yum.repos.d/dfs.repo https://s3.cubbit.eu/dittofs-binaries/rpm/dfs.repo
sudo yum install dfs
sudo systemctl enable --now dfs

Arch Linux:

curl -fsSLO https://github.com/marmos91/dittofs/releases/download/v0.35.0/dfs_0.35.0_amd64.pkg.tar.zst
sudo pacman -U dfs_0.35.0_amd64.pkg.tar.zst
sudo systemctl enable --now dfs

Scoop (Windows):

scoop bucket add marmos91 https://github.com/marmos91/scoop-bucket
scoop install dfs       # Server daemon
scoop install dfsctl    # Client CLI

Docker:

docker run -d \
  -p 12049:12049 \
  -p 12445:12445 \
  -p 8080:8080 \
  -v /path/to/config.yaml:/config/config.yaml:ro \
  -v dittofs-data:/data \
  marmos91c/dittofs:v0.35.0

Binary Download:

# dfs (server daemon)
curl -sfL https://github.com/marmos91/dittofs/releases/download/v0.35.0/dfs_0.35.0_Linux_x86_64.tar.gz | tar xz

# dfsctl (client CLI)
curl -sfL https://github.com/marmos91/dittofs/releases/download/v0.35.0/dfsctl_0.35.0_Linux_x86_64.tar.gz | tar xz

Changelog

Bug Fixes

Performance Improvements

Others


Full Changelog: v0.34.1...v0.35.0

Verifying Checksums

sha256sum -c checksums.txt

Verifying Signatures

Release checksums are signed with Sigstore cosign using keyless OIDC:

cosign verify-blob \
  --bundle checksums.txt.sigstore.json \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-identity-regexp '^https://github\.com/marmos91/dittofs/\.github/workflows/release\.yml@refs/tags/' \
  checksums.txt

Docker Images

Multi-architecture images are available for linux/amd64 and linux/arm64:

  • marmos91c/dittofs:v0.35.0 - This specific version
  • marmos91c/dittofs:latest - Latest stable release
  • marmos91c/dittofs:v0 - Latest v0.x.x
  • marmos91c/dittofs:v0.35 - Latest v0.35.x