Repository navigation
v0.35.0
DittoFS v0.35.0
A modular virtual filesystem that decouples file interfaces from storage backends.
What's new
The main change in 0.35.0 is that BadgerDB is now the only metadata store: the PostgreSQL, SQLite and legacy memory metadata store backends have been removed. Read the migration guide below before upgrading a server that uses any of them.
Removed
- Breaking: the PostgreSQL, SQLite and legacy
memorymetadata stores are gone; BadgerDB is the only metadata store. Thepostgres,sqliteandmemorymetadata store types no longer exist. A server whose configuration still names one refuses to start, withunsupported metadata store type, and changes nothing. There is no conversion tool: a share on a SQL store moves by copying its files to a share on a new instance. Amemorystore is removed before the upgrade and recreated as a Badger store kept in RAM, withdfsctl store metadata add --name <store> --in-memory. See the metadata store migration guide before upgrading. The control-plane database (database:, SQLite or PostgreSQL) is not affected.
Fixed
-
A write refused because the local store is full now reaches the client as "no space", not as an I/O error. SMB returns
STATUS_DISK_FULL, NFSv3NFS3ERR_NOSPCand NFSv4NFS4ERR_NOSPC, where all three used to return their generic I/O error. -
A request that arrives with no deadline gets 30 seconds at the block store. A cold read from a remote that stopped answering, or a write held while the local store was at capacity, used to wait out their own 60-second budgets, and the backpressure budget restarted whenever the remote drained a little.
-
Deleting a file no longer waits for a flush pass to finish uploading. The delete itself was already durable; only the best-effort reclaim at its end waited, and with the remote down it waited for upload slots that never freed. An SMB delete-on-close was measured held for 68 seconds, past the point where clients give up on the connection.
-
A write that lands while a CLONE replaces its destination is no longer read back as zeros. A write outside the cloned range could commit between the clone's metadata transaction and its journal discard and be lost, and a background carve or warm could republish the old content over the clone. CLONE now holds the source and destination exclusively for the whole replacement, and ordinary writes, carves and warms take shared ownership.
-
That ownership is no longer held across a clone's upload or a whole-share warm. A CLONE drains its source before taking the exclusive hold, and a warm holds one payload at a time, so reads and writes on other files no longer queue behind either.
-
Directory removal and rename are ordered against creates inside the directory. A directory could be removed after another operation had created or moved a child into it, a create could commit after its parent was gone, and a delayed removal could delete a name that a rename had already replaced. Creates of distinct names in one directory still run concurrently.
-
New files created over SMB get the user's own primary GID, as they do over NFS. A user with a GID and no group memberships used to write files owned by group 1000, and a user with groups got their first group's GID instead of their own.
-
dfsctl store block add --configno longer drops the flags given beside it.--compression, every--encryption-*flag,--parallel-uploadsand the S3 field flags were silently discarded, so a store asked for compression or encryption was created without it. Flags are now merged into the JSON, and a flag that contradicts a key the JSON already sets is refused.--type memorynow sends the compression, encryption and upload settings as well. -
An S3 endpoint refused by the private-address guard fails at once. The SDK retried the refused dial up to ten times with backoff, turning a definite refusal into a 30-second timeout.
-
The startup warning for a missing config file names
dfs init, notdittofs init.
Performance
- Sequential reads of a file that is already local no longer re-download it. Prefetch ignored what the journal already held, so each warm 32 MiB pass fetched 24 MiB from the remote again; it now fetches only the ranges that are not resident. Demand fetches and integrity checks are unchanged.
Installation
Quick install (macOS / Linux):
curl -fsSL https://github.com/marmos91/dittofs/releases/download/v0.35.0/install.sh | shHomebrew (macOS/Linux):
brew tap marmos91/tap
brew install marmos91/tap/dfs # Server daemon
brew install marmos91/tap/dfsctl # Client CLIDebian/Ubuntu (APT):
curl -fsSL https://s3.cubbit.eu/dittofs-binaries/apt/dittofs.gpg.key | gpg --dearmor --yes | sudo tee /usr/share/keyrings/dittofs.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/dittofs.gpg] https://s3.cubbit.eu/dittofs-binaries/apt stable main" | sudo tee /etc/apt/sources.list.d/dfs.list
sudo apt update && sudo apt install dfs
sudo systemctl enable --now dfsRHEL/Fedora (YUM):
sudo curl -fsSLo /etc/yum.repos.d/dfs.repo https://s3.cubbit.eu/dittofs-binaries/rpm/dfs.repo
sudo yum install dfs
sudo systemctl enable --now dfsArch Linux:
curl -fsSLO https://github.com/marmos91/dittofs/releases/download/v0.35.0/dfs_0.35.0_amd64.pkg.tar.zst
sudo pacman -U dfs_0.35.0_amd64.pkg.tar.zst
sudo systemctl enable --now dfsScoop (Windows):
scoop bucket add marmos91 https://github.com/marmos91/scoop-bucket
scoop install dfs # Server daemon
scoop install dfsctl # Client CLIDocker:
docker run -d \
-p 12049:12049 \
-p 12445:12445 \
-p 8080:8080 \
-v /path/to/config.yaml:/config/config.yaml:ro \
-v dittofs-data:/data \
marmos91c/dittofs:v0.35.0Binary Download:
# dfs (server daemon)
curl -sfL https://github.com/marmos91/dittofs/releases/download/v0.35.0/dfs_0.35.0_Linux_x86_64.tar.gz | tar xz
# dfsctl (client CLI)
curl -sfL https://github.com/marmos91/dittofs/releases/download/v0.35.0/dfsctl_0.35.0_Linux_x86_64.tar.gz | tar xzChangelog
Bug Fixes
- f218342: fix(adapter): give a request without a deadline 30 s at the block store (#2966) (@llucian-iws)
- 8d18712: fix(adapter): refuse a write to a full local store as no space, not I/O (#2961) (@llucian-iws)
- 468a373: fix(cli): the startup warning names the dfs binary, not dittofs (#3025) (@viquitorreis)
- 4dae556: fix(dfsctl): merge flags given with store block add --config (#2974) (@llucian-iws)
- f676b91: fix(docs): add client prerequisites and fix brew and SMB mount steps in Getting Started (@viquitorreis)
- 246e6c9: fix(docs): write the NFS test file to the right mount path on macOS (@viquitorreis)
- 68946fe: fix(engine): don't hold payload scopes across a clone's upload or a whole-share warm (#3030) (@llucian-iws)
- 230508c: fix(journal): don't make Delete wait for a flush pass to reclaim (#2965) (@llucian-iws)
- 2142145: fix(metadata): synchronize directory lifetime with namespace mutations (#2956) (@badrkamel)
- 2a3d7f7: fix(nfs): preserve concurrent writes across clone replacement (#2958) (@badrkamel)
- 1507bdd: fix(s3): do not retry a dial the endpoint guard refused (@viquitorreis)
- e10b208: fix(s3): merge pull request #3023 from viquitorreis/fix/s3-conformance-private-endpoint-2925 (@viquitorreis)
- 2150727: fix(smb): give new files the user's own primary GID, as NFS does (#2963) (@llucian-iws)
- d0efaa7: fix(test): grade WPTS timeout and aborted outcomes as failures (#2954) (@badrkamel)
Performance Improvements
- 3e49cea: perf(engine): skip prefetch for resident journal ranges (#2957) (@badrkamel)
Others
- 2910947: Delete test/scenarios/plan directory (#2933) (@rif)
- d5f0c87: chore(deps): bump sigs.k8s.io/controller-runtime (#3015) (@dependabot[bot])
- 31757d4: chore(deps): bump the go-dependencies group with 3 updates (#3016) (@dependabot[bot])
- 8c75faf: chore(deps): bump the go-dependencies group with 6 updates (#2935) (@dependabot[bot])
- 77062d9: chore(metadata): remove the SQL and in-memory metadata stores (#2914) (@marmos91)
- aea1948: chore(release): v0.35.0 (#3047) (@marmos91)
- f854d99: ci(conformance): create the S3 bucket for pjdfstest/pynfs -s3 cells (@didimelli)
- 7f025cc: ci(dfsbench): merge pull request #2976 from viquitorreis/ci/dfsbench-smoke-dit-29 (@viquitorreis)
- ca04709: ci(dfsbench): parse the mailbox job files in the smoke job (@viquitorreis)
- 0b9ec95: ci(dfsbench): run the dfsbench smoke benchmark on pull requests (@viquitorreis)
- e6283ac: ci(e2e): run the Kerberos tests on PRs that change GSS dispatch, SECINFO or SMB identity mapping (#3045) (@llucian-iws)
- ea9a308: ci(integration): merge pull request #3024 from viquitorreis/ci/portmap-test-count-2998 (@viquitorreis)
- 5027238: ci(integration): run the portmap system test with -count=1 so its result is never cached (@viquitorreis)
- 76dab85: ci(posix): address review nits on the S3 setup gate (@didimelli)
- 2846fd2: ci(posix): gate S3 profiles on block store health, not on uploads (@didimelli)
- 3752e7b: ci(posix): keep a sentinel file so an S3 teardown drain always uploads (@didimelli)
- 0d8dbdd: ci: move the test path filters inside the job so they can be required (#2972) (@marmos91)
- 1228d06: ci: replace the NFS sec=krb5 conformance job with the Kerberos e2e tests on PRs (#3035) (@llucian-iws)
- 19661d4: docs(bench): explain why the bucket is larger than the live files right after a mailbox run (@viquitorreis)
- 2ea7e35: docs(bench): mailbox README covers file growth, cold logon and the gc share argument (@viquitorreis)
- ba61645: docs(bench): mention the CI smoke job (@viquitorreis)
- 436c48c: docs(bench): say how to make the mailbox logon scenario read cold from the remote (@viquitorreis)
- df4d2ba: docs(bench): say how to make the mailbox logon scenario read cold from the remote (@viquitorreis)
- c2fba34: docs(bench): say the mailbox simulator needs an S3 block store (@viquitorreis)
- 589ca77: docs(bench): the gc command in the mailbox README needs the share name (@viquitorreis)
- 9f7b2d0: docs(changelog): file 0.34.0 under its own heading and describe 0.35.0 (#3046) (@marmos91)
- bf1444c: docs(guide): fix the migration copy step and say that dfsctl login switches servers (@viquitorreis)
- 71735a7: docs(guide): log in to the new instance under its own dfsctl context (@viquitorreis)
- 5aa495e: docs(guide): merge pull request #3033 from viquitorreis/docs/metadata-migration-guide-3032 (@viquitorreis)
- f910fe2: docs(guide): merge pull request #3034 from viquitorreis/docs/migration-guide-dfsctl-context-3032 (@viquitorreis)
- 9478d5f: docs(guide): remove the duplicated paragraph in the migration switch-over step (@viquitorreis)
- 6425b71: docs(guide): switch back to the new context after the copy setup, explain --server, drop a duplicated paragraph (@viquitorreis)
- 1303fab: docs(install): add Uninstall section (@viquitorreis)
- a27943a: docs(install): address review comments on Uninstall section (@viquitorreis)
- 89f00b4: docs(install): clarify local metadata store and Compose PostgreSQL removal (@viquitorreis)
- 9f80639: docs(install): merge pull request #2952 from viquitorreis/docs/uninstall-guide-dit-26 (@viquitorreis)
- d440098: docs(install): merge pull request #2953 from viquitorreis/docs/uninstall-review-fixes-dit-26 (@viquitorreis)
- f22600d: docs(test): RFC for one test harness: a suite registry, one runner, results (#2975) (@llucian-iws)
- ecaa67a: test(ad-dc): create the NETLOGON test's metadata store as in-memory badger (#2980) (@llucian-iws)
- 4574906: test(bench): add fio mailbox workload simulator (@viquitorreis)
- 6271b60: test(bench): document gc --grace-period 0 for reading S3 usage (@viquitorreis)
- 5c632f5: test(bench): fix wording in the mailbox simulator files (@viquitorreis)
- 633e92f: test(bench): label 03 results as -u1 and warn when USERS is set (@viquitorreis)
- 504c30c: test(bench): label mailbox throughput as MiB/s, which is what fio's bw divided by 1024 gives (@viquitorreis)
- 78693fb: test(bench): make the mailbox catalog log rate a variable, fix the CATALOG_IOPS description (@viquitorreis)
- b6ca088: test(bench): make the mailbox full sync scenario grow the file past EOF (@viquitorreis)
- 3ca0b85: test(bench): merge pull request #3004 from viquitorreis/test/fio-mailbox-simulator-dit-30 (@viquitorreis)
- bba649c: test(bench): print a usage line when mailbox run.sh gets no job file (@viquitorreis)
- 0830618: test(bench): skip the summary on an empty fio output, keep results next to the script (@viquitorreis)
- 311d5da: test(e2e): address review nits: mid-write drop, stale cache, CB_NOTIFY delivery (@didimelli)
- 4495357: test(e2e): address review: tighter delegation, packet-loss and session asserts (@didimelli)
- bc4b618: test(e2e): count distinct clients in the callback probe; mark the dir-deleg skip (@didimelli)
- 70421ee: test(e2e): drop the stale-handle row and trigger permission denial with utimes (@didimelli)
- cc660e9: test(e2e): run the SMB Kerberos mounts instead of skipping them (#3027) (@llucian-iws)
- d8b9b64: test(e2e): run the error-conformance cases that were always skipped (@didimelli)
- c5b6413: test(e2e): say which conformance shares have no peer and what AccessDenied pins (@didimelli)
- c2d4a14: test(e2e): tighten error-conformance comments and guard a missing peer (@didimelli)
- 6c5f843: test(kerberos): make TestRealKDC run again, and run it in CI (#3002) (@llucian-iws)
- e33edad: test(kmip): include live interoperability tests in integration CI (#2955) (@badrkamel)
- bdb1a55: test(nfs): address review: grant-backed no-conflict check, accurate identity comments (@didimelli)
- 562be59: test(nfs): assert delegation grant and recall in the delegation e2e tests (@didimelli)
- e049163: test(nfs): assert directory delegation grant and CB_NOTIFY in e2e (@didimelli)
- 5fef9b1: test(nfs): assert recall-then-return ordering instead of a recall flush (@didimelli)
- ed1b4a4: test(nfs): dump the server's delegation and I/O log when a recall test fails (@didimelli)
- 70a661c: test(nfs): make the Kerberos v4.0 e2e subtests act as the principal they name (@didimelli)
- d250176: test(nfs): mount the packet-loss test over IPv4 so the DROP rule applies (@didimelli)
- d48e8cc: test(nfs): mount v4.0 delegation clients through a non-loopback address (@didimelli)
- df6dbb8: test(nfs): name the v4.1 session e2e tests for what they check (@didimelli)
- dced568: test(nfs): run cthon04 and nfstest under named mount option sets (#3008) (@didimelli)
- 600d21d: test(s3): drop the issue and CI reference from the conformance test comment (@viquitorreis)
- 3933f6d: test(s3): fail the refused-dial test in 5s instead of 30s (@viquitorreis)
- 765d2b6: test(s3): let the endpoint conformance test reach a local S3 (@viquitorreis)
- 93582aa: test(s3): say what the refused-dial test checks, not its history (@viquitorreis)
- e75835c: test(s3): the skip message no longer names MinIO (@viquitorreis)
- 647d9a2: test(scenarios): an FSLogix Office container over SMB (#2945) (@llucian-iws)
- 61b013b: test(scenarios): compaction on a metadata store shared by several remotes (#2943) (@llucian-iws)
- 7d6acc8: test(scenarios): fail instead of exiting 0 when another run holds the lock (#3018) (@rif)
- 5983ff9: test(scenarios): run from a container, drop stale A-08 failure notes (DIT-27) (#2944) (@didimelli)
- 3ea5e5d: test(scenarios): truncating an uploaded file gives its tail back to the bucket after GC (#2969) (@llucian-iws)
- cbc99c4: test(scenarios): who may mount a share with default permission none (#2942) (@llucian-iws)
- dc10d8d: test(smb): require ENF-02/03 mount denials to be access denials (@didimelli)
Full Changelog: v0.34.1...v0.35.0
Verifying Checksums
sha256sum -c checksums.txtVerifying Signatures
Release checksums are signed with Sigstore cosign using keyless OIDC:
cosign verify-blob \
--bundle checksums.txt.sigstore.json \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity-regexp '^https://github\.com/marmos91/dittofs/\.github/workflows/release\.yml@refs/tags/' \
checksums.txtDocker Images
Multi-architecture images are available for linux/amd64 and linux/arm64:
marmos91c/dittofs:v0.35.0- This specific versionmarmos91c/dittofs:latest- Latest stable releasemarmos91c/dittofs:v0- Latest v0.x.xmarmos91c/dittofs:v0.35- Latest v0.35.x