0.6.0

Now with more NVMe/oF

This release contains an even-more-experimental NVMe/oF backend. As usual, it's torture-tested on my own hardware and setup, and works, but YMMV. File issues and pull requests please!

Full Changelog: v0.5.1...v0.6.0

0.5.1

Full Changelog: v0.5.0...v0.5.1

0.5.0

Full Changelog: v0.4.4...v0.5.0

0.4.4

A refactoring release: the same behaviour, with much more of it under test.

The FSKit extension's data path — read-modify-write against a 4Kn LUN, the
chunk cache that keeps a guest's journal a cache hit, and the lock that pins
cache-patch order to device order — lived inside an Xcode target where the test
suite could not reach a line of it. It now lives in the package, and is tested:
that a partial write preserves the bytes either side of it and never lets an
unaligned write reach the device, that a failed write drops its cached overlap
rather than serving bytes the device does not have, and that overlapping writes
arrive in submission order.

The keychain is behind a seam for the same reason. The bug fixed in 0.4.3 —
every CHAP secret discarded, in every release before it — was not a logic error
anywhere. The calls returned, the error handling was right, and the request was
addressed to a keychain a system-domain daemon cannot reach. Nothing looked at
the shape of the request, so there is now a test that does.

The suite is 368 tests at 87.1% line coverage, over 7,307 lines — 560 more than
were visible before this work, because the least-tested half of the product was
also the half nothing measured.

Also in this release:

• A large write no longer holds a copy of most of itself. Chunks are issued
  eight at a time rather than all at once, so a request costs a fixed window of
  memory regardless of its size. Throughput is unchanged, measured at both four
  and thirty-two chunks per request. In practice FSKit hands down writes well
  under that bound, so this is insurance rather than a fix — but it is now
  asserted rather than assumed.
• CI computes coverage on every run instead of only where it publishes it, so a
  pull request shows what it does to the number.

0.4.3

Reading a large file no longer grows the daemon by the size of the file.

iscsid's receive buffer reclaimed nothing. Every byte that arrived from the
target was kept, so resident memory tracked bytes read one-for-one: copying a
large file off a volume drove one daemon to a 37 GB peak, and 2 GiB of reads
cost 2 GB of memory in a controlled reproduction. The same reads now cost 25 MB,
with throughput unchanged.

The cause is a sharp edge in Data. The buffer advanced a cursor as PDUs were
consumed and periodically called removeFirst to drop the dead prefix — but
Data is a slice type, so removeFirst moves the slice's start and keeps the
original allocation. Every later append then reallocated that same allocation
larger. Nothing decoded incorrectly at any point, which is why a 326-test suite
never noticed; only a memory assertion catches it, and there is now one that
pushes 256 MiB through the deframer and watches the process.

This affects every release before this one, and anyone reading large files
should take it.

Also in this release:

• Large writes are 2.1-2.6x faster. A write bigger than one SCSI command
  used to issue its chunks one at a time, waiting for each round trip. They now
  go out together — the chunks of a single write are contiguous by construction,
  so nothing depends on the order they complete in. Measured against a 10GbE
  target, interleaved: 56.7 -> 118.6 MB/s with Force Unit Access on, and
  319.1 -> 815.4 MB/s with the target caching. Small writes are unchanged.
• CHAP secrets can be saved. They never could: the daemon wrote to the
  data-protection keychain, which is served by a per-user agent that a system
  LaunchDaemon cannot reach, so every save failed with "the secret was saved but
  could not be read back" and nothing was stored. Secrets now go to the System
  keychain.
• Mutual CHAP is hidden for now. It is implemented and correct on the wire,
  but the target it was developed against will not answer a challenge —
  TrueNAS SCALE writes the peer credential into its config and never loads it —
  so offering the switch only turned working targets into broken ones.
  docs/open-questions.md item 3a has the evidence.
• The login exchange now narrates itself to the log and to iscsictl --debug,
  naming the stage and decoding the status when a target refuses. Secrets,
  challenges and responses are never written there.

0.4.2

Full Changelog: v0.4.1...v0.4.2

0.4.1

Full Changelog: v0.4.0...v0.4.1

0.4.0

Reads get a real cache, and it tunes itself.

Speculative readahead used to hold blocks fetched for one exact request, and
threw the whole window away whenever the request size changed. It is now a
32 MiB cache of 256 KiB chunks, served by byte range: a miss fetches the whole
covering chunk, so a guest's neighbouring small reads become memory copies
instead of round trips to the target. Writes are patched into the cached chunks
rather than invalidating them, so a filesystem journal — written and re-read
thousands of times during a boot — stops punching refetch-sized holes.

How far to read ahead is no longer a guess, or a setting. The extension
measures what its speculation actually costs and adjusts every second: a clean
sequential stream earns more depth, and scattered or write-punctuated I/O loses
it. Measured on a 10GbE target, that means depth 3 under a mixed soak and the
full depth-32 window on an unbroken 100 GB read, with the same 93% hit rate at
both ends and 421 MB/s sustained on the sequential pass.

A brief per-target "type of workload" picker in between never shipped in a
tagged release, and the measurements are why: readahead depth changed how much
bandwidth was wasted — 5.9% of speculation at depth 2 against 48.8% at depth 32
— and did not change the hit rate at all. That is not a decision worth handing
to a user, so the machine makes it. Any leftover setting is cleared on upgrade.

Also in this release:

• Detach on a mounted volume asks before ejecting it, instead of silently
  forcing the eject out from under whatever is still using the files.
• A target can trade write-through durability for a timed cache flush (1–60 s)
  or none at all, for targets whose cache is genuinely non-volatile. The app is
  blunt about the consequence, because it is a real one.
• Security fixes across the daemon and login path from an audit: the daemon now
  refuses portals it has no saved record for, resolves CHAP credentials by
  record rather than by a client-supplied username, and treats a missing secret
  as an error rather than silently logging in unauthenticated.

Verified against real hardware with every read checked: no stale blocks, no
wrong data, and zero unanswered requests across roughly a million I/Os.

0.3.8

Reading is five times faster. Against a 10GbE target a volume read at 220
MB/s; it now reads at about 1100 MB/s, which is 94% of what the same link and
target sustain with nothing in the way.

The cause was not the network, and not the number of connections. macOS asks the
filesystem extension for one piece of the disk at a time and waits for each
before asking for the next, so the link sat idle for a full round trip between
every request. The extension now fetches ahead of a sequential read, keeping
enough in flight to cover the wait, and splits large requests into several
commands issued together instead of one after another.

It reads ahead only after two requests in a row have actually been consecutive —
guessing after a single read would fetch data nobody wanted and throw it away,
which is slower on a disk that has to seek. Any write discards what was fetched,
because data read before a write is wrong after it.

Verified rather than assumed. The same region checksums identically before and
after, and two 30-minute soaks interleaving writes and seeks into sequential
reads checked every block individually — 130 GB across 195,000 requests in the
second — with no mismatch.

Writes are unchanged. Every write is still sent write-through, because macOS
never tells the extension when a filesystem wanted its data made durable, and
without that signal the only safe assumption is that it always did. That costs
speed and buys the guarantee that an acknowledged write has landed. It is the
right trade for a disk on the far end of a network cable, and nothing here
changes it.

Attaching repairs itself. "File system named iSCSI not found" could appear
when more than one copy of the filesystem extension was registered — usually
after an update, or from an installer disk image left mounted. The app now
removes the extra copies and retries on its own instead of reporting the problem
and sending you to another screen to fix it.

Reinstalling the background service takes one click. It used to take two:
the button said "Reinstall", became "Install", and had to be pressed again.
macOS was refusing the first registration outright because it had not finished
releasing the previous one, so the second press was not confirming anything — it
was doing the work. The app now waits and asks again, and shows a spinner while
it does. It also notices when only the build number changed, which it previously
ignored, so a service left running from an older build no longer reports itself
as healthy.

Check for Updates is now in the application menu under About, where it
belongs, as well as the menu bar.

This release is licensed under Apache 2.0, which the project previously lacked
entirely.

0.3.7

Reading is five times faster.

Against a 10GbE target the volume read at 220 MB/s. It now reads at 1099 MB/s,
which is 94% of what the same link and target sustain when nothing is in the
way.

The cause was not the network and not the number of connections. macOS asks the
filesystem extension for one piece of the disk at a time and waits for each one
to come back before asking for the next, so the link sat idle for a full round
trip between every request — at a quarter of a millisecond each, that is most of
the time. The extension now fetches ahead of a sequential read, keeping enough
requests in flight to cover the wait.

It reads ahead only once two requests in a row have actually been consecutive.
Guessing after a single read would issue four requests nobody wanted and throw
them away, which is slower on a disk that has to seek, not faster. Any write
discards what was fetched, because data read before a write is wrong after it.

Verified rather than assumed: the same 256 MiB region checksums identically
before and after, and a 30-minute soak interleaving writes and seeks into
sequential reads checked 362,981 blocks individually and found no mismatch.

Writes are unchanged, and it is worth saying why. Every write is sent
write-through, because macOS never tells the extension when a filesystem wanted
its data made durable. Without that signal the only safe assumption is that it
always did. That costs about four and a half times the write speed and buys the
guarantee that an acknowledged write has actually landed — the right trade for a
disk on the other end of a network cable, and not one this release changes.

Also fixed: attaching could fail with "File system named iSCSI not found"
when more than one copy of the filesystem extension was registered, usually
after an update or from a disk image left mounted. The app now removes the extra
copies and retries by itself, instead of reporting the problem and asking you to
repair it from another screen.