Skip to content

How to add tests and fuzz coverage with Claude

Daniel Wagner edited this page Oct 27, 2026 · 1 revision

Adding tests and fuzz coverage with Claude Code

The issues labelled with tests: and fuzz: (see the list below) list the commands that have no mock test or aren't fuzzed yet. This page shows how to close them with Claude Code and the shared AI resources of nvme-cli-ai. Using an AI assistant is optional: every step works by hand as well. The in-tree reference is "Add tests for a command" in CONTRIBUTING.md (added by #4323), and Documentation/TESTING.md describes the tools.

Vendors have an advantage here: you know the log page layouts and have real drives. Give the assistant your log page specification as input data, and check what it produces against a real drive.

Setup

Clone both repositories next to each other:

git clone https://github.com/linux-nvme/nvme-cli.git
git clone https://github.com/linux-nvme/nvme-cli-ai.git
cd nvme-cli

Start Claude Code from nvme-cli with nvme-cli-ai added, so it loads the shared project context (CLAUDE.md), the rules in .claude/rules/ and the /nvme-spec skill:

CLAUDE_CODE_ADDITIONAL_DIRECTORIES_CLAUDE_MD=1 claude --add-dir ../nvme-cli-ai

The rule most relevant here is .claude/rules/fuzzing.md: how the device fuzz target works, what may go into its table, how to get coverage past model checks, and how to fix findings. The commit message, coding style and checkpatch rules apply to the result too.

You need three builds; the assistant can set them up:

meson setup .build                                   # mock tests
CC=clang CXX=clang++ meson setup .build-asan \
    -Db_sanitize=address,undefined -Db_lundef=false  # the same under ASan/UBSan
scripts/fuzz/nvme-fuzz build                         # libFuzzer build, .build-fuzz

Input data

The more precise the input, the better the result. Useful input:

  • The issue for your plugin: the commands without a test or fuzz entry, and the largest functions the tests don't reach.
  • The layout of your vendor log pages and features: a specification, a header from your firmware team, or a hex dump from a real drive (no serial numbers or other customer data, please).
  • The output of the command on a real drive, normal and -o json. It becomes the expected output of the test.
  • Which models support which command: the PCI vendor and device IDs your plugin checks, and the Identify Controller fields it looks at.

Mock tests

A mock test runs the real nvme binary against a mocked controller. The mock (tests/cli/libmock_nvme.c) intercepts the ioctls and forwards every command to a Python object in the test, which answers it. See tests/cli/nvme_mock.py and nvme_mock_ipc.py, and as examples tests/cli/ocp/ (one file per command, a shared controller class), tests/cli/wdc/ (several models) and tests/cli/micron/.

A test:

  1. subclasses the controller and fills in the answers: Identify data, log pages by log ID (and UUID index), features by feature ID;
  2. sets up the model if the plugin checks one: PCI IDs in the test's sysfs directory, Identify Controller fields;
  3. runs the command, normal and JSON, and checks the output, including the values that come from the device. A test that only checks the exit code misses most bugs;
  4. is registered in the meson.build of the plugin's test directory.

Example prompt:

Add mock tests for nvme <vendor> <command> in tests/cli/<vendor>/, following tests/cli/ocp/. The log page layout is in <file>; the output of a real drive is in <file>. Use non-zero values in every field so a wrong offset shows up. Check normal and JSON output. Run the tests in .build and .build-asan.

What to check before you send it:

  • meson test -C .build --suite <vendor> and the same in .build-asan pass.
  • The test fails when the code is wrong: break a field offset in the printer, rebuild (make sure the build really succeeded), and see the test fail.
  • The expected values come from your specification or a real drive, not from what the current code prints. Otherwise the test only freezes existing bugs.
  • The JSON output doesn't change. If a test shows that it is wrong, fix it in a separate commit and say so in the PR.

Fuzz table entries

The device fuzz target (tests/fuzz/fuzz-device.c) runs commands against the same mock, but every answer comes from the fuzzer. A command is only fuzzed if it has an entry in the commands[] table there. The rules (see .claude/rules/fuzzing.md for the details):

  • Append only: the inputs select a command by its index.
  • Read-and-print commands only. Nothing that writes a file (unless its output can go to /dev/null) or changes the device.
  • Commands behind a model check get .pci = true, and the plugin's model IDs go into PCI_IDS in scripts/fuzz/nvme_fuzz/device.py, one per capability group.
  • The mock tests are the best seeds: nvme-fuzz seeds turns each run of a table command in the mock suite into a seed with valid answers, so a mock test usually comes first.

Example prompt:

Add nvme <vendor> <command> to the device fuzz table, with .pci = true and the model IDs in PCI_IDS. Run nvme-fuzz sweep on the new entries, then compare nvme-fuzz coverage --entries ... before and after adding seeds from the mock tests. Fuzz the new entries for 30 minutes and triage what it finds.

What to check:

  • scripts/fuzz/nvme-fuzz sweep --entries N reports no crash, hang or written file.
  • scripts/fuzz/nvme-fuzz coverage --entries N shows the command gets past its checks (compare with and without your seeds).
  • Anything the fuzzer finds is a bug in the plugin: fix it first, in its own commit with a regression input in tests/fuzz/corpus/device/, then add the table entry.

Sending the changes

  • Check against a real drive: build master and your branch, run the commands on the drive with both and compare. Mocks encode assumptions; the drive doesn't.
  • Run ../nvme-cli-ai/scripts/checkpatch-series before you push. CI fails on checkpatch warnings.
  • Open a pull request against linux-nvme/nvme-cli, mention the issue, and tick off the commands in it.
  • Review what the assistant wrote as if a colleague had written it: you sign it off.

The issues

Plugin Mock tests Fuzz table
core #4322
Sandisk (sndk, wdc) #4312 #4303, #4289
Micron #4304
OCP #4313 #4305
Memblaze #4314 #4306
NetApp #4315 #4307
Samsung #4308
ScaleFlux #4316 #4309
Solidigm #4317 #4310
Amazon #4318
IBM #4319
MangoBoost #4320
Plugins without a maintainer #4321 #4311

The maintainers of each plugin are listed in MAINTAINERS (#4302).

Clone this wiki locally