Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nix-store --repair-path fails on ZFS on EIO #8121

Open
remexre opened this issue Mar 28, 2023 · 1 comment
Open

nix-store --repair-path fails on ZFS on EIO #8121

remexre opened this issue Mar 28, 2023 · 1 comment
Labels

Comments

@remexre
Copy link

remexre commented Mar 28, 2023

Describe the bug

ZFS can detect hardware-failure-related corruption of disk blocks. In situations where it cannot automatically recover the contents of the file, read()s from the file will return EIO.

nix-store --repair-path bails out when it gets these EIOs, and it's not clear whether it's possible to repair the files on a live NixOS system, since /nix/store appears read-only to... users' systemd slices? To sudo under a graphical shell, anyway.

Steps To Reproduce

  1. Break a mirror, incurring 7 years of bad luck.
  2. Run sudo zpool scrub whatever, then run sudo zpool status -v, notice an errors: Permanent errors have been detected in the following files: section.
  3. Run sudo nix-store --repair-path on one of the paths that appears.
  4. Get error: reading from file: Input/output error

(I expect a "synthetic" repro could use ptrace to inject EIOs?)

Expected behavior

The store path should be replaced with one that's freshly downloaded, without an error.

For massive bonus points, NixOS gets an option that hooks into services.zfs.autoScrub and --repair-paths any corrupted files in the Nix store once the scrub completes.

nix-env --version output

nix-env (Nix) 2.11.0

@remexre remexre added the bug label Mar 28, 2023
@SuperSandro2000
Copy link
Member

nix-store --repair-path bails out when it gets these EIOs, and it's not clear whether it's possible to repair the files on a live NixOS system, since /nix/store appears read-only to... users' systemd slices? To sudo under a graphical shell, anyway.

The nix store can repair the path by fetching it from a substituter or if it knows how to build, build it. I think it just needs to handle EIO correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants