Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

top-level/impure.nix: fix overlay directory check #249165

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

arcnmx
Copy link
Member

@arcnmx arcnmx commented Aug 14, 2023

Description of changes

nix 2.16 and newer return true for pathExists (path + "/.") regardless of whether path is a file or directory, resulting in a new error after updating:

error: /home/arc/.config/nixpkgs/overlays.nix should be a file

This issue likely also affects stable nixpkgs (when evaluated with a newer nix) and may be worth backporting.

This implementation of isDir was stolen from filesystem.nix

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Copy link
Member

@solson solson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to get this merged soon. Nix caused the regression but the merged fix from today wouldn't even fix this issue (because it only checks for trailing /): NixOS/nix#8869

And I'd rather not wait around for Nix to unbreak this regression if we can fix it more quickly in Nixpkgs.

@infinisil
Copy link
Member

We can just use lib.filesystem.isDirectory instead. It won't be a problem to import lib in this file

@solson
Copy link
Member

solson commented Sep 1, 2023

Ah, I assumed that would be circular (probably biased by thinking the original author had a good reason to redefine isDir inline), but I guess it will work just like this one in default.nix?

lib = import ../../lib;

Edit: I've confirmed locally that using isDir = (import ../../lib).filesystem.pathIsDirectory; (one-line diff) works and fixes my issues with overlays.

@infinisil
Copy link
Member

Actually no we can't do that, because lib.filesystem.pathIsDirectory always treats symlinks as non-directories, even if they link to a directory.

That's the main reason this pathExists (toString ./path + "/") hack is necessary.

Comment on lines 40 to 44
isDir = path: let
parent = dirOf path;
base = baseNameOf path;
type = (builtins.readDir parent).${base} or null;
in (/. + path) == /. || type == "directory";
Copy link
Member

@infinisil infinisil Sep 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
isDir = path: let
parent = dirOf path;
base = baseNameOf path;
type = (builtins.readDir parent).${base} or null;
in (/. + path) == /. || type == "directory";
isDir = path: builtins.pathExists (path + "/");

We can just remove the . for now, it was never necessary in the first place I'm pretty sure.

@arcnmx
Copy link
Member Author

arcnmx commented Sep 1, 2023

We can just remove the . for now, it was never necessary in the first place I'm pretty sure.

That seems like a reasonable partial mitigation for now... Though I think we should prefer using lib.filesystem.pathType path anyway and only fall back to using the hack when it == "symlink".

Then I'd argue that we should remove most of the checking here. Given that the check is an unreliable hack, there's no real benefit of displaying a message like

error: ~/.config/nixpkgs/overlays.nix should be a file

vs

error: opening directory '~/.config/nixpkgs/overlays.nix': Not a directory

(we may still need a form of isDir to handle <nixpkgs-overlays> properly but if we can avoid using it for homeOverlaysFile and homeOverlaysDir then that eliminates most of our problems)

@infinisil
Copy link
Member

It would be great to have a lib.filesystem.isDirectory as described in #224834 (comment)

nix 2.16 and newer return true for `pathExists (path + "/.")` regardless
of whether `path` is a file or directory.
Comment on lines +42 to +45
isDir = path: {
symlink = builtins.pathExists (toString path + "/");
directory = true;
}.${pathType path} or false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
isDir = path: {
symlink = builtins.pathExists (toString path + "/");
directory = true;
}.${pathType path} or false;
isDir = path: {
# This is hacky but the only way to check the path type of a symlink target
symlink = builtins.pathExists (toString path + "/");
directory = true;
}.${pathType path} or false;

Comment on lines -68 to +79
if isDir homeOverlaysFile then
throw (homeOverlaysFile + " should be a file")
else overlays homeOverlaysFile
overlaysFile homeOverlaysFile
else if builtins.pathExists homeOverlaysDir then
if !(isDir homeOverlaysDir) then
throw (homeOverlaysDir + " should be a directory")
else overlays homeOverlaysDir
overlaysDir homeOverlaysDir
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This leads to a worse error message, the isDir check should be kept because of that.

mjhoy added a commit to mjhoy/dotfiles that referenced this pull request Nov 18, 2023
Nix 2.17.x appears to fail when encountering an overlays.nix file
through a symbolic link. This PR may address although I haven't
tested: NixOS/nixpkgs#249165

For now, stick to Nix 2.16.x which seems to work.
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants