Improvements to pathType, pathIsDirectory and pathIsRegularFile

[infinisil]

Description of changes

This is related to the path library effort.

Improves the lib.sources.pathType, lib.sources.pathIsDirectory and lib.sources.pathIsRegularFile functions, each of which is a single commit:

• Moves them to lib.filesystem, since they only work with the filesystem, they don't import any sources. The functions are still available in lib.sources, but will be deprecated after the next release.
• Add test cases
• Minor refactor
• Fix them for when the filesystem root / is passed. Previously they threw an unintuitive error:

  error: attribute '' missing
    
       at nixpkgs/lib/filesystem.nix:24:20:
    
           23|   */
           24|   pathType = path: (readDir (dirOf path)).${baseNameOf path};
             |                    ^
           25|
  

• Improve the pathType error message for invalid paths. Previously this was the error:

  error: attribute 'nonexistent' missing
    
       at nixpkgs/lib/filesystem.nix:29:10:
    
           28|     if dirOf path == path then "directory"
           29|     else (readDir (dirOf path)).${baseNameOf path};
             |          ^
           30|
  

• Improve documentation, adding a better description, a type and examples.
• Indicate that pathType can get replaced with the new builtins.readFileType once Nix 2.14 is released

This work is sponsored by Antithesis ✨

Things done

• Wrote docs, built and checked the manual (nix-build doc)
• Wrote and ran the tests
  • The new script lib/tests/filesystem.sh on its own
  • And the integration into nix-build lib/tests/release.nix

[infinisil]

Something I don't want to go into for this PR, but maybe in the future: The semantics of pathIsDirectory and pathIsRegularFile are not obvious:

• They return false when the path doesn't exist, which is often not what you want.
• They always return false for symlinks without checking where the symlink points (it could point to a directory or a regular file)

We should consider having alternate functions with cleaner semantics in the future:

• isDirectory :: Path -> Bool: Returns true iff the path is a directory, following symlinks. An error is thrown if the path doesn't exist.
• isFile :: Path -> Bool: Returns true iff the path is a file, following symlinks. An error is thrown if the path doesn't exist.

These laws would then hold:

• If isDirectory path succeeds and is true, then builtins.readDir path succeeds
• If isFile path succeeds and is true, then builtins.readFile path succeeds
• isDirectory path != isFile path

Unfortunately the implementation of these functions would have to be a bit hacky (throwing for non-existent paths not implemented here):

{
  isDirectory = path: builtins.pathExists "${toString path}/";
  isFile = path: ! builtins.pathExists "${toString path}/";
}

This uses the fact that:

• toString can turn paths to strings, which would not be the case with lazy trees, and would violate the minimal assumptions of the path library
• builtins.pathExists works not only on paths, but also on strings, with the important difference that strings aren't normalised, therefore keeping the trailing /

To implement these I'd propose adding more builtins, either:

• builtins.readSymlink: return the target of a symlink
• builtins.isDirectory: A builtin version of the above isDirectory

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-47/27387/1

[roberth]

We should consider having alternate functions with cleaner semantics in the future:

• isDirectory :: Path -> Bool: Returns true iff the path is a directory, following symlinks. An error is thrown if the path doesn't exist.
• isFile :: Path -> Bool: Returns true iff the path is a file, following symlinks. An error is thrown if the path doesn't exist.

That's good. How about this?

• isDirectory :: Path -> Bool: Returns true iff the path is a directory, following symlinks. An error is thrown if the path doesn't exist.
• existsDirectory :: Path -> Bool: Returns pathExists f && isDirectory f
• same for *File

existsDirectory is just a rename of pathIsDirectory, but less ambiguous.

[roberth]

2.14 was not a success, but 2.15 is out now.
This can be upgraded to a polyfill now.

[infinisil]

Done. Also take a look at NixOS/nix#7447 (comment)

[infinisil]

@roberth I'd probably go for directoryExists instead of existsDirectory, this then also mimics pathExists

[roberth]

Some minor suggestions and a less minor idea to improve the developer experience. I'd add the comments myself, but it's better to share the idea.

[github-actions]

Successfully created backport PR for release-23.05:

• [Backport release-23.05] Improvements to pathType, pathIsDirectory and pathIsRegularFile #233560

[github-actions]

Git push to origin failed for release-23.05 with exitcode 1