yescrypt user passwords do not function

[bbjubjub2494]

Describe the bug

When setting roots hashedPassword to a yescrypt hash, login becomes seemingly impossible even with the correct password. All else being equal, sha512crypt does not exhibit this problem.

Steps To Reproduce

Steps to reproduce the behavior:

1. build this virtual machine: https://gist.github.com/lourkeur/771d190aeb026dba2a60897eab90de57
2. log in as root with password test
3. Womp womp

Expected behavior

Access should be granted iff the password is correct.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

This is a bit concerning given that yescrypt is now the mkpasswd default since #143661.

Should pam be built with the withLibxcrypt flag? If so how?

Notify maintainers

For pam+libxcrypt interaction @dottedmag

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[Artturin]

Related #112371

[Artturin]

If yescrypt is now the default then I consider this a release blocker as we recommend mkpasswd in the manual

@amarshall

[bbjubjub2494]

If yescrypt is now the default then I consider this a release blocker as we recommend mkpasswd in the manual

@amarshall

That might not be true with passwd, but for declarative passwords yes. I think for installation, the manual is still imperative, correct?

[amarshall]

I can at least confirm that enabling withLibxcrypt in linux-pam allows login with yescrypt-hashed passwords. I’m not certain the broader implications of that change, but it’s probably a long-term goal. FWIW the users-groups module presently warns if a yescrypt hash is configured.

Where in the manual does it recommend bare mkpasswd? In NixOS installation it has mkpasswd -m sha-512.

[dottedmag]

Enabling libxcrypt in linux-pam makes at least login and SSH accept yescrypt-hashed passwods.

I'm not sure about the rest of the system: there is too much use of libcrypt.so (see the file attached to #112371) to be sure that all other packages don't access /etc/shadow directly.

[Artturin]

I can at least confirm that enabling withLibxcrypt in linux-pam allows login with yescrypt-hashed passwords. I’m not certain the broader implications of that change, but it’s probably a long-term goal. FWIW the users-groups module presently warns if a yescrypt hash is configured.

Where in the manual does it recommend bare mkpasswd? In NixOS installation it has mkpasswd -m sha-512.

Ah nevermind then it's not as urgent now

[mweinelt]

Fixed in #195271.