nixos-rebuild with remote target fails due to lack of valid signature

[felschr]

Describe the bug

Trying to upgrade a remote machine fails with the following output:

$ sudo nixos-rebuild switch --flake '/etc/nixos/#felix-rpi4' --target-host user@host --use-remote-sudo
building the system configuration...
warning: Git tree '/etc/nixos' is dirty
copying 141 paths...
copying path '/nix/store/05rrbmrxpfmb4pj5i9wj2h6q5kbklw3z-system-shutdown' to 'ssh://user@host'...
error: cannot add path '/nix/store/05rrbmrxpfmb4pj5i9wj2h6q5kbklw3z-system-shutdown' because it lacks a valid signature
error: unexpected end-of-file

Steps To Reproduce

See sample code above.

Expected behavior

Remote deployment should succeed.

Additional context

My build system is x86_64 and the target is aarch64. Both systems use flakes.
I've enabled cross-compilation on the build system via boot.binfmt.emulatedSystems = [ "aarch64-linux" ];.
Thus far I've updated the remote machine by itself, but the rebuilds are sometimes quite slow on it (raspberry pi 4). That's why I'd like to build it on my desktop computer which is much faster.

Metadata

build system:

 - system: `"x86_64-linux"`
 - host os: `Linux 5.16.7, NixOS, 22.05 (Quokka)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.7.0pre20220127_558c4ee`
 - nixpkgs: `/nix/store/zwbw7dknw89ai257hicmwv8lhid3qb62-source`

target system:

 - system: `"aarch64-linux"`
 - host os: `Linux 5.10.52, NixOS, 22.05 (Quokka)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.7.0pre20220127_558c4ee`
 - channels(root): `"nixos-21.05pre275232.29b0d4d0b60"`
 - nixpkgs: `/nix/store/pi099gbqmvjaqk6d77c9r9g08mm70w8b-source`

[jkozak]

I had the same problem which I worked around by using --target-host root@host rather than --use-remote-sudo.

Unfortunately that revealed another problem which I'll report in a separate issue.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/way-to-build-nixos-on-x86-64-machine-and-serve-to-aarch64-over-local-network/18660/1

[felschr]

I'd really like to see this working with --use-remote-sudo as well, as I've disabled ssh for root.

[felschr]

While setting up deploy-rs I encountered the same issue again. Turns out, one has to add the user to nix.trustedUsers on the remote, e.g.:

{ nix.trustedUsers = [ "@wheel" ]; } # or specific user