Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/tests/installer.nix: fix hydra eval #220308

Merged
merged 1 commit into from Mar 10, 2023
Merged

nixos/tests/installer.nix: fix hydra eval #220308

merged 1 commit into from Mar 10, 2023

Conversation

ghost
Copy link

@ghost ghost commented Mar 9, 2023

Description of changes

Hydra Eval has been throwing these eval errors for the past four months, which makes the yellow "Eval Errors" bubble pretty useless:

https://hydra.nixos.org/eval/1790611#tabs-errors

in job ‘nixos.tests.installer.separateBoot.aarch64-linux’:
error: Non-EFI boot methods are only supported on i686 / x86_64

in job ‘nixos.tests.installer.simple.aarch64-linux’:
error: Non-EFI boot methods are only supported on i686 / x86_64

in job ‘nixos.tests.installer.lvm.aarch64-linux’:
error: Non-EFI boot methods are only supported on i686 / x86_64

This PR moves the failure for the !isEfi && !pkgs.stdenv.hostPlatform.isx86 case from eval-time to runtime, so the failure gets categorized under the test that produced it, rather than just being lumped in to the catch-all Eval Errors pile which...

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Hydra Eval has been throwing these eval errors for the past four
months, which makes the yellow "Eval Errors" bubble pretty useless:

https://hydra.nixos.org/eval/1790611#tabs-errors

```
in job ‘nixos.tests.installer.separateBoot.aarch64-linux’:
error: Non-EFI boot methods are only supported on i686 / x86_64

in job ‘nixos.tests.installer.simple.aarch64-linux’:
error: Non-EFI boot methods are only supported on i686 / x86_64

in job ‘nixos.tests.installer.lvm.aarch64-linux’:
error: Non-EFI boot methods are only supported on i686 / x86_64
```

This PR moves the failure for the `!isEfi &&
!pkgs.stdenv.hostPlatform.isx86` case from eval-time to runtime, so
the failure gets categorized under the test that produced it, rather
than just being lumped in to the catch-all Eval Errors pile
which... apparently nobody cares about.
@trofi
Copy link
Contributor

trofi commented Mar 9, 2023

Was added in 3131daa /cc @dezgeg

@ghost
Copy link
Author

ghost commented Mar 10, 2023

Was added in 3131daa /cc @dezgeg

Yes but that's not at fault. Everything was fine until four months ago.

The problem is that Hydra shouldn't even be attempting to eval this job on aarch64.

Unfortunately debugging the Hydra-related parts of nixpkgs basically requires setting up your own Hydra instance, since Hydra expects to have access to the Nix evaluator (rather than just the resulting derivations).

@trofi
Copy link
Contributor

trofi commented Mar 10, 2023

Was added in 3131daa /cc @dezgeg

Yes but that's not at fault. Everything was fine until four months ago.

Aha. I wonder if it's related to addition of aarch64 to the CI. I see that makeInstallerTest has a meta attribute, and nixos/tests/all-tests.nix has handleTestOn to limit the tests down to a subset of architectures.

Otheriwse it should be fine to turn the test into no-op.

@trofi trofi merged commit 93fc62f into NixOS:master Mar 10, 2023
@ghost ghost deleted the pr/hydra/fix-eval branch March 11, 2023 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant