New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Use `meta.tests` to link from packages to the tests that test them #44439

Open
wants to merge 5 commits into
base: master
from

Conversation

Projects
None yet
7 participants
@Ekleog
Contributor

Ekleog commented Aug 4, 2018

Rationale

Currently, tests are hard to discover. For instance, someone updating
dovecot might not notice that the interaction of dovecot with
opensmtpd is handled in the opensmtpd.nix test.

And even for someone updating opensmtpd, it requires manual work to go
check in nixos/tests whether there is actually a test, especially
given not so many packages in nixpkgs have tests and this is thus most
of the time useless.

Finally, for the reviewer, it is much easier to check that the “Tested
via one or more NixOS test(s)” has been checked if the file modified
already includes the list of relevant tests.

Implementation

Currently, this commit only adds the metadata in the package. Each
element of the meta.tests attribute is a derivation, that, when it
builds successfully, means the test has passed (ie. following the same
convention as NixOS tests).

Future Work

In the future, the tools could be made aware of this meta.tests
attribute, and for instance a --with-tests could be added to
nix-build so that it also builds all the tests. Or a --without-tests
to build without all the tests. @Profpatsch described in his NixCon talk
such systems.

Another thing that would help in the future would be the possibility to
reasonably easily have cross-derivation nix tests without the whole
NixOS VM stack. @7c6f434c already proposed such a system.

This RFC currently handles none of these concerns. Only the addition of
meta.tests as metadata to be used by maintainers to remember to run
relevant tests.

Still to do before merge

  • Use attribute paths instead of raw imports with ../../.. (likely requires a refactoring of release.nix to put it inside the fix-point)
  • Add documentation
  • Add metadata in the meta.tests checker like @Profpatsch's code
dovecot, opensmtpd: add link to test in `meta.tests`
Rationale
---------

Currently, tests are hard to discover. For instance, someone updating
`dovecot` might not notice that the interaction of `dovecot` with
`opensmtpd` is handled in the `opensmtpd.nix` test.

And even for someone updating `opensmtpd`, it requires manual work to go
check in `nixos/tests` whether there is actually a test, especially
given not so many packages in `nixpkgs` have tests and this is thus most
of the time useless.

Finally, for the reviewer, it is much easier to check that the “Tested
via one or more NixOS test(s)” has been checked if the file modified
already includes the list of relevant tests.

Implementation
--------------

Currently, this commit only adds the metadata in the package. Each
element of the `meta.tests` attribute is a derivation, that, when it
builds successfully, means the test has passed (ie. following the same
convention as NixOS tests).

Future Work
-----------

In the future, the tools could be made aware of this `meta.tests`
attribute, and for instance a `--with-tests` could be added to
`nix-build` so that it also builds all the tests. Or a `--without-tests`
to build without all the tests. @Profpatsch described in his NixCon talk
such systems.

Another thing that would help in the future would be the possibility to
reasonably easily have cross-derivation nix tests without the whole
NixOS VM stack. @7c6f434c already proposed such a system.

This RFC currently handles none of these concerns. Only the addition of
`meta.tests` as metadata to be used by maintainers to remember to run
relevant tests.
@samueldr

This comment has been minimized.

Show comment
Hide comment
@samueldr

samueldr Aug 4, 2018

Member

Any trick in mind to make it more palatable than a path full of ../ that will invariably change depending on the depth of the derivation? The tests are already being put in an attrset for release, maybe it could be reused or moved into a more reusable place?

Member

samueldr commented Aug 4, 2018

Any trick in mind to make it more palatable than a path full of ../ that will invariably change depending on the depth of the derivation? The tests are already being put in an attrset for release, maybe it could be reused or moved into a more reusable place?

@Ekleog

This comment has been minimized.

Show comment
Hide comment
@Ekleog

Ekleog Aug 4, 2018

Contributor

Hmm… From what I understand, it'd require a refactoring from release.nix, as that's not readily available from the fixed point? If so, maybe doing it once the overall design is more or less consensual would be better, in order for the refactoring not to hide the important parts during discussion :) Thank you for the idea!

Contributor

Ekleog commented Aug 4, 2018

Hmm… From what I understand, it'd require a refactoring from release.nix, as that's not readily available from the fixed point? If so, maybe doing it once the overall design is more or less consensual would be better, in order for the refactoring not to hide the important parts during discussion :) Thank you for the idea!

@7c6f434c

This comment has been minimized.

Show comment
Hide comment
@7c6f434c

7c6f434c Aug 4, 2018

Member

I think such a PR should include some addition to documentation and to validation scripts in Nixpkgs tree (I don't remember the precise location of this code) — unknown meta entries get reported as typos by various tools.

Member

7c6f434c commented Aug 4, 2018

I think such a PR should include some addition to documentation and to validation scripts in Nixpkgs tree (I don't remember the precise location of this code) — unknown meta entries get reported as typos by various tools.

@aszlig

This comment has been minimized.

Show comment
Hide comment
@aszlig

aszlig Aug 4, 2018

Member

I'm not quite sure whether this really should belong together, because the NixOS VM tests are more tailured towards the NixOS service, so tests should ideally be linked to the NixOS module (possibly in a way so you can easily say "run all tests for the current configuration", prototype here) rather than to the package. If the test is really a test for the package itself (like unit tests etc.), then something like this sounds more appropriate.

If this is only for running tests on pull requests, this sounds better, but I also don't like the ../../..., maybe use a list of attribute paths instead?

Member

aszlig commented Aug 4, 2018

I'm not quite sure whether this really should belong together, because the NixOS VM tests are more tailured towards the NixOS service, so tests should ideally be linked to the NixOS module (possibly in a way so you can easily say "run all tests for the current configuration", prototype here) rather than to the package. If the test is really a test for the package itself (like unit tests etc.), then something like this sounds more appropriate.

If this is only for running tests on pull requests, this sounds better, but I also don't like the ../../..., maybe use a list of attribute paths instead?

@aszlig

This comment has been minimized.

Show comment
Hide comment
@aszlig

aszlig Aug 4, 2018

Member

Using attribute paths btw. also has the benefit that nix-instantiate --eval --strict '<nixpkgs>' -A dovecot.meta will work, IIRC something like this was (or is) used in Hydra (edit: hydra-eval-jobs.cc:37).

Member

aszlig commented Aug 4, 2018

Using attribute paths btw. also has the benefit that nix-instantiate --eval --strict '<nixpkgs>' -A dovecot.meta will work, IIRC something like this was (or is) used in Hydra (edit: hydra-eval-jobs.cc:37).

@7c6f434c

This comment has been minimized.

Show comment
Hide comment
@7c6f434c

7c6f434c Aug 4, 2018

Member

@aszlig if some package is disproportionately likely to break a NixOS test, we might want to schedule that test anyway.

If the test is executed in a background batch together with some rebuilds, and the test is already debugged for other reasons, VM overhead is less of an issue.

Member

7c6f434c commented Aug 4, 2018

@aszlig if some package is disproportionately likely to break a NixOS test, we might want to schedule that test anyway.

If the test is executed in a background batch together with some rebuilds, and the test is already debugged for other reasons, VM overhead is less of an issue.

@Profpatsch

This comment has been minimized.

Show comment
Hide comment
@Profpatsch

Profpatsch Aug 4, 2018

Member

We (though I don’t remember who besides me) already had a type cobbled together that these tests should have:
Profpatsch@d74203a#diff-096a3f0fd04e519accb1cde6c483e72bR179

In plain words, tests should be an attribute set in meta that contains tests, where a test is a derivation that has a passthrough boolean field isVmTest (this way test runners are able to filter). That’s just a first idea of course.

Member

Profpatsch commented Aug 4, 2018

We (though I don’t remember who besides me) already had a type cobbled together that these tests should have:
Profpatsch@d74203a#diff-096a3f0fd04e519accb1cde6c483e72bR179

In plain words, tests should be an attribute set in meta that contains tests, where a test is a derivation that has a passthrough boolean field isVmTest (this way test runners are able to filter). That’s just a first idea of course.

@Ekleog

This comment has been minimized.

Show comment
Hide comment
@Ekleog

Ekleog Aug 4, 2018

Contributor

@7c6f434c I think the second commit of this PR (added after I noticed the ofborg check failed) should do the “updating the validation scripts” part :)


@aszlig To me, the point of meta.tests is to be run before accepting a package update into nixpkgs, indeed, given unit tests are (I think?) supposed to be run in the checkPhase. The aim is mostly to allow a single test (like here the OpenSMTPD test) to serve for two derivations (here, opensmtpd and dovecot) :)

I think a scheme, like you put forward, of tying tests to a NixOS configuration, would be useful too, but I'm not sure it'd fit the same aim, as your scheme appears to be more oriented towards individuals accepting (or not) a channel bump as an upgrade :)

Basically, here I'm using a NixOS test because that's all I have to smoke-test that OpenSMTPD / dovecot appear to be working relatively consistently (ie. are able to talk to each over on something that at least look like resp. SMTP and binary-MDA). It's not particularly tied to NixOS in my opinion, but NixOS is (currently) the only testing framework that we have :)


@Profpatsch Your proposal appears much more complete than mine! would you rather I close this PR and you open one about it, or I include your changes into mine?

Also, there are still one point I'm not fully comfortable with with your check-meta update (apart from the foo attribute, that I guess is a WIP test element?): isVmTest.

There are two types of test I can think of off the top of my head that could be used right now:

  1. NixOS tests
  2. Dependencies (eg. a library setting a light dependency as a smoke test to check it hasn't completely broken its API)

And potentially in the future, there could be a testing scheme that doesn't require a full-blown NixOS system, like the one @7c6f434c proposed (test category 3).

I think the filtering could be done along two axes:

  1. How long does it take to run the build? (the equivalent of your isVmTest, I guess, but test category 3 could also potentially take long)
  2. Should tests be recursive? (ie. should the tests of the test packages be executed? this could be useful for test category 2, eg. openssl could declare curl as a recursive-test, and curl would have a test that checks it against an https nginx -- this would smoke-test that the API of openssl appears not to have broken enough to break curl)

I'm not completely sure about case 2 (it could be done by directly including the test case for curl against nginx for instance), but I think anyway isVmTest could (should?) be renamed isSlowTest.

What do you think about this? :)


I've added the still-to-do points to the top post, will try to get some done tomorrow :)

Contributor

Ekleog commented Aug 4, 2018

@7c6f434c I think the second commit of this PR (added after I noticed the ofborg check failed) should do the “updating the validation scripts” part :)


@aszlig To me, the point of meta.tests is to be run before accepting a package update into nixpkgs, indeed, given unit tests are (I think?) supposed to be run in the checkPhase. The aim is mostly to allow a single test (like here the OpenSMTPD test) to serve for two derivations (here, opensmtpd and dovecot) :)

I think a scheme, like you put forward, of tying tests to a NixOS configuration, would be useful too, but I'm not sure it'd fit the same aim, as your scheme appears to be more oriented towards individuals accepting (or not) a channel bump as an upgrade :)

Basically, here I'm using a NixOS test because that's all I have to smoke-test that OpenSMTPD / dovecot appear to be working relatively consistently (ie. are able to talk to each over on something that at least look like resp. SMTP and binary-MDA). It's not particularly tied to NixOS in my opinion, but NixOS is (currently) the only testing framework that we have :)


@Profpatsch Your proposal appears much more complete than mine! would you rather I close this PR and you open one about it, or I include your changes into mine?

Also, there are still one point I'm not fully comfortable with with your check-meta update (apart from the foo attribute, that I guess is a WIP test element?): isVmTest.

There are two types of test I can think of off the top of my head that could be used right now:

  1. NixOS tests
  2. Dependencies (eg. a library setting a light dependency as a smoke test to check it hasn't completely broken its API)

And potentially in the future, there could be a testing scheme that doesn't require a full-blown NixOS system, like the one @7c6f434c proposed (test category 3).

I think the filtering could be done along two axes:

  1. How long does it take to run the build? (the equivalent of your isVmTest, I guess, but test category 3 could also potentially take long)
  2. Should tests be recursive? (ie. should the tests of the test packages be executed? this could be useful for test category 2, eg. openssl could declare curl as a recursive-test, and curl would have a test that checks it against an https nginx -- this would smoke-test that the API of openssl appears not to have broken enough to break curl)

I'm not completely sure about case 2 (it could be done by directly including the test case for curl against nginx for instance), but I think anyway isVmTest could (should?) be renamed isSlowTest.

What do you think about this? :)


I've added the still-to-do points to the top post, will try to get some done tomorrow :)

@Profpatsch

This comment has been minimized.

Show comment
Hide comment
@Profpatsch

Profpatsch Aug 4, 2018

Member

What do you think about this? :)

I think we should basically have an open record of properties for tests.

let
  testProperties = productOpt {
    open = true;
    req = {
      # times mostly defined by test runners, comments are proposals
      # test runners might decide to cap after certain time
      runningTime = sum {
        short = unit; # under 1 minute
        medium = unit; # under 5 minutes
        long = unit; # under 15 minutes
        unbound = unit; # more than 15 minutes
        maxMinutes = int;
      };

      needsVMSupport = bool;
    };
  };
in testProperties

These options are then used by the test runner to decide which tests to run and how long the run can last until it is aborted. More options might be added later.

(note: I’m using my types-simple module to define these types, which is not in nixpkgs (yet))

Member

Profpatsch commented Aug 4, 2018

What do you think about this? :)

I think we should basically have an open record of properties for tests.

let
  testProperties = productOpt {
    open = true;
    req = {
      # times mostly defined by test runners, comments are proposals
      # test runners might decide to cap after certain time
      runningTime = sum {
        short = unit; # under 1 minute
        medium = unit; # under 5 minutes
        long = unit; # under 15 minutes
        unbound = unit; # more than 15 minutes
        maxMinutes = int;
      };

      needsVMSupport = bool;
    };
  };
in testProperties

These options are then used by the test runner to decide which tests to run and how long the run can last until it is aborted. More options might be added later.

(note: I’m using my types-simple module to define these types, which is not in nixpkgs (yet))

@GrahamcOfBorg

This comment has been minimized.

Show comment
Hide comment
@GrahamcOfBorg

GrahamcOfBorg Aug 5, 2018

Success on aarch64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

copying path '/nix/store/qksj8mx1cdvhhblqxshjzfri4yc3qrdb-libevent-2.1.8' from 'https://cache.nixos.org'...
copying path '/nix/store/ljs86xdssnqnqvpa6xnyhvkzfv6w4prg-libsodium-1.0.16' from 'https://cache.nixos.org'...
copying path '/nix/store/29j6kf19lxj81hyb6mbzqp42w9s47vx9-lz4-1.8.2' from 'https://cache.nixos.org'...
copying path '/nix/store/gd222hhrhwm2rjzd7c4vi55vfxrrg5zk-cyrus-sasl-2.1.26' from 'https://cache.nixos.org'...
copying path '/nix/store/v2hsgafk6wgrbgzrai05wnncwf9dwkzm-libtool-2.4.6-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/wj214jamqiysbkw5dbvpgyvdnmlpkahr-openldap-2.4.45' from 'https://cache.nixos.org'...
copying path '/nix/store/vi9n4cszz9rqfmf3p5c8chvqv2pviywb-opensmtpd-6.0.3p1' from 'https://cache.nixos.org'...
copying path '/nix/store/8dyq4hg5p6c074kcxbrcl34miwg6n5bf-dovecot-2.3.2.1' from 'https://cache.nixos.org'...
/nix/store/8dyq4hg5p6c074kcxbrcl34miwg6n5bf-dovecot-2.3.2.1
/nix/store/vi9n4cszz9rqfmf3p5c8chvqv2pviywb-opensmtpd-6.0.3p1

GrahamcOfBorg commented Aug 5, 2018

Success on aarch64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

copying path '/nix/store/qksj8mx1cdvhhblqxshjzfri4yc3qrdb-libevent-2.1.8' from 'https://cache.nixos.org'...
copying path '/nix/store/ljs86xdssnqnqvpa6xnyhvkzfv6w4prg-libsodium-1.0.16' from 'https://cache.nixos.org'...
copying path '/nix/store/29j6kf19lxj81hyb6mbzqp42w9s47vx9-lz4-1.8.2' from 'https://cache.nixos.org'...
copying path '/nix/store/gd222hhrhwm2rjzd7c4vi55vfxrrg5zk-cyrus-sasl-2.1.26' from 'https://cache.nixos.org'...
copying path '/nix/store/v2hsgafk6wgrbgzrai05wnncwf9dwkzm-libtool-2.4.6-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/wj214jamqiysbkw5dbvpgyvdnmlpkahr-openldap-2.4.45' from 'https://cache.nixos.org'...
copying path '/nix/store/vi9n4cszz9rqfmf3p5c8chvqv2pviywb-opensmtpd-6.0.3p1' from 'https://cache.nixos.org'...
copying path '/nix/store/8dyq4hg5p6c074kcxbrcl34miwg6n5bf-dovecot-2.3.2.1' from 'https://cache.nixos.org'...
/nix/store/8dyq4hg5p6c074kcxbrcl34miwg6n5bf-dovecot-2.3.2.1
/nix/store/vi9n4cszz9rqfmf3p5c8chvqv2pviywb-opensmtpd-6.0.3p1

@GrahamcOfBorg

This comment has been minimized.

Show comment
Hide comment
@GrahamcOfBorg

GrahamcOfBorg Aug 5, 2018

Success on x86_64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

  /nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1
  /nix/store/5h9kpirrh5fw9imblwxm4d1lr8acwsk6-clucene-core-2.3.3.4
  /nix/store/llyvxfx789yrwbljy3pig9q7al64bg5k-libasr-1.0.2
  /nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1
copying path '/nix/store/5h9kpirrh5fw9imblwxm4d1lr8acwsk6-clucene-core-2.3.3.4' from 'https://cache.nixos.org'...
copying path '/nix/store/llyvxfx789yrwbljy3pig9q7al64bg5k-libasr-1.0.2' from 'https://cache.nixos.org'...
copying path '/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1' from 'https://cache.nixos.org'...
copying path '/nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1' from 'https://cache.nixos.org'...
/nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1
/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1

GrahamcOfBorg commented Aug 5, 2018

Success on x86_64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

  /nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1
  /nix/store/5h9kpirrh5fw9imblwxm4d1lr8acwsk6-clucene-core-2.3.3.4
  /nix/store/llyvxfx789yrwbljy3pig9q7al64bg5k-libasr-1.0.2
  /nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1
copying path '/nix/store/5h9kpirrh5fw9imblwxm4d1lr8acwsk6-clucene-core-2.3.3.4' from 'https://cache.nixos.org'...
copying path '/nix/store/llyvxfx789yrwbljy3pig9q7al64bg5k-libasr-1.0.2' from 'https://cache.nixos.org'...
copying path '/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1' from 'https://cache.nixos.org'...
copying path '/nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1' from 'https://cache.nixos.org'...
/nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1
/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1

@Ekleog

This comment has been minimized.

Show comment
Hide comment
@Ekleog

Ekleog Aug 5, 2018

Contributor

I think all the points that were requiring fixing are now fixed :)

@Profpatsch I have re-used the pre-existing timeout meta value for test timeouts, and put needsVMSupport in meta (from the idea that it may be useful for non-test derivations too). That said, the way I used of enforcing these two attributes are present in test derivations is currently pretty ugly… but until your types-simple gets merged I'm not sure it's possible to do much better :/

Contributor

Ekleog commented Aug 5, 2018

I think all the points that were requiring fixing are now fixed :)

@Profpatsch I have re-used the pre-existing timeout meta value for test timeouts, and put needsVMSupport in meta (from the idea that it may be useful for non-test derivations too). That said, the way I used of enforcing these two attributes are present in test derivations is currently pretty ugly… but until your types-simple gets merged I'm not sure it's possible to do much better :/

@GrahamcOfBorg

This comment has been minimized.

Show comment
Hide comment
@GrahamcOfBorg

GrahamcOfBorg Aug 5, 2018

Success on aarch64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

/nix/store/8dyq4hg5p6c074kcxbrcl34miwg6n5bf-dovecot-2.3.2.1
/nix/store/vi9n4cszz9rqfmf3p5c8chvqv2pviywb-opensmtpd-6.0.3p1

GrahamcOfBorg commented Aug 5, 2018

Success on aarch64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

/nix/store/8dyq4hg5p6c074kcxbrcl34miwg6n5bf-dovecot-2.3.2.1
/nix/store/vi9n4cszz9rqfmf3p5c8chvqv2pviywb-opensmtpd-6.0.3p1

@GrahamcOfBorg

This comment has been minimized.

Show comment
Hide comment
@GrahamcOfBorg

GrahamcOfBorg Aug 5, 2018

Success on x86_64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

  /nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1
  /nix/store/5h9kpirrh5fw9imblwxm4d1lr8acwsk6-clucene-core-2.3.3.4
  /nix/store/llyvxfx789yrwbljy3pig9q7al64bg5k-libasr-1.0.2
  /nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1
copying path '/nix/store/5h9kpirrh5fw9imblwxm4d1lr8acwsk6-clucene-core-2.3.3.4' from 'https://cache.nixos.org'...
copying path '/nix/store/llyvxfx789yrwbljy3pig9q7al64bg5k-libasr-1.0.2' from 'https://cache.nixos.org'...
copying path '/nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1' from 'https://cache.nixos.org'...
copying path '/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1' from 'https://cache.nixos.org'...
/nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1
/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1

GrahamcOfBorg commented Aug 5, 2018

Success on x86_64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

  /nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1
  /nix/store/5h9kpirrh5fw9imblwxm4d1lr8acwsk6-clucene-core-2.3.3.4
  /nix/store/llyvxfx789yrwbljy3pig9q7al64bg5k-libasr-1.0.2
  /nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1
copying path '/nix/store/5h9kpirrh5fw9imblwxm4d1lr8acwsk6-clucene-core-2.3.3.4' from 'https://cache.nixos.org'...
copying path '/nix/store/llyvxfx789yrwbljy3pig9q7al64bg5k-libasr-1.0.2' from 'https://cache.nixos.org'...
copying path '/nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1' from 'https://cache.nixos.org'...
copying path '/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1' from 'https://cache.nixos.org'...
/nix/store/0v05paacys9bnb9fxvvhwk8j5mqg34lw-dovecot-2.3.2.1
/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1

@GrahamcOfBorg

This comment has been minimized.

Show comment
Hide comment
@GrahamcOfBorg

GrahamcOfBorg Aug 7, 2018

Success on x86_64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

shrinking /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/sbin/dovecot
gzipping man pages under /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/share/man/
strip is /nix/store/1hi76hr87bd1y1q1qjk0lv8nmcjip1c8-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/lib  /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/libexec  /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/bin  /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/sbin
patching script interpreter paths in /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1
/nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/dkh7l9a4sx7zqh8riqbj3z21sz25p8xy-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1...
moving /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/sbin/* to /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/bin
/nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1
/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1

GrahamcOfBorg commented Aug 7, 2018

Success on x86_64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

shrinking /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/sbin/dovecot
gzipping man pages under /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/share/man/
strip is /nix/store/1hi76hr87bd1y1q1qjk0lv8nmcjip1c8-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/lib  /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/libexec  /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/bin  /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/sbin
patching script interpreter paths in /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1
/nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/dkh7l9a4sx7zqh8riqbj3z21sz25p8xy-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1...
moving /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/sbin/* to /nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1/bin
/nix/store/zjmkciznnc2n4vvkpnq7zqvwv25iry41-dovecot-2.3.2.1
/nix/store/lmjpbp2vh6k6via7lyb92zxjfb3gpw7j-opensmtpd-6.0.3p1

@GrahamcOfBorg

This comment has been minimized.

Show comment
Hide comment
@GrahamcOfBorg

GrahamcOfBorg Aug 7, 2018

Success on aarch64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

shrinking /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/bin/doveadm
gzipping man pages under /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/share/man/
strip is /nix/store/zrs21zqcchgyabjf4xfimncdq16njizc-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/lib  /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/libexec  /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/bin  /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/sbin
patching script interpreter paths in /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1
/nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/6vqgi9d49smsbr2qxra52yhipg0yxf9f-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1...
moving /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/sbin/* to /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/bin
/nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1
/nix/store/vi9n4cszz9rqfmf3p5c8chvqv2pviywb-opensmtpd-6.0.3p1

GrahamcOfBorg commented Aug 7, 2018

Success on aarch64-linux (full log)

Attempted: dovecot, opensmtpd

Partial log (click to expand)

shrinking /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/bin/doveadm
gzipping man pages under /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/share/man/
strip is /nix/store/zrs21zqcchgyabjf4xfimncdq16njizc-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/lib  /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/libexec  /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/bin  /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/sbin
patching script interpreter paths in /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1
/nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/6vqgi9d49smsbr2qxra52yhipg0yxf9f-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1...
moving /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/sbin/* to /nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1/bin
/nix/store/z17hyaagddgxlpncxmqk6a70hn2kh27d-dovecot-2.3.2.1
/nix/store/vi9n4cszz9rqfmf3p5c8chvqv2pviywb-opensmtpd-6.0.3p1

@Profpatsch

Not sure, but I think this needs to use some cross-stuff.

nixosTests =
let
system = builtins.currentSystem;

This comment has been minimized.

@Profpatsch

Profpatsch Aug 7, 2018

Member

This doesn’t work, I think. cc @Ericson2314

@Profpatsch

Profpatsch Aug 7, 2018

Member

This doesn’t work, I think. cc @Ericson2314

This comment has been minimized.

@Ekleog

Ekleog Aug 23, 2018

Contributor

I don't really know: NixOS test derivations are supposed to be built on the local system, which I'd guess to be builtins.currentSystem? Not sure at all though, so cc @Ericson2314 indeed :) I.must find it hard to imagine what's the cross-derivation story of NIxOS tests.

@Ekleog

Ekleog Aug 23, 2018

Contributor

I don't really know: NixOS test derivations are supposed to be built on the local system, which I'd guess to be builtins.currentSystem? Not sure at all though, so cc @Ericson2314 indeed :) I.must find it hard to imagine what's the cross-derivation story of NIxOS tests.

@Ekleog Ekleog referenced this pull request Aug 8, 2018

Merged

Call for content: 2018/06 #61

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment