nixos/tests/dendrite: init #121777

mjlbach · 2021-05-05T08:19:46Z

Motivation for this change

Matrix-nio is actively maintained, but also didn't support dendrite until tonight. I filed an upstream PR to fix this which was recently merged matrix-nio/matrix-nio#254. We'll need to wait until upstream cuts the next release in order to verify/merge this test (it worked on my local machine).

Things done

cc @dotlambda @infinisil

dotlambda · 2021-05-05T08:52:29Z

In light of #121728 (comment), can we please add TLS to the test with a non world-readable key? That should fail atm.

mjlbach · 2021-05-05T08:54:24Z

Yes, I can add that this weekend

mjlbach · 2021-05-06T18:02:36Z

Depends on #121942

mjlbach · 2021-05-09T15:29:58Z

My nixpkgs PR bumping matrix-nio is now in master, I'm cleaning up and including a couple additional features (including the TLS case). I assume the latter will fail, so I'll also add additional option we discussed on matrix to the systemd service

dotlambda

Please substitute matrix-dendrite with dendrite (also in the filename) and add the test to nixos/tests/all-tests.nix as well as dendrite's passthru.tests.

mjlbach · 2021-05-13T16:24:22Z

I went ahead and updated the test. Questions:

How do I test the pastthru test? I've validated the test works with NIX_PATH=nixpkgs=$(pwd) nix-build '<nixpkgs/nixos/tests/dendrite.nix>'
Will the passthru need to be special cased for darwin, as I'm not sure the test will work.

Re: hardening the module and adding a test, I'll do that in a separate PR.

dotlambda · 2021-05-13T16:56:27Z

pkgs/servers/dendrite/default.nix

@@ -13,6 +13,8 @@ buildGoModule rec {

  vendorSha256 = "1l1wydvi0yalas79cvhrqg563cvs57hg9rv6qnkw879r6smb2x1n";

+  passthru.tests = nixosTests.dendrite;


Suggested change

passthru.tests = nixosTests.dendrite;

passthru.tests = {

inherit (nixosTests) dendrite;

}

Is there a reason for this? The former pattern is very common in nixpkgs, and we aren't inheriting multiple tests.

It's what every other package does.

This isn't true, but I've changed it to what you suggested:

pkgs/servers/web-apps/engelsystem/default.nix 40: passthru.tests = nixosTests.engelsystem; pkgs/servers/keycloak/default.nix 58: passthru.tests = nixosTests.keycloak; pkgs/servers/miniflux/default.nix 34: passthru.tests = nixosTests.miniflux; pkgs/servers/pinnwand/steck.nix 25: passthru.tests = nixosTests.pinnwand; pkgs/servers/pinnwand/default.nix 46: passthru.tests = nixosTests.pinnwand; pkgs/servers/nextcloud/default.nix 16: passthru.tests = nixosTests.nextcloud; pkgs/applications/audio/musescore/default.nix 44: passthru.tests = nixosTests.musescore; pkgs/applications/networking/irc/convos/default.nix 83: passthru.tests = nixosTests.convos; pkgs/applications/misc/keepassx/community.nix 123: passthru.tests = nixosTests.keepassxc; pkgs/os-specific/linux/firejail/default.nix 72: passthru.tests = nixosTests.firejail; pkgs/os-specific/linux/usbguard/default.nix 77: passthru.tests = nixosTests.usbguard; pkgs/tools/security/bitwarden_rs/vault.nix 19: passthru.tests = nixosTests.bitwarden; pkgs/tools/security/bitwarden_rs/default.nix 38: passthru.tests = nixosTests.bitwarden;

You're right, but most of them do it correctly, i.e. as documented: https://nixos.org/manual/nixpkgs/unstable/#var-meta-tests.

dotlambda · 2021-05-13T16:57:21Z

nixos/tests/dendrite.nix

+          homeserver.succeed(
+              "/run/current-system/specialisation/running/bin/switch-to-configuration test >&2"
+          )


This shouldn't be necessary.

I just copied synapse, I can change it

This is actually necessary for the test to wait until the service is started, removing it causes the test to fail with

Exception: unit "dendrite.service" is inactive and there are no pending jobs

Maybe we have to do homeserver.wait_for_unit("network.target")?

dotlambda · 2021-05-13T17:00:44Z

How do I test the pastthru test?

nix-build -A dendrite.passthru.tests

Will the passthru need to be special cased for darwin, as I'm not sure the test will work.

No need to, ofborg only runs tests on Linux.

Re: hardening the module and adding a test, I'll do that in a separate PR.

I'm strongly opposed to that. We either fix the module now or remove it from 21.05.

mjlbach · 2021-05-13T17:09:46Z

I'm strongly opposed to that. We either fix the module now or remove it from 21.05.

You are strongly demotivating me to contribute to nixpkgs. You should adjust how you talk to contributors, because I am losing interest in working with you.

I was planning on submitting the hardening PR this weekend. This PR is adding a test. The other PR will add hardening and an associated test. It's normal to scope PRs based on the features they add.
There are modules which leak secrets into the nix store. TLS certs/registration secrets are an optional component of the module.

dotlambda · 2021-05-13T17:26:59Z

@mjlbach I wasn't talking about hardening, sorry if it sounded like I was. I would just like to see a test using tlsCert and tlsKey, and I guess a fix for these options using systemd credentials. Hardening is a nice to have thing and can definitely be done in a separate PR.

mjlbach · 2021-05-13T17:28:59Z

As part of the hardening PR, I'm also going to change the tlsCert/key as we discussed previously using systemd credentials. I just wanted to get the test infrastructure in as I had to fix a lot of things (both upstream, and in the matrix-nio dependency chain). I don't see the harm in having a dedicated PR that enables the test, which I can then expand on when I add tests re: the tls issue.

dotlambda · 2021-05-13T17:30:41Z

Okay, if you do it before 21.05 then it's fine with me. Please change the two things mentioned above though.

Btw, thanks a lot for going as far as fixing stuff in nio for this test!

dotlambda · 2021-05-13T19:54:32Z

nixos/tests/dendrite.nix

+
+    nodes = {
+      homeserver = { pkgs, ... }: {
+        specialisation.running.configuration = {


This is the culprit as it will build the configuration running but not run it. With

Suggested change

specialisation.running.configuration = {

it should work.
The option specialisation can be used for switching configuration within a test.

Should be resolved now, thanks!

github-actions bot added the 6.topic: nixos label May 5, 2021

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels May 5, 2021

dotlambda changed the title ~~matrix-dendrite: add test for nixos module~~ nixos/matrix-dendrite: add test May 5, 2021

mjlbach force-pushed the matrix_dendrite_module_test branch 2 times, most recently from cd439e1 to 22657d4 Compare May 9, 2021 15:28

dotlambda requested changes May 10, 2021

View reviewed changes

dotlambda mentioned this pull request May 12, 2021

21.05 Feature Freeze #121972

Closed

mjlbach force-pushed the matrix_dendrite_module_test branch from 22657d4 to 6136ef9 Compare May 13, 2021 16:22

mjlbach marked this pull request as ready for review May 13, 2021 16:23

mjlbach requested a review from dotlambda May 13, 2021 16:24

dotlambda requested changes May 13, 2021

View reviewed changes

dotlambda changed the title ~~nixos/matrix-dendrite: add test~~ nixos/dendrite: add test May 13, 2021

mjlbach requested a review from Ma27 May 13, 2021 17:17

mjlbach force-pushed the matrix_dendrite_module_test branch from 6136ef9 to 62785de Compare May 13, 2021 17:39

dotlambda reviewed May 13, 2021

View reviewed changes

mjlbach force-pushed the matrix_dendrite_module_test branch from 62785de to 5f00525 Compare May 14, 2021 09:03

matrix-dendrite: add test for nixos module

94e89ad

mjlbach force-pushed the matrix_dendrite_module_test branch from 5f00525 to 94e89ad Compare May 14, 2021 09:06

dotlambda changed the title ~~nixos/dendrite: add test~~ nixos/tests/dendrite: init May 14, 2021

dotlambda merged commit 4628449 into NixOS:master May 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/tests/dendrite: init #121777

nixos/tests/dendrite: init #121777

mjlbach commented May 5, 2021 •

edited

dotlambda commented May 5, 2021

mjlbach commented May 5, 2021

mjlbach commented May 6, 2021

mjlbach commented May 9, 2021 •

edited

dotlambda left a comment

mjlbach commented May 13, 2021

dotlambda May 13, 2021

mjlbach May 13, 2021

dotlambda May 13, 2021

mjlbach May 13, 2021 •

edited

dotlambda May 13, 2021

dotlambda May 13, 2021

mjlbach May 13, 2021

mjlbach May 13, 2021

dotlambda May 13, 2021

dotlambda commented May 13, 2021 •

edited

mjlbach commented May 13, 2021 •

edited

dotlambda commented May 13, 2021

mjlbach commented May 13, 2021 •

edited

dotlambda commented May 13, 2021

dotlambda May 13, 2021

mjlbach May 14, 2021

		@@ -13,6 +13,8 @@ buildGoModule rec {

		vendorSha256 = "1l1wydvi0yalas79cvhrqg563cvs57hg9rv6qnkw879r6smb2x1n";

		passthru.tests = nixosTests.dendrite;

-  passthru.tests = nixosTests.dendrite;
+  passthru.tests = {
+    inherit (nixosTests) dendrite;
+  }

nixos/tests/dendrite: init #121777

nixos/tests/dendrite: init #121777

Conversation

mjlbach commented May 5, 2021 • edited

Motivation for this change

Things done

dotlambda commented May 5, 2021

mjlbach commented May 5, 2021

mjlbach commented May 6, 2021

mjlbach commented May 9, 2021 • edited

dotlambda left a comment

Choose a reason for hiding this comment

mjlbach commented May 13, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjlbach May 13, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda commented May 13, 2021 • edited

mjlbach commented May 13, 2021 • edited

dotlambda commented May 13, 2021

mjlbach commented May 13, 2021 • edited

dotlambda commented May 13, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjlbach commented May 5, 2021 •

edited

mjlbach commented May 9, 2021 •

edited

mjlbach May 13, 2021 •

edited

dotlambda commented May 13, 2021 •

edited

mjlbach commented May 13, 2021 •

edited

mjlbach commented May 13, 2021 •

edited