Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port gnome3 like tests to python and udisks2 #72860

Merged
merged 5 commits into from Nov 7, 2019

Conversation

@worldofpeace
Copy link
Member

worldofpeace commented Nov 5, 2019

Motivation for this change

#72828

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @

Copy link
Contributor

jtojnar left a comment

It would be nice to have a function for gnome-desktop-testing-runner too.

nixos/tests/fwupd.nix Outdated Show resolved Hide resolved
# send a dbus message to activate the service
$machine->succeed("dbus-send --system --type=method_call --print-reply --dest=org.freedesktop.PackageKit /org/freedesktop/PackageKit org.freedesktop.DBus.Introspectable.Introspect");
machine.succeed("dbus-send --system --type=method_call --print-reply --dest=org.freedesktop.PackageKit /org/freedesktop/PackageKit org.freedesktop.DBus.Introspectable.Introspect")

This comment has been minimized.

Copy link
@jtojnar

jtojnar Nov 5, 2019

Contributor

Was the dbus API added or will it come later?

This comment has been minimized.

Copy link
@worldofpeace

worldofpeace Nov 5, 2019

Author Member

Do you mean in the testing driver? I don't think so, but I have some great ideas of all the things that should be added there. Currently they've tried to go for a 1:1 port from the perl one.

nixos/tests/packagekit.nix Outdated Show resolved Hide resolved
nixos/tests/udisks2.nix Outdated Show resolved Hide resolved
@worldofpeace

This comment has been minimized.

Copy link
Member Author

worldofpeace commented Nov 5, 2019

It would be nice to have a function for gnome-desktop-testing-runner too.

I can make it similar to #34987 (comment) but maybe makeGnomeInstalledTest and have them in one flat file like what I did for flashback

(description is Make NixOS test for gnome-desktop-testing-runner using software)

@worldofpeace worldofpeace force-pushed the worldofpeace:port-gnome3y-tests branch from 88244d9 to 3be01be Nov 6, 2019
@worldofpeace

This comment has been minimized.

Copy link
Member Author

worldofpeace commented Nov 6, 2019

@jtojnar I've added a gnome-installed-tests.nix with the function and the tests.
That does make all the tests under nixosTests.gnome-installed-tests.*.

Would you prefer having everything in one file like I've done, or perhaps the colord = makeGnomeInstalledTests could be split out?

@jtojnar

This comment has been minimized.

Copy link
Contributor

jtojnar commented Nov 6, 2019

I think one file per test is still a good idea, some of them are slightly more complicated, recall also python-packages.nix. The really trivial ones could perhaps be created directly in all-tests.nix.

I am also not sure about the GnomeInstalledTests name. While it originated in GNOME project, it spread to various freedesktop projects as well.

@worldofpeace

This comment has been minimized.

Copy link
Member Author

worldofpeace commented Nov 6, 2019

I think one file per test is still a good idea, some of them are slightly more complicated, recall also python-packages.nix. The really trivial ones could perhaps be created directly in all-tests.nix.

I am also not sure about the GnomeInstalledTests name. While it originated in GNOME project, it spread to various freedesktop projects as well.

I just wanted the function name to express that it's just for projects that use gnome-desktop-testing.

I will change the structure though, python-packages.nix was kinda a mess and all-tests.nix is structured that way for a similar reason.

@worldofpeace worldofpeace force-pushed the worldofpeace:port-gnome3y-tests branch from 3be01be to 2aaf62a Nov 6, 2019
@FRidh

This comment has been minimized.

Copy link
Member

FRidh commented Nov 6, 2019

I am a bit confused by the moving of the tests. In the package set, I thought the plan was to get rid of the gnome3 namespace. Why not with tests then?

@jtojnar

This comment has been minimized.

Copy link
Contributor

jtojnar commented Nov 6, 2019

gnome-desktop-testing is just a single implementation of the test runner but the installed spec itself is not specific to GNOME. There is at least ibus-desktop-testing-runner. I plan to add that in #71442

@worldofpeace worldofpeace force-pushed the worldofpeace:port-gnome3y-tests branch from 2aaf62a to 189ff39 Nov 6, 2019
@worldofpeace

This comment has been minimized.

Copy link
Member Author

worldofpeace commented Nov 6, 2019

From IRC:

@worldofpeace
jtojnar: what are the requested changes in #72860 (comment)? I just wanted a good overview of the installed tests specifically, and having them under an attr nixosTests.gnome-installed-tests.* just made that easy. No relation to the gnome3 attr deprecation

@jtojnar
worldofpeace I would just use nixosTests.installed-tests, and maybe pester upstream to drop gnome from its name when I have some time

@worldofpeace
jtojnar: did that 👍️

nixos/tests/udisks2.nix Outdated Show resolved Hide resolved
$machine->succeed("udisksctl info -b /dev/vda >&2");
$machine->fail("udisksctl info -b /dev/sda1");
with lzma.open(

This comment has been minimized.

Copy link
@jtojnar

jtojnar Nov 6, 2019

Contributor

Why space this around, "${stick}" is much shorter than the line below.

This comment has been minimized.

Copy link
@worldofpeace

worldofpeace Nov 6, 2019

Author Member

black fails on line length, as this becomes a nix store path so it demands I format it this way. We should probably make that more relaxed. Similarly I've had to format makeInstalledTest testscript injection specifically with concatenating multi-line strings so it doesn't error out on it not being pretty.

@worldofpeace worldofpeace force-pushed the worldofpeace:port-gnome3y-tests branch from 189ff39 to e4bf4cd Nov 6, 2019
nixos/tests/installed-tests/gdk-pixbuf.nix Outdated Show resolved Hide resolved
nixos/tests/installed-tests/gnome-photos.nix Outdated Show resolved Hide resolved
nixos/tests/installed-tests/default.nix Outdated Show resolved Hide resolved
nixos/tests/installed-tests/default.nix Show resolved Hide resolved
};

extraTestScript = ''
# dogtail needs accessibility enabled

This comment has been minimized.

Copy link
@jtojnar

jtojnar Nov 6, 2019

Contributor

In the future, we should probably have a module for this. Depends on #54150

nixos/tests/installed-tests/ostree.nix Outdated Show resolved Hide resolved
nixos/tests/installed-tests/default.nix Outdated Show resolved Hide resolved
, testConfig ? {} # Config to inject into machine
, extraTestScript ? "" # Test script snippet to inject before gnome-desktop-testing-runnner begins
, withX11 ? false # Does test need X11?
, testRunnerFlags ? "" # Extra flags to pass to gnome-desktop-testing-runner

This comment has been minimized.

Copy link
@jtojnar

jtojnar Nov 6, 2019

Contributor

Maybe we should make this an array in line with structuredAttrs proposal. I can imagine python test driver to accept list of args like Python’s subprocess module does.

nixos/tests/installed-tests/default.nix Outdated Show resolved Hide resolved
@jtojnar jtojnar mentioned this pull request Nov 6, 2019
@worldofpeace worldofpeace force-pushed the worldofpeace:port-gnome3y-tests branch from e4bf4cd to 6c1bd19 Nov 6, 2019
@worldofpeace worldofpeace force-pushed the worldofpeace:port-gnome3y-tests branch from 6c1bd19 to e237fd2 Nov 6, 2019
@worldofpeace

This comment has been minimized.

Copy link
Member Author

worldofpeace commented Nov 6, 2019

@GrahamcOfBorg test udisks2 fontconfig-default-fonts packagekit

@worldofpeace worldofpeace force-pushed the worldofpeace:port-gnome3y-tests branch from e237fd2 to d7c8ad7 Nov 7, 2019
@worldofpeace worldofpeace merged commit a46c781 into NixOS:master Nov 7, 2019
12 checks passed
12 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
@worldofpeace worldofpeace deleted the worldofpeace:port-gnome3y-tests branch Nov 7, 2019
@worldofpeace worldofpeace mentioned this pull request Nov 7, 2019
3 of 10 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.