Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nixos/tests: Leftover tasks towards unified Python integration tests #72828

Closed
tfc opened this issue Nov 5, 2019 · 69 comments
Closed

nixos/tests: Leftover tasks towards unified Python integration tests #72828

tfc opened this issue Nov 5, 2019 · 69 comments
Labels
1.severity: blocker 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: testing Tooling for automated testing of packages and modules
Projects
20.09 Blockers
Milestone
21.05

Comments

@tfc
Copy link
Contributor

tfc commented Nov 5, 2019 •
edited by doronbehar
Loading

Thanks to #71684 being merged recently, we do now have a test driver implementation in Python that allows for writing the imperative part of NixOS integration tests in Python instead of Perl.

Thanks @flokli @edolstra @Lassulus @adisbladis @domenkozar @garbas @jonringer for the support and @blitz @jtraue @marijanp for helping me porting the remaining Perl tests to Python.

In order to finish this project, we are going to perform the following tasks over the next weeks/months:

Additional work that would be awesome:
@FRidh FRidh added this to the 20.03 milestone Nov 5, 2019
@flokli
Copy link
Contributor

flokli commented Nov 5, 2019 •
edited
Loading

some tests updated in #72810, which was merged.
Updated the checkboxes above.

@prusnak
Copy link
Member

prusnak commented Nov 5, 2019

cc @mmahut : jormungandr metabase trezord
cc @1000101: trickster

@flokli flokli mentioned this issue Nov 5, 2019
Merged
10 tasks
@marijanp marijanp mentioned this issue Nov 5, 2019
Merged
10 tasks
@mweinelt mweinelt mentioned this issue Nov 5, 2019
Merged
10 tasks
@tfc tfc mentioned this issue Nov 5, 2019
Merged
10 tasks
@flokli flokli mentioned this issue Nov 5, 2019
Merged
10 tasks
@worldofpeace
Copy link
Contributor

See #72835 (comment).

With getWindowNames and waitForWindow I will begin porting the tests mentioned there.

@worldofpeace worldofpeace mentioned this issue Nov 5, 2019
Merged
10 tasks
@jtojnar
Copy link
Member

jtojnar commented Nov 5, 2019

It would be nice if the driver had type annotations and was checked with mypy.

@jonringer
Copy link
Contributor

would also help with IDE experience, as it will warn you of type mismatches. At least pycharm will ;)

This was referenced Nov 6, 2019
Merged
Merged
This was referenced Nov 6, 2019
Merged
Merged
Merged
@WilliButz WilliButz mentioned this issue Nov 6, 2019
Merged
10 tasks
This was referenced Nov 6, 2019
Merged
Merged
@veprbl veprbl added the 6.topic: testing Tooling for automated testing of packages and modules label Nov 7, 2019
This was referenced Nov 8, 2019
Merged
Merged
flokli added a commit to flokli/nixpkgs that referenced this issue Nov 8, 2019
@flokli
nixos/smokeping: port test to python
85bf23b 
cc NixOS#72828
This was referenced Nov 8, 2019
Merged
Merged
Merged
@minijackson minijackson mentioned this issue Nov 8, 2019
Merged
10 tasks
minijackson added a commit to minijackson/nixpkgs that referenced this issue Nov 8, 2019
@minijackson
nixos/jellyfin: port test to python (NixOS#72828)
4029748
@minijackson minijackson mentioned this issue Nov 8, 2019
Merged
10 tasks
@liff liff mentioned this issue Jul 18, 2020
Merged
10 tasks
@worldofpeace
Copy link
Contributor

I'm thinking it's a good idea to have the perl driver removed on unstable very soon.

@jonringer
Copy link
Contributor

agreed

@liff liff mentioned this issue Aug 7, 2020
Merged
10 tasks
@worldofpeace worldofpeace added this to To Do in 20.09 Blockers via automation Aug 15, 2020
@flokli
Copy link
Contributor

flokli commented Aug 15, 2020

Here is my attempt at porting the os-prober test to python. https://github.com/symphorien/nixpkgs/tree/os-prober-python
When I run it, qemu simply hangs. Does anyone have an idea ?

@symphorien I rebased your branch onto latest master, and got further until this:

machine: must succeed: nixos-rebuild boot >&2
machine # error: The option `virtualisation.qemu.consoles' defined in `/nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/testing/test-instrumentation.nix' does not exist.
machine # (use '--show-trace' to show detailed location information)
machine # building Nix...
machine # error: The option `virtualisation.qemu.consoles' defined in `/nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/testing/test-instrumentation.nix' does not exist.

The machine doesn't hang anymore, and the failure also exists like this in master.

Can you take another look, and maybe open a PR to carry the discussion over there?

This was referenced Aug 15, 2020
Merged
Merged
Closed
@symphorien
Copy link
Member

Not in the coming days.

dasJ added a commit to helsinki-systems/nixpkgs that referenced this issue Aug 21, 2020
@dasJ
nixos/hardened: Port test to Python and fix it
540c033 
Related to NixOS#72828
Replaces and closes NixOS#76708

Looks like `nix ping-store` does not output anything anymore but still
fails when the connection does not work.
@dasJ dasJ mentioned this issue Aug 21, 2020
Merged
10 tasks
@flokli flokli mentioned this issue Aug 23, 2020
Merged
10 tasks
@blitz
Copy link
Contributor

blitz commented Aug 23, 2020 •
edited
Loading

If someone has bandwidth, we could use some eyes on #94858 to remove quite a big piece of perl test code. :)

@ajs124 ajs124 mentioned this issue Aug 24, 2020
Merged
10 tasks
@flokli flokli mentioned this issue Aug 26, 2020
Merged
10 tasks
@flokli
Copy link
Contributor

flokli commented Aug 26, 2020 •
edited
Loading

With all remaining tests ported over, I opened #96396 removing the (now unused) perl code.

Thanks a lot for everyone doing the porting! :-)

@worldofpeace
Copy link
Contributor

Soo proud that we did this ✨

@worldofpeace worldofpeace moved this from To Do to Done in 20.09 Blockers Aug 30, 2020
This was referenced Sep 5, 2020
Merged
Closed
@worldofpeace worldofpeace added this to To do in 20.09 Blockers via automation Oct 5, 2020
@worldofpeace worldofpeace removed this from Done in 20.09 Blockers Oct 5, 2020
@worldofpeace worldofpeace moved this from To do to Done in 20.09 Blockers Oct 5, 2020
@FRidh FRidh modified the milestones: 20.09, 21.03 Dec 20, 2020
@stale
Copy link

stale bot commented Jun 18, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 18, 2021
@ajs124 ajs124 closed this as completed Jun 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
1.severity: blocker 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: testing Tooling for automated testing of packages and modules
Projects
No open projects
20.09 Blockers
  
Done
Milestone
21.05
Development

No branches or pull requests