Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve calculating preferred machine #2249

Merged

Conversation

Martchus
Copy link
Contributor

@Martchus Martchus commented Aug 8, 2019

* Make result deterministic in case multiple machines
  occur equally often (see https://progress.opensuse.org/issues/16354)
* Improve comments
@codecov
Copy link

codecov bot commented Aug 8, 2019

Codecov Report

Merging #2249 into master will decrease coverage by 0.07%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2249      +/-   ##
==========================================
- Coverage    86.1%   86.03%   -0.08%     
==========================================
  Files         166      166              
  Lines       10825    10826       +1     
==========================================
- Hits         9321     9314       -7     
- Misses       1504     1512       +8
Impacted Files Coverage Δ
lib/OpenQA/WebAPI/Controller/Test.pm 62.64% <100%> (+0.11%) ⬆️
lib/OpenQA/Worker/Settings.pm 84.9% <0%> (-13.21%) ⬇️
lib/OpenQA/Worker/Engines/isotovideo.pm 93.68% <0%> (-0.98%) ⬇️
lib/OpenQA/WebSockets.pm 94.11% <0%> (+1.47%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 695ec80...c3c0b42. Read the comment docs.

2 similar comments
@codecov
Copy link

codecov bot commented Aug 8, 2019

Codecov Report

Merging #2249 into master will decrease coverage by 0.07%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2249      +/-   ##
==========================================
- Coverage    86.1%   86.03%   -0.08%     
==========================================
  Files         166      166              
  Lines       10825    10826       +1     
==========================================
- Hits         9321     9314       -7     
- Misses       1504     1512       +8
Impacted Files Coverage Δ
lib/OpenQA/WebAPI/Controller/Test.pm 62.64% <100%> (+0.11%) ⬆️
lib/OpenQA/Worker/Settings.pm 84.9% <0%> (-13.21%) ⬇️
lib/OpenQA/Worker/Engines/isotovideo.pm 93.68% <0%> (-0.98%) ⬇️
lib/OpenQA/WebSockets.pm 94.11% <0%> (+1.47%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 695ec80...c3c0b42. Read the comment docs.

@codecov
Copy link

codecov bot commented Aug 8, 2019

Codecov Report

Merging #2249 into master will decrease coverage by 0.07%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2249      +/-   ##
==========================================
- Coverage    86.1%   86.03%   -0.08%     
==========================================
  Files         166      166              
  Lines       10825    10826       +1     
==========================================
- Hits         9321     9314       -7     
- Misses       1504     1512       +8
Impacted Files Coverage Δ
lib/OpenQA/WebAPI/Controller/Test.pm 62.64% <100%> (+0.11%) ⬆️
lib/OpenQA/Worker/Settings.pm 84.9% <0%> (-13.21%) ⬇️
lib/OpenQA/Worker/Engines/isotovideo.pm 93.68% <0%> (-0.98%) ⬇️
lib/OpenQA/WebSockets.pm 94.11% <0%> (+1.47%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 695ec80...c3c0b42. Read the comment docs.

@@ -393,24 +393,25 @@ sub job_next_previous_ajax {

sub _calculate_preferred_machines {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a unit test for this somewhere? This looks to be completed untested? At least in the unit tests I can't find it...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be covered by tests somehow at least implicitly because codecov reports for the diff to be completely covered :)

Copy link
Member

@kalikiana kalikiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty nice. Also thanks for making the code a little more readable! Note my comment inline.

Copy link
Member

@okurz okurz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

giving my approval to allow @Martchus to merge as-is unless he wants to follow the suggestion to check unit-test coverage which I would also prefer

@Martchus
Copy link
Contributor Author

Martchus commented Aug 9, 2019

This is covered and only checks are missing. I don't want to add checks for this behavior which seems arbitrarily too me. With this PR it at least always takes the same arbitrary choice all the time. It still seems weird to allow the most popular machine to be omitted and then merge the rows. So I honestly don't understand what the purpose of this "preferred" machine thing is. To me it would actually look best and most space-efficient if all architectures for one test would be in a single row and the machine wouldn't be mentioned at all on this level or maybe only in a tooltip.

@Martchus
Copy link
Contributor Author

Martchus commented Aug 9, 2019

Ok, in this overview it actually make sense: https://openqa.suse.de/tests/overview?result=failed&arch=&modules=&distri=sle&version=12-SP5&build=0216&groupid=240 (link from the duplicated ticked https://progress.opensuse.org/issues/54008)

Here the machine differs between jobs of the same architecture. So the two x86_64 jobs for xfstests_btrfs-generic-401-999 and xfstests_btrfs-generic-401-999@64bit-smp could not be shown at the same time without the "preferred machine" behavior. Maybe I can try to add such a case to the fixtures. Not sure whether it is worth, though.

@kalikiana
Copy link
Member

Here the machine differs between jobs of the same architecture. So the two x86_64 jobs for xfstests_btrfs-generic-401-999 and xfstests_btrfs-generic-401-999@64bit-smp could not be shown at the same time without the "preferred machine" behavior. Maybe I can try to add such a case to the fixtures. Not sure whether it is worth, though.

Since people ran into this and reported it, I think that makes it worth testing that it works as expected.

However I second re-evaluating if this is the behavior we want. It reminds me of the original generated YAML which was too smart and ended up being confusing even when I made it deterministic.

@Martchus
Copy link
Contributor Author

Martchus commented Aug 9, 2019

I personally would prefer to merge this and think of checks later.

@okurz
Copy link
Member

okurz commented Aug 11, 2019

IMHO making the current behaviour deterministic is highest priority. Let's discuss general alternatives to "preferred machine" not in this PR, ok?

@kalikiana kalikiana dismissed their stale review August 12, 2019 09:05

I don't see the urgency here... I'll abstain

@okurz okurz merged commit 117b28d into os-autoinst:master Aug 16, 2019
@Martchus Martchus deleted the improve-calculating-preferred-machines branch August 19, 2019 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants