Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent useless downloads of uefi-vars assets #4261

Merged
merged 5 commits into from
Oct 6, 2021

Conversation

Martchus
Copy link
Contributor

@Martchus Martchus commented Oct 4, 2021

lib/OpenQA/Script/CloneJob.pm Outdated Show resolved Hide resolved
lib/OpenQA/Script/CloneJob.pm Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Oct 4, 2021

Codecov Report

Merging #4261 (a5bc31b) into master (5beafc3) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4261      +/-   ##
==========================================
- Coverage   97.93%   97.92%   -0.02%     
==========================================
  Files         369      369              
  Lines       33441    33475      +34     
==========================================
+ Hits        32752    32781      +29     
- Misses        689      694       +5     
Impacted Files Coverage Δ
lib/OpenQA/Script/CloneJob.pm 96.10% <100.00%> (+0.45%) ⬆️
t/35-script_clone_job.t 100.00% <100.00%> (ø)
t/lib/OpenQA/Test/FullstackUtils.pm 88.00% <0.00%> (-5.00%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5beafc3...a5bc31b. Read the comment docs.

@okurz
Copy link
Member

okurz commented Oct 4, 2021

Mind the decreased coverage in t/35-script_clone_job.t which should have 100% statement coverage as it's a test

* Test skipping asset downloads if the (grand) parent is cloned anyways
* Test skipping download of uefi-vars assets
* Test doing asset downloads if skipping download of dependencies
* Test doing download of uefi-vars assets when a parent job is expected
  to generate one (has the `PUBLISH_PFLASH_VARS` setting)
* Skip the download similar to `PUBLISH_HDD_1` assets when the generating
  job is cloned as well anyways.
* Skip the download if none of the parent jobs is actually supposed to
  publish the asset (corresponding `PUBLISH_PFLASH_VARS` is absent)
  because then the asset is not needed anyways and likely does not exist
  anyways.
* See https://progress.opensuse.org/issues/99672
  and https://progress.opensuse.org/issues/98388
So far transitive chained parents are not considered so HDDs would not be
skipped as expected if they would be generated by e.g. a "grand parent"
job.
to prevent the function from becoming too big.
Comment on lines +108 to +113
for my $j (@$parents, $job) {
for my $setting (qw(PUBLISH_HDD_1 PUBLISH_PFLASH_VARS)) {
return 1 if _job_setting_is $j, $setting => $file;
}
}
return 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to combine grep+map here, something like the following maybe?

Suggested change
for my $j (@$parents, $job) {
for my $setting (qw(PUBLISH_HDD_1 PUBLISH_PFLASH_VARS)) {
return 1 if _job_setting_is $j, $setting => $file;
}
}
return 0;
grep { _job_setting_is $_, $setting => $file } …

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the early return makes sense and the for loops are quite easy to read (compared to squashing too much logic into one line).

# skip downloading "uefi-vars" assets if not actually generated by
# any parent
if ($file =~ qr/uefi-vars/) {
my $parent_publishes_pflash_vars;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why declare the variable here and not immediately assign? If this is just to prevent line-break then maybe you need to extract more functions to reduce indentation further :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Perl a variable is initialized to be undef by default. This is wanted here - although any falsy value would do. So I could assign 0. (Note that my current version is according to the coding style we generally use. We definitely don't generally use my $variable = undef.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand. To me it looks like you can combine this with the next line as you unconditionally assign a value there, don't you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next line is a for loop and the lifetime of the variable is outside of that loop. Even if I could get away with it in Perl this seems conceptionally wrong to me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oooh, it's a for-loop. I misread the and last for @$parents; :D Awesome!

# skip downloading "uefi-vars" assets if not actually generated by
# any parent
if ($file =~ qr/uefi-vars/) {
my $parent_publishes_pflash_vars;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand. To me it looks like you can combine this with the next line as you unconditionally assign a value there, don't you?

# skip downloading "uefi-vars" assets if not actually generated by
# any parent
if ($file =~ qr/uefi-vars/) {
my $parent_publishes_pflash_vars;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oooh, it's a for-loop. I misread the and last for @$parents; :D Awesome!

@mergify mergify bot merged commit 31df415 into os-autoinst:master Oct 6, 2021
@Martchus Martchus deleted the clone-pflash-vars branch October 6, 2021 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants