Add _ONLYOBSOLETESAME parameter, obsolete only same BUILD jobs #1493
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1493 +/- ##
==========================================
- Coverage 88.6% 88.47% -0.14%
==========================================
Files 106 106
Lines 8075 8078 +3
==========================================
- Hits 7155 7147 -8
- Misses 920 931 +11
Continue to review full report at Codecov.
|
|
I don't think the codecov result is correct. I don't think this actually decreased coverage. |
|
@AdamWill We're back into that day where unexpected coverage changes arise :( |
docs/UsersGuide.asciidoc
Outdated
| @@ -217,6 +217,12 @@ builds but rather deprioritize them up to a configurable limit of priority. | |||
| _DEPRIORITIZE_LIMIT:: The configurable limit of priority up to which jobs | |||
| should be deprioritized. Needs `_DEPRIORITIZEBUILD`. Default 100. | |||
|
|
|||
| _ONLYOBSOLETESAME:: Only obsolete (or deprioritize) jobs for the same BUILD. | |||
There was a problem hiding this comment.
I struggle with the word "SAME". Can we find better names than just "BUILD" and "SAME"? I guess the name "_NOOBSOLETEBUILD" should be adjusted accordingly already. I would be ok to change the old variable as well because it is kind of an "internal" one. Do you have good suggestions for the both variables?
There was a problem hiding this comment.
IAGREETHATTHEWORD is hard to parse
There was a problem hiding this comment.
I agree they're not the best names, but I can't immediately think of anything better :/ gimme half an hour and I'll let it percolate a bit.
There was a problem hiding this comment.
Random idea for names:_NO_JOB_BUILD_OBSOLENCE=1 and _EQUAL_JOB_BUILD_OBSOLENCE=1?
There was a problem hiding this comment.
It's OBSOLESCENCE . But yeah, not bad...
There was a problem hiding this comment.
hm, I opt for just "_NO_OBSOLETE" and "_ONLY_OBSOLETE_SAME_BUILD".
- OBSOLESCENSE is hard to type right apparently :)
- underscore_separation_helps_reading
- I would not mention "job" when it's most likely about a whole lot of them, as in "a whole build"
- "no obsolete build" was ambiguous already because not a "build" was obsoleted but any "build" with matching other product parameters
By default, whenever openQA receives an ISO POST request and schedules a set of jobs for the posted image, any pending jobs with the same DISTRI, VERSION, FLAVOR and ARCH are canceled. This can be too aggressive, so a _NOOBSOLETEBUILD parameter is available which disables this behaviour entirely. However, there may be a need for a happy medium. For instance, in Fedora, it's too aggressive to cancel any job with the same DISTRI, VERSION, FLAVOR and ARCH as a newly-POSTed ISO, especially at present, since we are currently producing two streams of composes for the same release (VERSION), modular and non-modular; we don't want pending jobs for a non-modular compose to be canceled when a modular compose with the same VERSION lands, and vice versa. However, there's also a problem with _NOOBSOLETEBUILD. Sometimes it happens that a build lands, 100+ jobs are created, but we realize something's wrong - we need to tweak something and re-run all the jobs. The normal way to do this is just to re-POST the images from the same build and rely on the obsoletion behaviour to cancel the existing jobs. But if we do this with _NOOBSOLETEBUILD set, the existing jobs for the same build will not be canceled. So this commit adds _ONLY_OBSOLETE_SAME_BUILD, restricting this obsoletion (or, if _DEPRIORITIZEBUILD is used, deprioritization) behaviour to only apply to jobs with the exact same BUILD. So any pending jobs with the same DISTRI, VERSION, FLAVOR, ARCH and BUILD get obsoleted or deprioritized, but no other jobs do. This commit also renames _NOOBSOLETEBUILD to _NO_OBSOLETE, for reasons discussed in the pull request: os-autoinst#1493 (review) For now both names are respected, but the documentation now only refers to _NO_OBSOLETE. In future we could remove handling of _NOOBSOLETEBUILD entirely.
AdamWill
force-pushed
the
only-obsolete-build
branch
from
89fc77e
to
16af146
Compare
|
OK, I liked @okurz 's suggestion best, so I (tried to) implement that but with backward compatibility for |
|
BTW, I chickened out of making the tests cover both names, as it'd involve lots of trying to figure out what the hell the appropriate expected job counts and jobs are for the subsequent tests if I did. Man, this perl test framework stinks compared to py.test. :) |
commit af1fd2e Merge: 31ec90c 16af146 Author: Stephan Kulow <coolo@kde.org> AuthorDate: Fri Nov 17 10:16:31 2017 +0100 Commit: GitHub <noreply@github.com> CommitDate: Fri Nov 17 10:16:31 2017 +0100 Merge pull request #1493 from AdamWill/only-obsolete-build Add _ONLYOBSOLETESAME parameter, obsolete only same BUILD jobs
commit af1fd2e Merge: 31ec90c 16af146 Author: Stephan Kulow <coolo@kde.org> AuthorDate: Fri Nov 17 10:16:31 2017 +0100 Commit: GitHub <noreply@github.com> CommitDate: Fri Nov 17 10:16:31 2017 +0100 Merge pull request #1493 from AdamWill/only-obsolete-build Add _ONLYOBSOLETESAME parameter, obsolete only same BUILD jobs
By default, whenever openQA receives an ISO POST request and schedules a set of jobs for the posted image, any pending jobs with the same DISTRI, VERSION, FLAVOR and ARCH are canceled. This can be too aggressive, so a _NOOBSOLETEBUILD parameter is available which disables this behaviour entirely. However, there may be a need for a happy medium. For instance, in Fedora, it's too aggressive to cancel any job with the same DISTRI, VERSION, FLAVOR and ARCH as a newly-POSTed ISO, especially at present, since we are currently producing two streams of composes for the same release (VERSION), modular and non-modular; we don't want pending jobs for a non-modular compose to be canceled when a modular compose with the same VERSION lands, and vice versa. However, there's also a problem with _NOOBSOLETEBUILD. Sometimes it happens that a build lands, 100+ jobs are created, but we realize something's wrong - we need to tweak something and re-run all the jobs. The normal way to do this is just to re-POST the images from the same build and rely on the obsoletion behaviour to cancel the existing jobs. But if we do this with _NOOBSOLETEBUILD set, the existing jobs for the same build will not be canceled. So this commit adds _ONLY_OBSOLETE_SAME_BUILD, restricting this obsoletion (or, if _DEPRIORITIZEBUILD is used, deprioritization) behaviour to only apply to jobs with the exact same BUILD. So any pending jobs with the same DISTRI, VERSION, FLAVOR, ARCH and BUILD get obsoleted or deprioritized, but no other jobs do. This commit also renames _NOOBSOLETEBUILD to _NO_OBSOLETE, for reasons discussed in the pull request: os-autoinst#1493 (review) For now both names are respected, but the documentation now only refers to _NO_OBSOLETE. In future we could remove handling of _NOOBSOLETEBUILD entirely.
By default, whenever openQA receives an ISO POST request and
schedules a set of jobs for the posted image, any pending jobs
with the same DISTRI, VERSION, FLAVOR and ARCH are canceled.
This can be too aggressive, so a _NOOBSOLETEBUILD parameter is
available which disables this behaviour entirely. However, there
may be a need for a happy medium.
For instance, in Fedora, it's too aggressive to cancel any job
with the same DISTRI, VERSION, FLAVOR and ARCH as a newly-POSTed
ISO, especially at present, since we are currently producing two
streams of composes for the same release (VERSION), modular and
non-modular; we don't want pending jobs for a non-modular
compose to be canceled when a modular compose with the same
VERSION lands, and vice versa. However, there's also a problem
with _NOOBSOLETEBUILD. Sometimes it happens that a build lands,
100+ jobs are created, but we realize something's wrong - we
need to tweak something and re-run all the jobs. The normal way
to do this is just to re-POST the images from the same build and
rely on the obsoletion behaviour to cancel the existing jobs.
But if we do this with _NOOBSOLETEBUILD set, the existing jobs
for the same build will not be canceled.
So this commit adds _ONLYOBSOLETESAME, which restricts this
obsoletion (or, if _DEPRIORITIZEBUILD is used, deprioritization)
behaviour to only apply to jobs with the exact same BUILD. So
any pending jobs with the same DISTRI, VERSION, FLAVOR, ARCH and
BUILD get obsoleted or deprioritized, but no other jobs do.