Make Pulp Smash test MODIFIED issues #54
Conversation
|
@Ichimonji10 Modified means that the change is in our git repos, but that it is not in an RPM build. I think this make the questions maybe a little complicated. Does pulp-smash know if it is testing a source install (like the dev environment) vs. an RPM install? Because a MODIFIED bug will be fixed in a source install and be broken in the RPM install. I know that sounds complicated, but it's our way of tracking WHERE a bug fix can be expected to be found.\ Would it be crazy to make pulp-smash aware of whether it is testing a source install vs. an RPM install, and let it dynamically choose whether MODIFIED or ON_QA is appropriate? If the goal is to only test source installs then I think this change is good. |
|
@rbarlow I think that it is safe to say that pulp-smash will be running on an RPM install that contains the fix. We currently build RPMs nightly, deploy Pulp using those RPMs, run pulp-smash. So if something is in MODIFIED state then it should be in the newly build RPM. |
Nope. You can tell Pulp Smash which version of Pulp it is testing, but you cannot at this time tell it whether it's testing a source or RPM install. |
Gotcha. And then an ON_QA bug will be fixed in both a source and RPM install. |
|
If we wait until the issue is at ON_QA there will be a multi-week, perhaps multi-month delay in tests becoming enabled. We really need them enabled with MODIFIED. I think dkliban's comment suggests that it will work in both RPM based installs and source installs since RPM based installs are rebuilt from source nightly. |
The `bug_is_testable` and `bug_is_untestable` functions talk to the pulp.plan.io
bug tracker, inspect the state of a named bug and then tell the caller whether
the named bug is testable or not. Make these functions treat MODIFIED issues as
test-worthy, instead of waiting until issues enter the ON_QA state. This allows
Pulp Smash to test fixes that have been implemented and are available through
source installs and the nightly RPM builds.
Both before and after this patch:
============ ==================
Pulp Version Test Suite Results
============ ==================
2.6 OK (skipped=13)
2.7 OK (skipped=12)
dev OK (skipped=12)
============ ==================
|
Merged as 8ce9eda |
The
bug_is_testableandbug_is_untestablefunctions talk to the pulp.plan.io bug tracker, inspect the state of a named bug and then tell the caller whether the named bug is testable or not. Make these functions treat MODIFIED issues as test-worthy. Both before and after this patch: