Enable running install_ltp and ltp_run in the same job #8329

mdoucha · 2019-08-30T11:29:29Z

This is a convenience patch for LTP testcase debugging. When both INSTALL_LTP and LTP_COMMAND_FILE are specified in the same job, run the testcase immediately after install_ltp and VM reboot.

Currently, running an LTP testcase from fresh sources requires two separate jobs: One to install LTP from Git and publish the HDD image, second to actually run the testcase. This is perfectly fine for regression tests but it's too cumbersome for quick one-off debug runs on special hardware.

Related ticket: N/A
Needles: N/A
Verification run: https://openqa.suse.de/tests/3439742 (install_ltp + ltp_math)

pevik · 2019-08-30T11:38:27Z

@mdoucha thanks for handling this. FYI, my unfinished PR #7812

pevik · 2019-08-30T11:40:46Z

I haven't test it my PR well, but expect chicken & egg problems as in main_ltp.pm is needed to have runtest files, which are only after installation. If it works (don't have this problem, it'd be better solution for IPMI). It'd be good to check if /opt/ltp exist and install LTP only when it does not.

pevik · 2019-08-30T11:41:22Z

FYI: https://progress.opensuse.org/issues/53948

mdoucha · 2019-08-30T12:17:28Z

I haven't test it my PR well, but expect chicken & egg problems as in main_ltp.pm is needed to have runtest files, which are only after installation. If it works (don't have this problem, it'd be better solution for IPMI). It'd be good to check if /opt/ltp exist and install LTP only when it does not.

Hmm, I didn't notice that loadtest_from_runtest_file depends on extra files uploaded by install_ltp (I did a few debug runs the hard way so the runtest file was already uploaded when I started writing this patch). I'll have to dig deeper and try to delay the runtest file loading until after install_ltp finishes (if possible).

FYI: https://progress.opensuse.org/issues/53948

Unfortunately, I don't have access to this issue (error 403).

mdoucha · 2019-09-03T09:46:06Z

I've solved the chicken & egg problem by scheduling testcases at the end of install_ltp if both INSTALL_LTP and LTP_COMMAND_FILE are set. But this feature now depends on os-autoinst/openQA#2302

If you run install and tests in the same job on OpenQA without the above PR applied, the testcases will execute just fine but the results will not show up in the web UI.

pevik · 2019-09-04T13:06:55Z

@mdoucha nice :).
verification run: http://quasar.suse.cz/tests/3635

And yes, ajax or page redirection would be great when autotest::write_test_order(); returns something.
@Martchus is there any way to do it?

pevik · 2019-09-04T13:08:12Z

@mdoucha please keep [WIP] or something in the subject so nobody merges it before openQA functionality in os-autoinst/openQA#2302 is merged and deployed on both osd and o3.

pevik

Elegant solution, it just need to wait for os-autoinst/openQA#2302 deployment.

pevik · 2019-09-04T13:09:12Z

@richiejp FYI

mdoucha · 2019-09-05T11:56:05Z

One more dependency: os-autoinst/os-autoinst#1208

autotest::loadtest() will now call write_test_order() for you.

mdoucha · 2019-09-26T14:24:04Z

There were some conflicting changes in install_ltp so I've rebased to latest master commit, fixed a bug where loadtest_from_runtest_file() was looking for freshly uploaded runtest files in the wrong directory and consolidated all fixes into a single commit.

pevik · 2019-09-27T15:33:30Z

OK, we're waiting for os-autoinst/openQA#2302 and os-autoinst/os-autoinst#1208 to be released and deployed :).

mdoucha · 2019-10-02T10:24:50Z

Dependencies have been deployed, this PR is ready to be merged.

baierjan · 2019-10-03T08:17:11Z

Verification runs maybe?

pevik · 2019-10-03T08:29:39Z

I did one back when the PR was created, but agree a new one wouldn't harm.

mdoucha · 2019-10-03T08:35:03Z

Verification runs maybe?

Now that the dependencies have been deployed, I can finally create one: https://openqa.suse.de/tests/3435241 (install_ltp + ltp_math)

baierjan · 2019-10-03T10:23:47Z

Verification runs maybe?

Now that the dependencies have been deployed, I can finally create one: https://openqa.suse.de/tests/3435241 (install_ltp + ltp_math)

I see only install_ltp, is that correct?

mdoucha · 2019-10-03T10:36:29Z

I see only install_ltp, is that correct?

No, that's not correct. It appears that os-autoinst wasn't deployed together with openQA...

okurz · 2019-10-03T15:45:53Z

that last statement is incorrect. As can be seen from https://openqa.suse.de/tests/3435241/file/autoinst-log.txt the version of os-autoinst running is "4.5.1569914332.5870aebd" corresponding to https://github.com/os-autoinst/os-autoinst/commits/5870aebd which includes os-autoinst/os-autoinst#1208 so the only logical explanation I have at this time is that the feature does not work as expected due to other reasons

mdoucha · 2019-10-04T08:33:50Z

that last statement is incorrect.

*Facepalm* You're right. Now I know where the problem is. I've created the verification run from an install job and didn't remove the PUBLISH_HDD_1/PUBLISH_PFLASH_VARS variables. These somehow block the usual result upload loop if the test is too short (which it is here) and OpenQA won't notice that test_order.json has changed.

This verification run will work: https://openqa.suse.de/tests/3439736

mdoucha · 2019-10-04T08:54:07Z

This verification run will work: https://openqa.suse.de/tests/3439736

...or at least it would have worked if I set the right boot image... One more try: https://openqa.suse.de/tests/3439742

okurz · 2019-10-05T19:39:04Z

I've created the verification run from an install job and didn't remove the PUBLISH_HDD_1/PUBLISH_PFLASH_VARS variables. These somehow block the usual result upload loop if the test is too short (which it is here) and OpenQA won't notice that test_order.json has changed.

This sounds like actually the known issue https://progress.opensuse.org/issues/39845 and this is being worked on.

pevik · 2019-10-30T13:07:11Z

@mdoucha @okurz @cfconrad @perlpunk Dependency os-autoinst/openQA#2327 (poo#39845) has been deployed. Although it does not display during running test (poo#58826), we might want to merge it before. Or do we want to wait till poo#39845 is solved?

Anyway, verification runs for some LTP jobs (cloned jobs with added INSTALL_LTP=repo):
http://quasar.suse.cz/tests/3770
http://quasar.suse.cz/tests/3769
http://quasar.suse.cz/tests/3771

mdoucha · 2019-10-30T13:35:29Z

@mdoucha @okurz @cfconrad @perlpunk Dependency os-autoinst/openQA#2327 (poo#39845) has been deployed. Although it does not display during running test (poo#58826), we might want to merge it before. Or do we want to wait till poo#39845 is solved?

The real problem is that the dependencies are not deployed. Period. I can see the expected log entries only in jobs executed on openqaworker2, openqaworker10, all ppc64le workers and all aarch64 workers.

I've started 2 more verification runs.
This job will finish with LTP tests missing from schedule: https://openqa.suse.de/tests/3541697
This job will have complete schedule because it's forced to run on openqaworker2: https://openqa.suse.de/tests/3541705

pevik · 2019-10-30T13:46:20Z

Good point, let's wait.

mdoucha · 2019-10-30T14:07:50Z

And the results are in - exactly as predicted in my previous comment. So my statement from almost a month ago is correct after all:

No, that's not correct. It appears that os-autoinst wasn't deployed together with openQA...

os-autoinst on openqaworker6 may be up to date but it's apparently talking to OpenQA API which is more than a month old. That's why the test schedule doesn't get updated - because the code that is supposed to reload test_order.json and write the new test schedule into database still hasn't been deployed there.

okurz · 2019-10-31T07:37:07Z

So what exactly are you waiting for? Or who should do what? As a side-note: you can check the version of os-autoinst in the log file of every job and the version of the openQA webui in the webui itself, including the changelog

pevik · 2019-10-31T07:40:33Z

@okurz IMHO os-autoinst being installed on workers (and maybe openQA as well).

pevik · 2019-10-31T07:42:04Z

As a side-note: you can check the version of os-autoinst in the log file of every job and the version of the openQA webui in the webui itself, including the changelog

I know that, but it'd be nice to have internal site showing os-autoinst, openQA* package versions of all workers. That'd help a lot to see the actual deployment.

mdoucha · 2019-10-31T08:17:44Z

So what exactly are you waiting for? Or who should do what? As a side-note: you can check the version of os-autoinst in the log file of every job and the version of the openQA webui in the webui itself, including the changelog

The OpenQA webUI version is irrelevant because the worker is not talking to it in the first place. The os-autoinst process on the worker is talking to OpenQA API on the worker.

mdoucha · 2019-10-31T08:38:14Z

So what exactly are you waiting for? Or who should do what?

To answer your question, you need to update the openQA-common package on the following workers:

openqaw1
openqaw2 (if you plan to bring openqaw1 & 2 back online sometime in the future)
openqaworker3
openqaworker5
openqaworker6
openqaworker7
openqaworker8
openqaworker9
openqaworker13
grenache-1

Then we can merge this.

okurz · 2019-10-31T09:43:42Z

@okurz IMHO os-autoinst being installed on workers (and maybe openQA as well).

@pevik yes, but this is done already. We have a recent version installed – unless you can show me a machine where we really missed this.

As a side-note: you can check the version of os-autoinst in the log file of every job and the version of the openQA webui in the webui itself, including the changelog

I know that, but it'd be nice to have internal site showing os-autoinst, openQA* package versions of all workers. That'd help a lot to see the actual deployment.

Well, on top of the log files that show the versions you can also check https://openqa.opensuse.org/admin/workers (or correspondingly for every other server) that shows the "os-autoinst API version" and "Websocket Api version" which should be the relevant numbers. It can happen that we miss to update these numbers in pull requests but everyone can propose this with simple code change. For o3 there is an automatic nightly deployment so you can assume that by default the current git master is also running on the o3 webUI and workers.

So what exactly are you waiting for? Or who should do what?

To answer your question, you need to update the openQA-common package on the following workers:
* openqaw1
[…]

Then we can merge this.

Let's discuss these OSD specifics internally.

okurz · 2019-10-31T15:25:57Z

I have merged now after all services have been restarted to ensure the new code is used.

mdoucha changed the title ~~Enable running ltp_install and ltp_run in the same job~~ [WIP] Enable running ltp_install and ltp_run in the same job Aug 30, 2019

mdoucha changed the title ~~[WIP] Enable running ltp_install and ltp_run in the same job~~ [WIP] Enable running install_ltp and ltp_run in the same job Aug 30, 2019

mdoucha mentioned this pull request Sep 3, 2019

Allow scheduling additional test modules at job runtime os-autoinst/openQA#2302

Merged

mdoucha changed the title ~~[WIP] Enable running install_ltp and ltp_run in the same job~~ Enable running install_ltp and ltp_run in the same job Sep 3, 2019

pevik approved these changes Sep 4, 2019

View reviewed changes

mdoucha changed the title ~~Enable running install_ltp and ltp_run in the same job~~ [WIP] Enable running install_ltp and ltp_run in the same job Sep 4, 2019

pevik mentioned this pull request Sep 4, 2019

[WIP] LTP fallback installation #7812

Closed

pevik mentioned this pull request Sep 23, 2019

LTP install runtime dependencies only for git instalation #8028

Merged

mdoucha added 2 commits September 26, 2019 16:09

Enable running ltp_install and ltp_run in the same job

02dba89

Document dynamic test module scheduling

0d42a54

mdoucha force-pushed the ltp_quickrun branch from d824ff2 to 0d42a54 Compare September 26, 2019 14:16

pevik mentioned this pull request Sep 27, 2019

Update test_order.json if test schedule changes at runtime os-autoinst/os-autoinst#1208

Merged

mdoucha changed the title ~~[WIP] Enable running install_ltp and ltp_run in the same job~~ Enable running install_ltp and ltp_run in the same job Oct 2, 2019

okurz approved these changes Oct 3, 2019

View reviewed changes

baierjan added the notready label Oct 3, 2019

okurz merged commit dbab0b1 into os-autoinst:master Oct 31, 2019

mdoucha deleted the ltp_quickrun branch October 31, 2019 15:26

baierjan removed the notready label Nov 1, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable running install_ltp and ltp_run in the same job #8329

Enable running install_ltp and ltp_run in the same job #8329

mdoucha commented Aug 30, 2019 •

edited

Loading

pevik commented Aug 30, 2019

pevik commented Aug 30, 2019

pevik commented Aug 30, 2019

mdoucha commented Aug 30, 2019 •

edited

Loading

mdoucha commented Sep 3, 2019

pevik commented Sep 4, 2019

pevik commented Sep 4, 2019

pevik left a comment

pevik commented Sep 4, 2019

mdoucha commented Sep 5, 2019

mdoucha commented Sep 26, 2019

pevik commented Sep 27, 2019

mdoucha commented Oct 2, 2019

baierjan commented Oct 3, 2019

pevik commented Oct 3, 2019

mdoucha commented Oct 3, 2019

baierjan commented Oct 3, 2019

mdoucha commented Oct 3, 2019

okurz commented Oct 3, 2019

mdoucha commented Oct 4, 2019

mdoucha commented Oct 4, 2019

okurz commented Oct 5, 2019

pevik commented Oct 30, 2019

mdoucha commented Oct 30, 2019

pevik commented Oct 30, 2019

mdoucha commented Oct 30, 2019 •

edited

Loading

okurz commented Oct 31, 2019

pevik commented Oct 31, 2019

pevik commented Oct 31, 2019

mdoucha commented Oct 31, 2019

mdoucha commented Oct 31, 2019 •

edited

Loading

okurz commented Oct 31, 2019

okurz commented Oct 31, 2019

Enable running install_ltp and ltp_run in the same job #8329

Enable running install_ltp and ltp_run in the same job #8329

Conversation

mdoucha commented Aug 30, 2019 • edited Loading

pevik commented Aug 30, 2019

pevik commented Aug 30, 2019

pevik commented Aug 30, 2019

mdoucha commented Aug 30, 2019 • edited Loading

mdoucha commented Sep 3, 2019

pevik commented Sep 4, 2019

pevik commented Sep 4, 2019

pevik left a comment

Choose a reason for hiding this comment

pevik commented Sep 4, 2019

mdoucha commented Sep 5, 2019

mdoucha commented Sep 26, 2019

pevik commented Sep 27, 2019

mdoucha commented Oct 2, 2019

baierjan commented Oct 3, 2019

pevik commented Oct 3, 2019

mdoucha commented Oct 3, 2019

baierjan commented Oct 3, 2019

mdoucha commented Oct 3, 2019

okurz commented Oct 3, 2019

mdoucha commented Oct 4, 2019

mdoucha commented Oct 4, 2019

okurz commented Oct 5, 2019

pevik commented Oct 30, 2019

mdoucha commented Oct 30, 2019

pevik commented Oct 30, 2019

mdoucha commented Oct 30, 2019 • edited Loading

okurz commented Oct 31, 2019

pevik commented Oct 31, 2019

pevik commented Oct 31, 2019

mdoucha commented Oct 31, 2019

mdoucha commented Oct 31, 2019 • edited Loading

okurz commented Oct 31, 2019

okurz commented Oct 31, 2019

mdoucha commented Aug 30, 2019 •

edited

Loading

mdoucha commented Aug 30, 2019 •

edited

Loading

mdoucha commented Oct 30, 2019 •

edited

Loading

mdoucha commented Oct 31, 2019 •

edited

Loading