Multinode #152
Multinode #152
Conversation
As lava supports multinode jobs, we are adding this patch to parse lava multinode jobs and stores its data in the data base. The multinode data (key/value) are stored into the test_group collection. Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
When it's a multinode job, there is two callbacks with the same metadata so one will overwrite the other if they have the same names, so we are adding this patch to check if it's a multinode job or not and decide what name should be given. Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
| @@ -90,6 +90,7 @@ def __init__(self, name, lab_name): | |||
| self.time = -1 | |||
| self.vcs_commit = None | |||
| self.warnings = 0 | |||
| self.lava_multinode = {} | |||
There was a problem hiding this comment.
The test groups are LAVA agnostic, so leaking LAVA multinode things here is not going to work.
There was a problem hiding this comment.
I didn't get what you mean, I tested with a multinode job and a none multinode job and there is no error.
There was a problem hiding this comment.
I mean that this data model should not assume anything about LAVA. So if you need to add some extra data to each test group or test case, you can do that but don't make it follow the LAVA multinode design. It needs to be something that works on its own in a self-contained way.
There was a problem hiding this comment.
So what we need is an abstraction that's not LAVA-centric to describe a group of jobs that are all connected/related.
It appears that the LAVA multinode data has a key called target_group which is a unique ID (generated by LAVA) that is shared across all sub-jobs of a multi-node job.
For starters, this would be the minimum thing needed to know that a group of (sub)jobs were all part of the same main (multinode) job.
|
@touilkhouloud Can you please explain how the multinode LAVA data is meant to be used in KernelCI? |
|
The idea came when we noticed that a multinode job has 2 callbacks that shares the same metadata and the json/logs files will have the same name so one will overwrite the otherone so we need some data that differs between the 2 callbacks which is in our case the "role" data for ex one is the host and the other is the target, with this data, @khilman suggested that we may store other multinode data as the group size and others. |
|
Is there any plan to get multinode job results sent to kernelci.org? I would be interested to see what kind of data would be added to the test results as that is what would be driving the changes in the database models. |
|
We still working on creating templates for multinode job, the use of a multinode job is so big and it differs, in our case we are using a host and a target but you can use more then 2 divices per job and it will create callbacks as many as the number of the devices. |
@gctucker yes, the goal is to contribute test results that require multinode to kernelCI. For starters, the tests we're working on are power measurements during boot and also during specific tests.
The kind of data needed in the model is primarily a way to find all the (sub)jobs that were part of a single multinode job. That could simply be a unique ID (LAVA seems to provide this already in the callback data for multi-node jobs as the In addition to some unique ID for all related jobs, I think we'll also need some arbitrary string for human-readable display of the related jobs. In LAVA multinode terms, we would use the "role" name for that (e.g. "client" or "server" in a network test) or ("measurement host" and "DUT") for a power measurement test. For starters, those are the minimum requirements that would be needed in the data model. |
Support lava multinode jobs.