Leap Micro has different build numbers for isos and images #210

jlausuch · 2023-03-26T21:31:56Z

The xml was merged into single one:
#208 The project contains the string ToTest, which is normally used in Factory, but ToTest is also used for Leap Micro, so the script does not allow having different build numbers in the files.

jlausuch · 2023-03-26T21:33:11Z

This is the output before applying this change

geekotest@ariel:/opt/openqa-trigger-from-obs> bash -x script/rsync.sh openSUSE:Leap:Micro:5.4:ToTest
+ set -e
+ environ=openSUSE:Leap:Micro:5.4:ToTest
+++ dirname script/rsync.sh
++ cd script
++ pwd
+ DIR=/opt/openqa-trigger-from-obs/script
+ cd /opt/openqa-trigger-from-obs/script/..
+ '[' -d openSUSE:Leap:Micro:5.4:ToTest ']'
+ '[' -e openSUSE:Leap:Micro:5.4:ToTest/rsync.lock ']'
+ trap 'rm -f openSUSE:Leap:Micro:5.4:ToTest/rsync.lock; exit' INT TERM EXIT
+ echo 13759
+ '[' '!' -f rsync.secret ']'
++ cat rsync.secret
+ export RSYNC_PASSWORD=tZNMx.gSU38NE3EV
+ RSYNC_PASSWORD=tZNMx.gSU38NE3EV
+ failures_count=0
+ for subfolder in $environ/*/
+ '[' -d 'openSUSE:Leap:Micro:5.4:ToTest/*/' ']'
+ subfolder=openSUSE:Leap:Micro:5.4:ToTest
+ set +e
+ set -e
+ '[' -e openSUSE:Leap:Micro:5.4:ToTest/read_files.sh ']'
+ '[' '!' -e openSUSE:Leap:Micro:5.4:ToTest/print_rsync_iso.sh ']'
+ '[' -e openSUSE:Leap:Micro:5.4:ToTest/print_openqa.sh ']'
+ bash -e openSUSE:Leap:Micro:5.4:ToTest/read_files.sh
+ '[' -e openSUSE:Leap:Micro:5.4:ToTest/.run_last ']'
+ [[ openSUSE:Leap:Micro:5.4:ToTest == *ToTest* ]]
+ [[ openSUSE:Leap:Micro:5.4:ToTest != *Factory* ]]
+ [[ openSUSE:Leap:Micro:5.4:ToTest != *MicroOS* ]]
++ grep -h -o -E 'Build[0-9](\.|[0-9]+)*[0-9]+' openSUSE:Leap:Micro:5.4:ToTest/files_iso.lst 'openSUSE:Leap:Micro:5.4:ToTest/files_repo_000product:Leap-Micro-ftp-ftp*.lst' 'openSUSE:Leap:Micro:5.4:ToTest/Media1_000product:Leap-Micro-ftp-ftp*.lst'
+ builds='Build3.23
Build3.23
Build3.23
Build3.23
Build13.2
Build13.2
Build14.2
Build14.2'
+ '[' -n 'Build3.23
Build3.23
Build3.23
Build3.23
Build13.2
Build13.2
Build14.2
Build14.2' ']'
++ echo 'Build3.23
Build3.23
Build3.23
Build3.23
Build13.2
Build13.2
Build14.2
Build14.2'
++ sort
++ uniq
++ wc -l
+ '[' 3 -gt 1 ']'
+ echo 'Conflicting builds found {Build3.23
Build3.23
Build3.23
Build3.23
Build13.2
Build13.2
Build14.2
Build14.2}, exiting because of conflict in {openSUSE:Leap:Micro:5.4:ToTest}'
Conflicting builds found {Build3.23
Build3.23
Build3.23
Build3.23
Build13.2
Build13.2
Build14.2
Build14.2}, exiting because of conflict in {openSUSE:Leap:Micro:5.4:ToTest}
+ exit 0
+ res=0
+ '[' 0 -eq 0 ']'
+ '[' openSUSE:Leap:Micro:5.4:ToTest '!=' openSUSE:Leap:Micro:5.4:ToTest ']'
+ break
+ exit
+ rm -f openSUSE:Leap:Micro:5.4:ToTest/rsync.lock
+ exit

jlausuch · 2023-03-26T21:39:39Z

After trying this, I managed to run script/rsync.sh openSUSE:Leap:Micro:5.4:ToTest with properly triggering the tests in openQA:
https://openqa.opensuse.org/tests/overview?distri=leap-micro&version=5.4&build=13.2_3.23&groupid=101

but the CI complains about

F-obs/openSUSE:Leap:Micro:5.4:ToTest test_repo_consistent.sh:Referenced REPO file wasnt found in print_rsync_repo output {Leap-Micro-5.4-aarch64-Build14.2}   {Leap-Micro-5.4-aarch64.license-Build13.2
Leap-Micro-5.4-aarch64-Build13.2
Leap-Micro-5.4-x86_64.license-Build13.2
Leap-Micro-5.4-x86_64-Build13.2}
make: *** [Makefile:25: test] Error 1

Exited with code exit status 2

andrii-suse · 2023-03-27T07:04:43Z

So CI is correct that scripts generate inconsistent commands: in the sync that has been triggered aarch64 folder is created:
Leap-Micro-5.4-aarch64-Build13.2,
but in openqa command it is referenced as
REPO_0=Leap-Micro-5.4-aarch64-Build3.23

It is possible that the test doesn't need REPO variable and succeeds, but the folder will be deleted by the automatic job (of openQA assets cleanup) on next run. Because it deletes every repo, which is not referenced by any test.

The scripts are not expected to run with different build id for different architectures, at least there was no attempt to run them before. I will have a look what can be done.

jlausuch · 2023-03-27T07:12:44Z

So CI is correct that scripts generate inconsistent commands: in the sync that has been triggered aarch64 folder is created: Leap-Micro-5.4-aarch64-Build13.2, but in openqa command it is referenced as REPO_0=Leap-Micro-5.4-aarch64-Build3.23

It is possible that the test doesn't need REPO variable and succeeds, but the folder will be deleted by the automatic job (of openQA assets cleanup) on next run. Because it deletes every repo, which is not referenced by any test.

The scripts are not expected to run with different build id for different architectures, at least there was no attempt to run them before. I will have a look what can be done.

The build number is the same for different architectures in the same flavor (DVD-x86_64 and DVD-aarch64).
The difference is in DVD vs Images. This is the same case for SLE Micro, where we combine both in the same place, but we don't use REPO_X there as we have proxy scc.

I see that
rsync --timeout=3600 -rtlp4 --delete --specials --link-dest /var/lib/openqa/factory/repo/Leap-Micro-5.4-x86_64.license-CURRENT/ /var/lib/openqa/factory/repo/Leap-Micro-5.4-x86_64.license-CURRENT/ /var/lib/openqa/factory/repo/Leap-Micro-5.4-x86_64.license-Build13.2
is right, but the actual problem I see is that the REPO_LEAP variable points to the wrong build number:
REPO_LEAP_MICRO=Leap-Micro-5.4-aarch64-Build13.2

andrii-suse · 2023-03-27T07:18:29Z

Ah, I see now. The DVD files were just deleted from files_iso.lst in #208 , so CI allowed the run: https://github.com/os-autoinst/openqa-trigger-from-obs/pull/208/files#diff-81261a81443178a1b917136e161d9c0c79834451235b4186d0f91b0d01cfce68L1-L2

I will check if it is simple to provide a patch.

jlausuch · 2023-03-27T07:20:46Z

Ah, I see now. The DVD files were just deleted from files_iso.lst in #208 , so CI allowed the run: https://github.com/os-autoinst/openqa-trigger-from-obs/pull/208/files#diff-81261a81443178a1b917136e161d9c0c79834451235b4186d0f91b0d01cfce68L1-L2

I will check if it is simple to provide a patch.

Thanks!

ALP produces a non-compressed RAW images for the encrypted flavor. Without this change, it's set as ASSET_1 instead of HDD_1: ASSET_1=ALP-Micro.x86_64-0.1-Default-encrypted-Build2.2.raw and the test fails: https://openqa.opensuse.org/tests/3193166

jlausuch · 2023-03-27T12:18:17Z

Thank you for the help, Andrii.

nilxam · 2023-03-31T08:15:53Z

I really suspicious this break totest-manager on Leap Micro, when it query by build number via api/v1/jobs?group=openSUSE%20Leap%20Micro%205.4&build=NUMBER, take an example: when openSUSE:Leap:Micro:5.4 has product number 16.3, it tries to query build number 16.3 on Leap Micro group on o3, however, this is none, becasue the build number is now merged with the image from :Images project, like 13.2_3.27, therefore ttm releaser/publisher just won't work since it can not get the proper job results from o3, there are two ttm were affected, ttm for openSUSE:Leap:Micro:5.4 and openSUSE:Leap:Micro:5.4:Images ... Unless someone fixed it in totest manager, I'd not recommend that we take this change for Leap Micro...

CC @Vogtinator @DimStar77

Vogtinator · 2023-03-31T08:23:06Z

Yes, this approach will not work with ttm.

Vogtinator · 2023-03-31T08:26:44Z

There are three options:

a) Revert this change
b) Merge the openSUSE:Leap:Micro:5.4 and :Images projects on OBS, then use setrelease to have a single release.
c) Add some new complexity into ttm to explicitly look for this version or change the code to behave like rabbit-openqa.py which looks for ISO= matching instead of BUILD= matching.

AFAIK, the purpose of the separate :Images project in OBS is to allow images to get rebuild separately also after final release. That makes merging the project both on OBS and in the syncing invalid and thus a) is the only correct option.

jlausuch · 2023-03-31T08:40:41Z

Ok. I will then revert all the changes and move to the previous state. BUT the implication is that the images will use the repo they defined by default, not any repo from the ISO project.

I would like that for the next version we go for option b) as we do for SLE Micro (images and isos are in the same IBS project, and they get different build numbers.)

nilxam · 2023-03-31T08:48:51Z

maybe we need to revert #208 also?

jlausuch · 2023-03-31T08:50:18Z

maybe we need to revert #208 also?

yes, #213
but first I Wanted to revert the latest change.
and ofc os-autoinst/opensuse-jobgroups#301

nilxam · 2023-03-31T08:52:54Z

Ok. I will then revert all the changes and move to the previous state. BUT the implication is that the images will use the repo they defined by default, not any repo from the ISO project.

I would like that for the next version we go for option b) as we do for SLE Micro (images and isos are in the same IBS project, and they get different build numbers.)

we can revisit this topic in the next Leap Micro, since Leap Micro 5.4 entered Beta stage(close to RC almost) I would appareciate that we can keep the previous working approach :)

jlausuch requested a review from andrii-suse March 26, 2023 21:32

jlausuch force-pushed the leap_micro branch 2 times, most recently from ae88276 to da4b2ab Compare March 26, 2023 21:39

jlausuch force-pushed the leap_micro branch 2 times, most recently from 73a92a4 to 3660dca Compare March 26, 2023 22:27

jlausuch force-pushed the leap_micro branch from 3660dca to d8b6d28 Compare March 27, 2023 08:36

jlausuch changed the title ~~[Draft] Leap Micro has different build numbers for isos and images~~ Leap Micro has different build numbers for isos and images Mar 27, 2023

jlausuch force-pushed the leap_micro branch 2 times, most recently from 3aa5aa4 to d4766cd Compare March 27, 2023 09:06

Use .raw assets as HDD_1

a2d4f47

ALP produces a non-compressed RAW images for the encrypted flavor. Without this change, it's set as ASSET_1 instead of HDD_1: ASSET_1=ALP-Micro.x86_64-0.1-Default-encrypted-Build2.2.raw and the test fails: https://openqa.opensuse.org/tests/3193166

jlausuch force-pushed the leap_micro branch from d4766cd to a2d4f47 Compare March 27, 2023 09:06

xml: fix pwd in VARIABLE depends on cwd

cbe0463

andrii-suse force-pushed the leap_micro branch from 09b7a6e to cbe0463 Compare March 27, 2023 10:49

jlausuch merged commit 5515340 into os-autoinst:master Mar 27, 2023
2 checks passed

This was referenced Mar 31, 2023

Revert "Leap Micro has different build numbers for isos and images" #212

Merged

Revert "Use single job group for Leap Micro" os-autoinst/opensuse-jobgroups#301

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Leap Micro has different build numbers for isos and images #210

Leap Micro has different build numbers for isos and images #210

jlausuch commented Mar 26, 2023

jlausuch commented Mar 26, 2023

jlausuch commented Mar 26, 2023 •

edited

andrii-suse commented Mar 27, 2023 •

edited

jlausuch commented Mar 27, 2023

andrii-suse commented Mar 27, 2023

jlausuch commented Mar 27, 2023

jlausuch commented Mar 27, 2023

nilxam commented Mar 31, 2023 •

edited

Vogtinator commented Mar 31, 2023

Vogtinator commented Mar 31, 2023

jlausuch commented Mar 31, 2023

nilxam commented Mar 31, 2023

jlausuch commented Mar 31, 2023

nilxam commented Mar 31, 2023

Leap Micro has different build numbers for isos and images #210

Leap Micro has different build numbers for isos and images #210

Conversation

jlausuch commented Mar 26, 2023

jlausuch commented Mar 26, 2023

jlausuch commented Mar 26, 2023 • edited

andrii-suse commented Mar 27, 2023 • edited

jlausuch commented Mar 27, 2023

andrii-suse commented Mar 27, 2023

jlausuch commented Mar 27, 2023

jlausuch commented Mar 27, 2023

nilxam commented Mar 31, 2023 • edited

Vogtinator commented Mar 31, 2023

Vogtinator commented Mar 31, 2023

jlausuch commented Mar 31, 2023

nilxam commented Mar 31, 2023

jlausuch commented Mar 31, 2023

nilxam commented Mar 31, 2023

jlausuch commented Mar 26, 2023 •

edited

andrii-suse commented Mar 27, 2023 •

edited

nilxam commented Mar 31, 2023 •

edited