Implement $ option to randomly choose yamls.

[wusui]

This implements tracker #23208

Signed-off-by: Warren Usui wusui@redhat.com

[wusui]

The previous change randomly picked distro yamls but ran for each distro anyway. The problem was that the build_matrix changes were not sufficient to prevent this. This new code implements a PickRandom class in matrix.py which effectively models the Concat code, but instead on Concatenating all the yamls together, it randomly picks one instead. This seems to have fixed things when running the following test:

./virtualenv/bin/teuthology-suite -vv --ceph-repo https://github.com/ceph/ceph.git --suite-repo https://github.com/ceph/ceph.git --dry-run -c $CEPH_BRANCH -m ovh -s smoke -k distro -p 10000 -e $CEPH_QA_MAIL -t wusui-23208 --suite-branch wip-23208A

wip-2308A is a smoke test suite with $ entries in distro directories. grepping for os_type in the above output shows 27 runs (instead of the 50+ previously) and both ubuntu and Centos are checked.

Yuri -- can you test this please?

[yuriw]

👍 yamls look better, maybe we compare old vs new to see how they are formed?

[yuriw]

@wusui @zmc we probably need to update docs for this change. Where is it?

[wusui]

I'm looking around for docs right now. All I have seen is the convolution description in the ceph doc. I think that there also has to be some documentation for teuthology somewhere

…

On Tue, Mar 20, 2018 at 2:04 PM, Yuri Weinstein ***@***.***> wrote: @wusui <https://github.com/wusui> @zmc <https://github.com/zmc> we probably need to update docs for this change. Where is it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1157 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADeLY9Fdv5qPO-eDi1Tk1kTOKqE7FyD7ks5tgW73gaJpZM4SxKkS> .

[yuriw]

@wusui from @dmick =>
https://github.com/ceph/teuthology has a link, to http://docs.ceph.com/teuthology/docs/

[yuriw]

@wusui @zmc
I tried running smoke with "--distro ubuntu --distro-version 16.04" => http://pulpito.ceph.com/yuriw-2018-03-21_23:38:37-smoke-luminous-distro-basic-ovh/ and expected to see all jobs on ubuntu. We need to clarify correct behavior.

Existing behaviors - it seems as distro dir is included to the jobs yaml's, then command line args won't overwrite it (opposite what I expected, but as is)

[zmc]

This looks good! Could you add some tests to these files where appropriate please?

teuthology/suite/test/test_build_matrix.py
teuthology/suite/test/test_matrix.py

[wusui]

I have added suite tests for this change and pushed to wip-23208

[zmc]

@wusui I don't see your changes

[smithfarm]

Changes should be pushed to wusui-23208 branch (not wip-23208) for them to appear in this PR

[zmc]

test this please
Edit: I want to see the output, and the last job has been deleted

[zmc]

Thanks @wusui!