[WIP] Rancher UI selenium test #11863

ilausuch · 2021-01-28T12:44:12Z

This test checks some initial pages of the rancher UI.
This differs from #11807 because uses Selenium instead of traditional openQA click commands. This difference allows to check the individual pages using the DOM instead of using needles. There are several benefits in this way:

Not adding more needles to the needles repository in this case is not necessary.
In case some page text changes, for instance in the main page, where a "What's new" dialog appears, we don't need to redo a new needle every time this dialog changes.
Using DOM we could accelerate the test, because we can check if one element is visible or not, preventing long sleeps.

Test: https://openqa.opensuse.org/tests/1607472#

See: https://progress.opensuse.org/issues/88183

tests/rancher/rancher_ui_selenium.pm

pdostal · 2021-01-29T15:03:33Z

Do you Ivan think that it would be good approach to create a cluster using Selenium?

ilausuch · 2021-02-04T06:28:40Z

Do you Ivan think that it would be good approach to create a cluster using Selenium?

Yes of course (but with #11807 too). The big question here, is what cloud provider we are going to use, or we are going to use bare-metal.

I think the creation of this extended test would exceed the objective of this PR.

pdostal · 2021-02-04T07:33:12Z

Do you Ivan think that it would be good approach to create a cluster using Selenium?

Yes of course (but with #11807 too). The big question here, is what cloud provider we are going to use, or we are going to use bare-metal.

I think the creation of this extended test would exceed the objective of this PR.

I plan to do this over the weekend and I will use multi-machine tests.
Each running test is one QEMU VM so no provider is needed.

ge0r

Perfect! Nice kick start for selenium testing Ivan!

tests/rancher/rancher_ui_selenium.pm

pdostal

Good! I have just one suggestion :)

data/rancher/ui_test.py

ilausuch · 2021-02-11T14:09:01Z

Last test https://openqa.opensuse.org/tests/1627261#step/rancher_ui_selenium/15 (failed)

okurz · 2021-02-11T15:26:44Z

data/rancher/ui_test.py

+  driver.find_element_by_xpath("//button[@type='submit']").click()
+
+  # Initial page: set the URL
+  time.sleep(2)


You are using sleep() or are simulating a sleep-like functionality. This is introducing the risk of either unnecessarily wasting time or just shifting race conditions and making them even harder to investigate. You can use sleep but there are only very limited cases where is the best solution (e.g. yield threads in low-level system code). Here you have the classical problem in software design of Synchronization . sleep should not be used in place of proper synchronization methods. https://blogs.msmvps.com/peterritchie/2007/04/26/thread-sleep-is-a-sign-of-a-poorly-designed-program/ is one of multiple references that explains this: "The original problem is likely a timing/synchronization issue, ignoring it by hiding it with Thread.Sleep is only going to delay the problem and make it occur in random, hard to reproduce ways." And if the problem does not occur at least time is wasted.

We are waiting because some condition changes some time ... keyword(s) is/are some time. It is important to know the exact condition that the code should wait for and check for that instead of delaying program execution for an arbitrary amount of time. For openQA tests in general just try to do what you as a human user would also do. You would not look on your watch waiting for exactly 100s before you look at the screen again, right? :)

Same applies for other uses of sleep

See https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/CONTRIBUTING.md#coding-style for more details

okurz · 2021-02-11T15:27:59Z

tests/rancher/rancher_ui_selenium.pm

+    x11_start_program('xterm');
+
+    type_string "python3 ui_test.py && touch /tmp/ok\n";
+    assert_script_run("[ -f /tmp/ok ]");


Instead of [...] better use

Suggested change

assert_script_run("[ -f /tmp/ok ]");

assert_script_run('test -f /tmp/ok');

okurz · 2021-02-11T15:28:35Z

tests/rancher/rancher_ui_selenium.pm

+    my ($self) = @_;
+
+    my ($self) = @_;


okurz · 2021-02-11T15:29:00Z

tests/rancher/rancher_ui_selenium.pm

+    $self->select_serial_terminal;
+
+    assert_script_run("pip3 install selenium");
+    assert_script_run('curl -O ' . data_url("rancher/ui_test.py"), timeout => 300);    


please delete the w/s at EOL

kalikiana · 2021-02-15T15:08:25Z

Closing this as per discussions since this is just a proof of concept at this point.

pdostal · 2021-02-15T15:21:19Z

Closing this as per discussions since this is just a proof of concept at this point.

I'm sorry but I don't understand. @kalikiana did you speak with @ilausuch about this?

kalikiana · 2021-02-15T16:54:45Z

Closing this as per discussions since this is just a proof of concept at this point.

I'm sorry but I don't understand. @kalikiana did you speak with @ilausuch about this?

Yes. See also poo#88183#note-10. I assume a follow-up conversation will be organized soon to answer questions like where tests live, if Python-Selenium is the right choice etc (afair Perl was also suggested before) and then whatever follows from that will be tested in openQA.

ilausuch · 2021-02-17T11:40:41Z

This is a proof of concept to check if we can use selenium to check the rancher UI and other people is involved and interested in that.

So, I understand that we could close the progress progress ticket because of the implications @kalikiana mentioned. But I think is not a good idea to close the PR because it is still in progress.

For this reason I have to reopen this.

okurz · 2021-02-17T12:33:23Z

So, I understand that we could close the progress progress ticket because of the implications @kalikiana mentioned. But I think is not a good idea to close the PR because it is still in progress.

For this reason I have to reopen this.

There is no point in keeping the PR open for a closed ticket. If you want this PR to be merged then there should be an open ticket covering that. On the other side, a closed PR will still stay as a reference that can be used.

ilausuch · 2021-03-16T11:07:23Z

Closing this, is covered on #11807

ilausuch force-pushed the rancher_ui_test branch 3 times, most recently from de2cf13 to b218735 Compare January 28, 2021 13:07

ilausuch requested a review from ge0r January 28, 2021 15:16

okurz reviewed Jan 28, 2021

View reviewed changes

tests/rancher/rancher_ui_selenium.pm Show resolved Hide resolved

ilausuch force-pushed the rancher_ui_test branch from b218735 to d4b4461 Compare February 4, 2021 06:16

ge0r reviewed Feb 4, 2021

View reviewed changes

ilausuch force-pushed the rancher_ui_test branch 2 times, most recently from 3c216d2 to 2ae61dd Compare February 4, 2021 11:58

pdostal reviewed Feb 4, 2021

View reviewed changes

tests/rancher/rancher_ui_selenium.pm Outdated Show resolved Hide resolved

pdostal approved these changes Feb 4, 2021

View reviewed changes

ilausuch force-pushed the rancher_ui_test branch 2 times, most recently from 344e936 to 0c79c9f Compare February 4, 2021 16:14

grisu48 approved these changes Feb 5, 2021

View reviewed changes

data/rancher/ui_test.py Outdated Show resolved Hide resolved

kalikiana marked this pull request as draft February 8, 2021 11:43

ilausuch force-pushed the rancher_ui_test branch from 0c79c9f to d6a1c1c Compare February 11, 2021 12:16

Rancher UI selenium test

1025959

ilausuch force-pushed the rancher_ui_test branch from d6a1c1c to 1025959 Compare February 11, 2021 13:01

okurz requested changes Feb 11, 2021

View reviewed changes

kalikiana closed this Feb 15, 2021

ilausuch reopened this Feb 17, 2021

ilausuch closed this Mar 16, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Rancher UI selenium test #11863

[WIP] Rancher UI selenium test #11863

ilausuch commented Jan 28, 2021 •

edited by kalikiana

Loading

pdostal commented Jan 29, 2021

ilausuch commented Feb 4, 2021

pdostal commented Feb 4, 2021

ge0r left a comment

pdostal left a comment

ilausuch commented Feb 11, 2021 •

edited

Loading

okurz Feb 11, 2021

okurz Feb 11, 2021

okurz Feb 11, 2021

okurz Feb 11, 2021

kalikiana commented Feb 15, 2021

pdostal commented Feb 15, 2021

kalikiana commented Feb 15, 2021 •

edited

Loading

ilausuch commented Feb 17, 2021

okurz commented Feb 17, 2021

ilausuch commented Mar 16, 2021

	assert_script_run("[ -f /tmp/ok ]");
	assert_script_run('test -f /tmp/ok');

[WIP] Rancher UI selenium test #11863

[WIP] Rancher UI selenium test #11863

Conversation

ilausuch commented Jan 28, 2021 • edited by kalikiana Loading

pdostal commented Jan 29, 2021

ilausuch commented Feb 4, 2021

pdostal commented Feb 4, 2021

ge0r left a comment

Choose a reason for hiding this comment

pdostal left a comment

Choose a reason for hiding this comment

ilausuch commented Feb 11, 2021 • edited Loading

okurz Feb 11, 2021

Choose a reason for hiding this comment

okurz Feb 11, 2021

Choose a reason for hiding this comment

okurz Feb 11, 2021

Choose a reason for hiding this comment

okurz Feb 11, 2021

Choose a reason for hiding this comment

kalikiana commented Feb 15, 2021

pdostal commented Feb 15, 2021

kalikiana commented Feb 15, 2021 • edited Loading

ilausuch commented Feb 17, 2021

okurz commented Feb 17, 2021

ilausuch commented Mar 16, 2021

ilausuch commented Jan 28, 2021 •

edited by kalikiana

Loading

ilausuch commented Feb 11, 2021 •

edited

Loading

kalikiana commented Feb 15, 2021 •

edited

Loading