Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schedule: Disable sporadically failing HA test modules (poo#95788) #13475

Closed

Conversation

okurz
Copy link
Member

@okurz okurz commented Oct 15, 2021

For example as observed in
https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-HA-Updates&machine=64bit&test=qam_ha_rolling_upgrade_migration_node01&version=12-SP4
the test module "check_cluster_integrity" is failing often. auto-review
is catching these cases and linking the related progress issue. The
fail-rate from the above scenario seems to be around 70%. As the related
progress issue is "Normal" priority I do not assume this issue to be
fixed soon so despite of the obvious test coverage loss we need to
prevent sporadically failing tests that do not provide clear feedback
about the cause of the issue hence removing the test modules that fail
or are dependant on the failing module from the schedule.

Related progress issue: https://progress.opensuse.org/issues/95788

For example as observed in
https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-HA-Updates&machine=64bit&test=qam_ha_rolling_upgrade_migration_node01&version=12-SP4
the test module "check_cluster_integrity" is failing often. auto-review
is catching these cases and linking the related progress issue. The
fail-rate from the above scenario seems to be around 70%. As the related
progress issue is "Normal" priority I do not assume this issue to be
fixed soon so despite of the obvious test coverage loss we need to
prevent sporadically failing tests that do not provide clear feedback
about the cause of the issue hence removing the test modules that fail
or are dependant on the failing module from the schedule.

Related progress issue: https://progress.opensuse.org/issues/95788
@alvarocarvajald
Copy link
Contributor

alvarocarvajald commented Oct 18, 2021

It seems this PR is targeting the wrong schedule: linked test is using schedule/ha/qam/common/qam_ha_rolling_upgrade_migration_node01_sle12.yaml, but PR is changing schedule/ha/qam/common/qam_ha_rolling_upgrade_migration_node01.yaml.

Even if this is addressed, I think this PR will introduce other issues: even if the linked job qam_ha_rolling_upgrade_migration_node01 passes, the commented modules absence will impact node 2 and the support server, which will fail with lockapi related timeouts.

I have added more details at https://progress.opensuse.org/issues/95788#note-22.

@okurz
Copy link
Member Author

okurz commented Oct 18, 2021

I see. Then this approach is not helpful. You provided better suggestions in the ticket. Thanks.

@okurz okurz closed this Oct 18, 2021
@okurz okurz deleted the fix/poo95788_avoid_failing_modules branch October 18, 2021 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants