Flush out genuine flakes

[full ci]
bugsnag · Jan 17, 2021 · 6fc7114 · 6fc7114
1 parent ddf8712
commit 6fc7114
Show file tree

Hide file tree

Showing 9 changed files with 12 additions and 12 deletions.
diff --git a/features/full_tests/batch_1/async_error_flush.feature b/features/full_tests/batch_1/async_error_flush.feature
@@ -1,21 +1,21 @@
 Feature: Flushing errors does not send duplicates
 
-@Flaky
+#@Flaky
 Scenario: Only 1 request sent if connectivity change occurs before launch
     When I run "AsyncErrorConnectivityScenario" and relaunch the app
     And I configure Bugsnag for "AsyncErrorConnectivityScenario"
     Then I wait to receive a request
     And the request is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
     And the event "context" equals "AsyncErrorConnectivityScenario"
 
-@Flaky
+#@Flaky
 Scenario: Only 1 request sent if multiple connectivity changes occur
     When I run "AsyncErrorDoubleFlushScenario"
     Then I wait to receive a request
     And the request is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
     And the event "context" equals "AsyncErrorDoubleFlushScenario"
 
-@Flaky
+#@Flaky
 Scenario: Only 1 request sent if connectivity change occurs after launch
     When I run "AsyncErrorLaunchScenario"
     Then I wait to receive a request

diff --git a/features/full_tests/batch_1/bugsnag_init.feature b/features/full_tests/batch_1/bugsnag_init.feature
@@ -1,6 +1,6 @@
 Feature: Verify the Bugsnag Initialization methods
 
-@Flaky
+#@Flaky
 Scenario: Test Bugsnag initializes correctly
     When I run "BugsnagInitScenario"
     And I wait to receive a request

diff --git a/features/full_tests/batch_1/in_foreground.feature b/features/full_tests/batch_1/in_foreground.feature
@@ -7,7 +7,8 @@ Scenario: Test handled exception after delay
     And the event "app.inForeground" is true
 
 # Skip due to an issue on later Android platforms - [PLAT-5464]
-@Flaky @skip_android_11 @skip_android_10
+# Possible @Flaky
+@skip_android_11 @skip_android_10
 Scenario: Test handled exception in background
     When I run "InForegroundScenario"
     And I send the app to the background for 1 seconds

diff --git a/features/full_tests/batch_1/load_configuration.feature b/features/full_tests/batch_1/load_configuration.feature
@@ -1,6 +1,6 @@
 Feature: Loading values into the configuration
 
-@Flaky
+#@Flaky
 Scenario: Load configuration initialised from the Manifest
     When I run "LoadConfigurationFromManifestScenario"
     Then I wait to receive a request

diff --git a/features/full_tests/batch_2/native_api.feature b/features/full_tests/batch_2/native_api.feature
@@ -9,7 +9,7 @@ Feature: Native API
         And the exception "message" equals "9 out of 10 adults do not get their 5-a-day"
         And the event "unhandled" is false
 
-    @Flaky
+    #@Flaky
     Scenario: Starting a session, notifying, followed by a C crash
         When I run "CXXSessionInfoCrashScenario" and relaunch the app
         And I configure the app to run in the "non-crashy" state

diff --git a/features/full_tests/batch_2/naughty_strings.feature b/features/full_tests/batch_2/naughty_strings.feature
@@ -1,6 +1,6 @@
 Feature: The notifier handles user data containing unusual strings
 
-@Flaky
+#@Flaky
 Scenario: Test handled JVM error
     When I run "NaughtyStringScenario"
     Then I wait to receive a request

diff --git a/features/full_tests/batch_2/session_stopping.feature b/features/full_tests/batch_2/session_stopping.feature
@@ -25,7 +25,7 @@ Scenario: When a session is resumed the error uses the previous session informat
     And the payload field "events.0.session.events.handled" equals 2
     And the payload field "events.0.session.id" equals the stored value "resumed_session_id"
 
-@Flaky
+#@Flaky
 Scenario: When a new session is started the error uses different session information
     When I run "NewSessionScenario"
     Then I wait to receive 4 requests

diff --git a/features/full_tests/batch_2/user.feature b/features/full_tests/batch_2/user.feature
@@ -28,7 +28,7 @@ Scenario: Only User email field set
     And the event "user.email" equals "user@example.com"
     And the event "user.name" is null
 
-@Flaky
+#@Flaky
 Scenario: All user fields set
     When I run "UserEnabledScenario"
     And I wait to receive a request

diff --git a/features/support/env.rb b/features/support/env.rb
@@ -35,8 +35,7 @@
 end
 
 Before('@skip_android_8_1') do |scenario|
-  # TODO See what happens nowadays
-  # skip_this_scenario("Skipping scenario") if MazeRunner.config.os_version == 8.1
+  skip_this_scenario("Skipping scenario") if MazeRunner.config.os_version == 8.1
 end
 
 Before('@skip_android_10') do |scenario|