[Security Solution] Ensure full test coverage for existing workflows of installing and upgrading prebuilt rules

[banderror]

Epic: https://github.com/elastic/security-team/issues/1974 (internal)

Summary

We're going to add support for the security_detection_engine package with historical versions of prebuilt rules. It's going to be supported within the existing functionality and by the existing API endpoints. The difference will be: under the hood, instead of having only the latest versions of prebuilt security-rule asset saved objects, we will have all historical versions.

Before we do that, we need to make sure we have full test coverage for existing workflows of installing and upgrading prebuilt rules. When we're about to release the package with historical versions, we will have a trustworthy test suite to verify that we won't break the app by releasing this package.

Todo

• Write a test plan for existing workflows of installing and upgrading prebuilt rules. Decide what scenarios should be automated at which level (e2e, integration, unit). (Test plan)
• Revisit API integration test coverage for the 2 existing rules/prepackaged endpoints. Make it match the test plan: add missing tests, etc.
• Revisit Cypress test coverage for the workflows. Make it match the test plan: add missing tests, etc.
• Add tests to ensure Kibana works fine with both package types: with historical versions and without.
• Add e2e tests to ensure that detection rules package installation doesn't fail with OOM: Increase the maximum number of saved objects that could be installed with a Fleet package #148441 (comment)
• Test installation with a large number of rules in a package (like 15000)

Non-goal: unit test coverage. We can do it when we add support for historical versions, where needed.

Test plan

https://docs.google.com/document/d/1d_1DYnHlnCaPznWTjeCxhoaRUwxc2O_V0LToAPG0xLE/edit

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[xcrzx]

Note: add e2e tests to ensure that detection rules package installation doesn't fail with OOM: #148441 (comment)

[xcrzx]

Non-functional requirements

• All scenarios should work regardless of detection engine package type: with historical rule versions or without.
• All scenarios should work regardless of the package registry availability. I.e., they should also work in air-gapped environments without access to the internet.
• Rule installation or upgrade workflow should work with packages containing up to 15000 historical rules versions
• Kibana should not OOM during package installation
• For test purposes, it should be possible to use detection rules package versions lower than the latest.

Test scenarios

Feature: Rule update/installation notifications
  Prebuilt rules are updated, and new rules are added to the detection engine on a regular basis. Users should be notified when there are updates available for installed prebuilt rules and when new prebuilt rules are available to install.

  Scenario: Users notified about prebuilt rules
    Given users don't have prebuilt detection rules installed in Kibana
    When they navigate to the rules management page
    Then they should see a call to action to install prebuilt rules

  Scenario: Users are notified about prebuilt rule updates
    Given users have prebuilt rules installed in Kibana
    And there are updates available for prebuilt rules
    When they navigate to the rules management page
    Then they should see a call to action to update prebuilt rules
    And they should see the number of outdated rules

  Scenario: Users are notified about new prebuilt rules available to install
    Given users have prebuilt rules installed in Kibana
    And there are new prebuilt rules available to install
    When they navigate to the rules management page
    Then they should see a call to action to install new prebuilt rules
    And they should see the number of new rules available to install

  Scenario: No updates available
    Given users have the latest prebuilt rules installed in Kibana
    When they navigate to the rules management page
    Then they should not see a call to action to update or install prebuilt detection rules

Feature: Rule update/installation workflow
  Scenario: Users install the latest prebuilt rules
    Given users don't have prebuilt detection rules installed in Kibana
    When they navigate to the rules management page
    And they click install rules
    Then they should see a loading state
    And they should see a success message
    And they should see the prebuilt rules installed in the rules table
    And the install rules call to action should disappear

  Scenario: Users update prebuilt rules
    Given users have outdated prebuilt rules installed in Kibana
    When they navigate to the rules management page
    And they click update rules
    Then they should see a loading state
    And they should see a success message
    And they should see the prebuilt rules updated in the rules table
    And the update, rules call to action should disappear

  Scenario: Installation or update of prebuilt rules fails
    Given users have updates available for prebuilt rules
    When they navigate to the rules management page
    And they click install or update rules
    And the installation of prebuilt rules fails
    Then they should see a loading state
    And they should see an error message
    And they should see the prebuilt rules not updated in the rules table
    And the install/update rules call to action should still be visible

  Scenario: Users should not see prerelease rules
    Given users have prebuilt rules installed in Kibana
    And there is a new prerelease rules package available
    When users navigate to the rules management page
    Then they should not see rule updates available