Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SecuritySolution] [8.11] Fix fleet_integration.ts API integration test flake #170510

Merged
merged 1 commit into from
Nov 7, 2023

Conversation

jpdjere
Copy link
Contributor

@jpdjere jpdjere commented Nov 3, 2023

Fixes: #167056

See comment in ticket.
This is only the fix for 8.11.

Summary

  • Changes the installation of the security_detection_engine package from the current Fleet bulk install packages endpoint to use the endpoint that installs a single package, in order to get to a more stable test
  • Notice that the errors reported that Kibana machine are socket hangups/timeouts and are solved on retry, so they do not block any pipelines)

For maintainers

@jpdjere jpdjere added test backport:skip This commit does not require backporting Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules v8.11.0 labels Nov 3, 2023
@jpdjere jpdjere self-assigned this Nov 3, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jpdjere

Copy link
Contributor

@banderror banderror left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpdjere Is there any evidence, besides the green CI, that these changes reduce the flakiness?

@@ -12,7 +12,7 @@ import {
getPrebuiltRulesAndTimelinesStatus,
} from '../../utils';
import { deleteAllPrebuiltRuleAssets } from '../../utils/prebuilt_rules/delete_all_prebuilt_rule_assets';
import { installPrebuiltRulesFleetPackage } from '../../utils/prebuilt_rules/install_prebuilt_rules_fleet_package';
import { installPrebuiltRulesPackageViaFleetAPI } from '../../utils/prebuilt_rules/install_fleet_package_by_url';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think we should do some refactoring in main in another PR. There are 2 files containing 4 functions for doing something with the fleet package with prebuilt rules.

@jpdjere
Copy link
Contributor Author

jpdjere commented Nov 7, 2023

@jpdjere Is there any evidence, besides the green CI, that these changes reduce the flakiness?

No, actually: the issue has been extremely difficult to reproduce, it couldn't even be resurfaced with the flaky test runner. And Kibana Machine was reopening the ticket even when the test passed on a retry. The socket hangups/timeouts errors that were reported pointed to an issue on Fleet serverside (the endpoint timeing out, maybe), so switching to an individual package installation endpoint will hopefully give us a stabler test. There's no firm evidence, but I will keep an eye on the ticket to see if the flake is re-reported on 8.11.

@jpdjere jpdjere merged commit 4ee1728 into elastic:8.11 Nov 7, 2023
24 checks passed
@mistic mistic added v8.11.1 and removed v8.11.0 labels Nov 8, 2023
@mistic
Copy link
Member

mistic commented Nov 8, 2023

This PR didn't make it into the last BC of 8.11.0. Updating the labels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test v8.11.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants