[Security Solution] Support testing of prerelease detection rules with Kibana

[xcrzx]

Epic: https://github.com/elastic/security-team/issues/1974 (internal)

Summary

Use prerelease rules package versions for local development and CI

Fleet recently removed support of snapshot and staging registries. All environments now use production packages, including local setups and CI. That means there is no way to test prerelease versions of detection rules with Kibana.

Fleet proposes using semantic versioning to mark prerelease packages, such as 8.5.1-next. To ensure proper testing, we need to update our code to install prerelease versions of the prepackaged rules package during local development and in CI. The logic should be similar to what we had previously:

kibana/x-pack/plugins/fleet/server/services/epm/registry/registry_url.ts

Lines 22 to 31 in ba367bc

	const getDefaultRegistryUrl = (): string => {
	const branch = appContextService.getKibanaBranch();
	if (branch === 'main') {
	return SNAPSHOT_REGISTRY_URL_CDN;
	} else if (appContextService.getKibanaVersion().includes('-SNAPSHOT')) {
	return STAGING_REGISTRY_URL_CDN;
	} else {
	return PRODUCTION_REGISTRY_URL_CDN;
	}
	};

We should call methods that install packages with prerelease: true to install prerelease packages.

diff --git a/x-pack/plugins/security_solution/public/common/hooks/use_upgrade_security_packages.ts b/x-pack/plugins/security_solution/public/common/hooks/use_upgrade_security_packages.ts
index 848f1458502..e476a91cc93 100644
--- a/x-pack/plugins/security_solution/public/common/hooks/use_upgrade_security_packages.ts
+++ b/x-pack/plugins/security_solution/public/common/hooks/use_upgrade_security_packages.ts
@@ -24,6 +24,9 @@ const sendUpgradeSecurityPackages = async (
 ): Promise<BulkInstallPackagesResponse> => {
   return http.post<BulkInstallPackagesResponse>(epmRouteService.getBulkInstallPath(), {
     ...options,
+    query: {
+      prerelease: true,
+    },
     body: JSON.stringify({
       packages: ['endpoint', 'security_detection_engine'],
     }),

Allow selecting the rules package version for testing

Currently, it is impossible to install the prebuilt rules package of a version other than the latest. Therefore, to alleviate the prebuilt rules package testing (see comment), we need to add the ability to specify desired package version in the Kibana config.

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[spong]

Additional artifacts from working this now that #148426 is merged:

Internal docs PR for outlining our different test configurations: https://github.com/elastic/security-team/pull/5762
Sample Kibana PR for testing cypress against the 8.3.4-beta.1 package #149081