[Buildkite] Fix logic to check which packages are excluded in Kibana (Serverless) #9849

mrodm · 2024-05-13T15:35:06Z

Proposed commit message

~~Add support to exclude some packages by defining a new environment variable.~~

~~There are some packages that are trying to be installed in a Observability project in Elastic Serverless, but they cannot be installed.~~

~~Example of build: https://buildkite.com/elastic/integrations-serverless/builds/400~~

Updated Buildkite scripts to check which packages are excluded in the current Kibana configuration of each Serverless project type.

Here in this build it is shown that universal_profiling_agent (profiler_agent) package is skipped because it is added into the excluded list of Kibana.

andrewkroh · 2024-05-13T18:12:59Z

but they cannot be installed

Why can they not be installed? Is this a temporary limitation? Do these packages need to define a capabilities value in their manifest?

elasticmachine · 2024-05-13T20:30:13Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

mrodm · 2024-05-14T08:45:32Z

but they cannot be installed

Why can they not be installed? Is this a temporary limitation? Do these packages need to define a capabilities value in their manifest?

This is the error for security_detection_engine package:

Error: error running package asset tests: could not complete test run: can't install the package: there was an apply error: installation failed: can't install the package: could not zip-install package; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"Encountered 5224 errors creating saved objects: ...

This is the error for universal_profile_agent package:

Error: can't install the package: could not zip-install package; API status code = 403; response body = {"statusCode":403,"error":"Forbidden","message":"profiler_agent installation is not authorized"}

Probably security_detection_engine would require to define capabilities in its manifest, but there are already quite a few versions published with spec version >= 3.0.0. Those versions would be available in Observability projects too.

About universal_profiling_agent not sure what would be needed, but currently that package cannot be installed in Observability projects. In Security projects it works. This package has also a couple of versions using format_version >= 3.0.0.

Do you know if it can be done something else for these packages ? @jsoriano
Should universal_profiling_agent be available in Observability projects too ?

jsoriano · 2024-05-14T10:28:52Z

This is the error for security_detection_engine package:

Error: error running package asset tests: could not complete test run: can't install the package: there was an apply error: installation failed: can't install the package: could not zip-install package; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"Encountered 5224 errors creating saved objects: ...

Probably security_detection_engine would require to define capabilities in its manifest, but there are already quite a few versions published with spec version >= 3.0.0. Those versions would be available in Observability projects too.

Do you have the full error message? This is likely complaining about an unsupported saved object type. This package should probably define the security capability.

For packages already published, we may need to end up removing packages from the registry if they cause problems. In any case this package should be included in the exclude list for observability till these issues are handled https://github.com/elastic/kibana/blob/6c7907a2b7aa907ab06db758a7bf01c6c65290aa/config/serverless.oblt.yml#L52

This is the error for universal_profile_agent package:

Error: can't install the package: could not zip-install package; API status code = 403; response body = {"statusCode":403,"error":"Forbidden","message":"profiler_agent installation is not authorized"}

About universal_profiling_agent not sure what would be needed, but currently that package cannot be installed in Observability projects. In Security projects it works. This package has also a couple of versions using format_version >= 3.0.0.

No idea where this error comes from, but looks like this package should also include the security capability. And might be included in the observability exclude list in the meantime.

mrodm · 2024-05-14T11:01:35Z

Do you have the full error message? This is likely complaining about an unsupported saved object type. This package should probably define the security capability.

The error message is too long. This is the first saved objects failing:

Error: error running package asset tests: could not complete test run: can't install the package: there was an apply error: installation failed: can't install the package: could not zip-install package; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"Encountered 5224 errors creating saved objects: [{\"type\":\"security-rule\",\"id\":\"000047bb-b27a-47ec-8b62-ef1a5d2c9e19_102\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"000047bb-b27a-47ec-8b62-ef1a5d2c9e19_103\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"000047bb-b27a-47ec-8b62-ef1a5d2c9e19_104\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"000047bb-b27a-47ec-8b62-ef1a5d2c9e19_105\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"000047bb-b27a-47ec-8b62-ef1a5d2c9e19_106\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"000047bb-b27a-47ec-8b62-ef1a5d2c9e19_207\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"00140285-b827-4aee-aa09-8113f58a08f3_105\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"00140285-b827-4aee-aa09-8113f58a08f3_106\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"00140285-b827-4aee-aa09-8113f58a08f3_107\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"00140285-b827-4aee-aa09-8113f58a08f3_108\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"00140285-b827-4aee-aa09-8113f58a08f3_109\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"00140285-b827-4aee-aa09-8113f58a08f3_110\",\"error\":{\"type\":\"unsupported_type\"}},{\"type\":\"security-rule\",\"id\":\"00140285-b827-4aee-aa09-8113f58a08f3_111\",\"error\":{\"type\":\"unsupported_type\"}}...

Full log can be downloaded from the buildkite step: https://buildkite.com/elastic/integrations-serverless/builds/407#018f74bb-d02c-45e1-95bf-c865ae95f05b

mrodm · 2024-05-14T11:05:11Z

About universal_profiling_agent package probably should be available in Observability ?
This is the documentation about it: https://www.elastic.co/guide/en/observability/current/universal-profiling.html

jsoriano · 2024-05-14T11:13:37Z

About universal_profiling_agent package probably should be available in Observability ? This is the documentation about it: https://www.elastic.co/guide/en/observability/current/universal-profiling.html

Ah yes, you are right, profiling should work in observability. @elastic/profiling do you know what could be causing the failure mentioned in #9849 (comment): profiler_agent installation is not authorized.

mrodm · 2024-05-21T09:58:37Z

About universal_profiling_agent package probably should be available in Observability ? This is the documentation about it: https://www.elastic.co/guide/en/observability/current/universal-profiling.html

Ah yes, you are right, profiling should work in observability. @elastic/profiling do you know what could be causing the failure mentioned in #9849 (comment): profiler_agent installation is not authorized.

@elastic/profiling should the universal_profiling_agent package be available for observability projects in Elastic Serverless? Currently, this package fails with this error:

Error: can't install the package: could not zip-install package; API status code = 403; response body = {"statusCode":403,"error":"Forbidden","message":"profiler_agent installation is not authorized"}

If so, I could update the list of packages to be skipped to just "security_detection_engine" package for now.

danielmitterdorfer · 2024-05-21T11:44:28Z

should the universal_profiling_agent package be available for observability projects in Elastic Serverless?

No, Universal Profiling is currently not supported for Observability projects.

mrodm · 2024-05-22T11:34:21Z

univeral_profiliing_agent should be skipped according to the kibana config, it has been fixed here: f4f41bd

In this test build, it is already shown as skipped: https://buildkite.com/elastic/integrations-serverless/builds/425

For packages already published, we may need to end up removing packages from the registry if they cause problems. In any case this package should be included in the exclude list for observability till these issues are handled https://github.com/elastic/kibana/blob/6c7907a2b7aa907ab06db758a7bf01c6c65290aa/config/serverless.oblt.yml#L52

According to this, I think it would be better to create a PR to update the exclude list to add security_detection_engine there. WDYT? @kpollich @jsoriano

Added security capability in security_detection_engine here: #9938

jsoriano · 2024-05-22T12:30:58Z

According to this, I think it would be better to create a PR to update the exclude list to add security_detection_engine there. WDYT? @kpollich @jsoriano

Yes, I think we need to do both things at this point, adding it to the exclude list, and adding the capability for the future.

elasticmachine · 2024-05-22T14:41:39Z

⏳ Build in-progress, with failures

Buildkite Build
Commit: c90f356

Failed CI Steps

History

💔 Build #11770 failed aed44c3
💔 Build #11759 failed f54b7e3
💔 Build #11757 failed 813d901
💔 Build #11396 failed b9bf0bf

cc @mrodm

mrodm · 2024-05-22T14:54:03Z

.buildkite/scripts/common.sh

+    local package_name=""
+    package_name=$(package_name_manifest)
+
+    if echo "${excluded_packages}" | grep -q -E "\"${package_name}\""; then


Retrieved the package name from the manifest to compare with the values from Kibana.

elastic-sonarqube · 2024-05-22T17:10:51Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

mrodm · 2024-05-22T17:39:12Z

CI failures are unrelated to this change, merging it.

…jects (#184022) ## Summary Include [`security_detection_engine` package](https://github.com/elastic/integrations/tree/d2a74171c8fc32fd8754af0a2dd733669b5f6578/packages/security_detection_engine) into the list of excluded packages for Observability projects in Elastic Serverless. Relates elastic/integrations#9849 Relates elastic/integrations#9938 ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

…jects (elastic#184022) ## Summary Include [`security_detection_engine` package](https://github.com/elastic/integrations/tree/d2a74171c8fc32fd8754af0a2dd733669b5f6578/packages/security_detection_engine) into the list of excluded packages for Observability projects in Elastic Serverless. Relates elastic/integrations#9849 Relates elastic/integrations#9938 ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Filter security packages in Elastic Serverless (Observability) daily job

a1341ab

mrodm self-assigned this May 13, 2024

mrodm requested a review from a team as a code owner May 13, 2024 15:35

mrodm changed the title ~~Filter security packages in Elastic Serverless (Observability) daily job~~ [Buildkite] Filter security packages in Elastic Serverless (Observability) daily job May 13, 2024

jlind23 approved these changes May 13, 2024

View reviewed changes

mrodm added 2 commits May 13, 2024 19:40

Move to common.sh

849fe3b

Update function name

b9bf0bf

mrodm added 3 commits May 22, 2024 11:44

Use package names to check if it is excluded

f4f41bd

Test trigger serverless

813d901

debug - Add missing popd

f54b7e3

mrodm mentioned this pull request May 22, 2024

Add security capabilities for security_detection_engine #9938

Merged

4 tasks

mrodm mentioned this pull request May 22, 2024

Add security_detection_engine into exclude list for observability projects elastic/kibana#184022

Merged

1 task

mrodm added 5 commits May 22, 2024 16:08

Restore main pipeline

8ea6653

Remove code for excluded list env. var

38ad60b

Add retry for docker

6527d35

Remove empty line

aed44c3

Fix retry command

c90f356

mrodm changed the title ~~[Buildkite] Filter security packages in Elastic Serverless (Observability) daily job~~ [Buildkite] Fix logic to check which packages are excluded in Kibana (Serverless) May 22, 2024

mrodm commented May 22, 2024

View reviewed changes

mrodm requested review from a team and jlind23 May 22, 2024 14:54

jsoriano approved these changes May 22, 2024

View reviewed changes

mrodm merged commit 13f1f6f into elastic:main May 22, 2024
4 of 5 checks passed

mrodm deleted the filter-security-packages branch May 22, 2024 17:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Buildkite] Fix logic to check which packages are excluded in Kibana (Serverless) #9849

[Buildkite] Fix logic to check which packages are excluded in Kibana (Serverless) #9849

mrodm commented May 13, 2024 •

edited

andrewkroh commented May 13, 2024

elasticmachine commented May 13, 2024 •

edited

mrodm commented May 14, 2024

jsoriano commented May 14, 2024

mrodm commented May 14, 2024

mrodm commented May 14, 2024

jsoriano commented May 14, 2024

mrodm commented May 21, 2024

danielmitterdorfer commented May 21, 2024

mrodm commented May 22, 2024

jsoriano commented May 22, 2024

elasticmachine commented May 22, 2024 •

edited

mrodm May 22, 2024

elastic-sonarqube bot commented May 22, 2024

mrodm commented May 22, 2024

[Buildkite] Fix logic to check which packages are excluded in Kibana (Serverless) #9849

[Buildkite] Fix logic to check which packages are excluded in Kibana (Serverless) #9849

Conversation

mrodm commented May 13, 2024 • edited

Proposed commit message

andrewkroh commented May 13, 2024

elasticmachine commented May 13, 2024 • edited

🚀 Benchmarks report

mrodm commented May 14, 2024

jsoriano commented May 14, 2024

mrodm commented May 14, 2024

mrodm commented May 14, 2024

jsoriano commented May 14, 2024

mrodm commented May 21, 2024

danielmitterdorfer commented May 21, 2024

mrodm commented May 22, 2024

jsoriano commented May 22, 2024

elasticmachine commented May 22, 2024 • edited

⏳ Build in-progress, with failures

Failed CI Steps

History

mrodm May 22, 2024

Choose a reason for hiding this comment

elastic-sonarqube bot commented May 22, 2024

Quality Gate passed

mrodm commented May 22, 2024

mrodm commented May 13, 2024 •

edited

elasticmachine commented May 13, 2024 •

edited

elasticmachine commented May 22, 2024 •

edited