[Integrations] Show Ingest Pipelines (and other assets) under Integration Assets tab #160555

nicpenning · 2023-06-26T17:13:36Z

Currently the displayed assets on the Integration Assets tab are based on these allowedAssetsTypes:

kibana/x-pack/plugins/fleet/common/constants/epm.ts

Lines 89 to 92 in 7d5a275

    
           KibanaAssetType.dashboard, 
        
           KibanaAssetType.search, 
        
           KibanaAssetType.visualization, 
        
           ElasticsearchAssetType.transform,

The tab was added in this PR, no more assets were added to it.

Todo:

Display Ingest Pipelines under the integration assets tab
Make sure that the links exist before displaying them.

The missing assets were previously reported as a bug in the description below

**Kibana version:** 8.8.1 **Elasticsearch version:** 8.8.1 **Server OS version:** N/A **Browser version:** Edge 114+ **Browser OS version:** N/A **Original install method (e.g. download page, yum, from source, etc.):** elastic-package stack up -d -v --version=8.8.1 **Describe the bug:** Currently, when installing an Integration that has Ingest Pipelines as Elasticsearch Assets, they do not show under the assets tab.

Steps to reproduce:

elastic-package stack up -d -v --version=8.8.1
Go to Microsoft DHCP integration (install)
Look at Assets tab and see that it is blank

Expected behavior:
I expect that all Elasticsearch assets be populated in the Assets tab

Screenshots (if relevant):

Here you can see that there are 3 Elasticsearch Assets that are Ingest Pipelines

Here you can see no Ingest Pipelines are shown as assets.

The text was updated successfully, but these errors were encountered:

elasticmachine · 2023-06-30T16:32:53Z

Pinging @elastic/fleet (Team:Fleet)

jlind23 · 2023-07-03T05:43:02Z

@criamico could this be a side effect of: #159653 ?

criamico · 2023-07-03T07:37:00Z

@jlind23 No I think that this is related to the changes done in #158350 or #159396

jlind23 · 2023-07-03T09:10:27Z

@criamico would you mind taking a look then? Looks like we have multiple occurence of this issue already.

criamico · 2023-07-03T10:44:53Z

I tested this same integration (Microsoft DHCP) on older versions (8.6.2 and 8.7.1) and looks like the assets are not available on any of them. It's possible that this bug has been there for a while.

@jlind23 do you know if there are other integrations impacted? I checked nginx, which also has ingest pipelines, and it appears to work fine.

jlind23 · 2023-07-03T10:52:11Z

@criamico when I said multiple occurrence I was referring to impact potentially caused by changed on Kibana API.
If this is related to this integration only then it is rather something we should transfer to @elastic/security-external-integrations team.

ShourieG · 2023-07-03T12:23:46Z

@criamico @jlind23 I can see that the dhcp integration does not have any kibana dashboards or visual assets except a logo img. Are kibana dashboards now mandatory in integrations with ingest pipelines ?

criamico · 2023-07-03T12:55:43Z

I checked as well and the only assets I can see in this integration are logs:

Are kibana dashboards now mandatory in integrations with ingest pipelines ?

@ShourieG I'm not aware of such a change. If you can confirm that there are no other assets to be displayed we can close this ticket.

ShourieG · 2023-07-03T13:23:46Z

@criamico those are the 3 ingest pipelines right ? I can see the type as ingest pipeline for them. This seems like a UI bug. The pipelines are present in the response but not showing up on the asset dashboard.

criamico · 2023-07-03T14:55:19Z

Yes those are the 3 ingest pipelines for this package, for some reason they're not displayed in the UI. I'll investigate why they're not visible anymore.

nicpenning · 2023-07-12T04:48:12Z

Perhaps ingest pipelines are not considered Assets? Looking at other integrations, the Kibana assets (dashboards, visualizations, etc) show up as assets.

My theory is that the assets tab is only for Kibana assets only, but I really don't know.

nicpenning · 2023-10-10T10:49:48Z

Pinging for an update here.

criamico · 2023-10-10T13:50:47Z

Hi @nicpenning,

Sorry for the late response, I missed your last comment.

I went back on checking the Assets tab and I can confirm that we are only displaying Kibana Assets there, not ingest pipelines, that are part of "ES" assets instead. That explains why you don't see them for "Microsoft DHCP" Integration. I'm not aware why we only show one type of assets and not the other.

I think that we could easily add a comment explaining what type of assets are expected there.

@nimarezainia any thoughts on this?

nimarezainia · 2023-10-11T02:22:50Z

@criamico what would be the effort involved in displaying all the assets?

For Nginx as an example I see:

but in the assets tab:

... so even within the Kibana assets there's a mismatch.

nicpenning · 2023-10-11T03:19:52Z

Ideally, it would be amazing that if all assets had direct links to them then you can pivot to any of the installed assets as needed.

criamico · 2023-10-11T07:51:59Z

@nimarezainia Currently the displayed assets on that tab are based on these allowedAssetsTypes:

kibana/x-pack/plugins/fleet/common/constants/epm.ts

Lines 89 to 92 in 7d5a275

    
           KibanaAssetType.dashboard, 
        
           KibanaAssetType.search, 
        
           KibanaAssetType.visualization, 
        
           ElasticsearchAssetType.transform,

I did a little bit of digging and that tab was added in this PR, I think that probably we never got around to add more assets to it. Ideally it shouldn't be a problem to display all of them (and solve that mismatch), the main thing would be to make sure that the links exist before displaying them.

I can open a separate enhancement ticket to describe the work needed for it.

cc @kpollich

nimarezainia · 2023-10-12T00:38:22Z

Thank you @criamico that would be fantastic.

jlind23 · 2023-10-12T04:29:05Z

I can open a separate enhancement ticket to describe the work needed for it.

@criamico I will repurpose this one and convert it to an enhancement request and not a bug fix.

jen-huang · 2024-04-09T22:35:05Z

@nimarezainia @kpollich Hi, I'm looking for feedback specifically around how to handle the ingest pipeline(s) that Fleet installs on top of the ones defined by the package. For example there are 5 total pipelines for Nginx, but the count today says 4:

Should we include Fleet pipelines as part of the total ingest pipelines count and show it on the asset list?

nimarezainia · 2024-04-10T04:05:12Z

@jen-huang I would say we should. I can't see a reason why we wouldn't want to show everything that touches the data collected.

jen-huang · 2024-04-29T20:50:53Z

@nimarezainia I will consider it in the spirit of this issue to also show index templates and component templates too. We already track and consider these as "ES assets" that are installed by Fleet. Let me know if you feel differently.

nicpenning · 2024-05-01T22:28:20Z

🎉

## Summary Resolves elastic#160555. This PR allows all assets to be shown on Integration details > Assets tab and links them to other apps in Kibana whenever possible for viewing. Previously, only dashboards, visualizations, and saved searches were shown in this view. Now all Kibana and Elasticsearch assets are shown if the integration was installed in the user's current space. If an integration was installed in a different space, only ES assets will be shown. #### Caveats 1) This page lists all assets tracked on the package installation saved object *after* the package is installed (`installed_es` and `installed_kibana`). This list differs from the summary of assets shown on the Overview tab because it includes Fleet-installed assets that are not part of the package, notably index templates, component templates, and default ingest pipelines. elastic#182197 was created to decide how to resolve this asset count discrepency. 2) ML and Security assets are shown but not linked. The following issues have been created for downstream teams to decide where their assets should link to: elastic#182199, elastic#182200 ### Screenshots <details> <summary>Nginx (including in a different space)</summary> ![image](https://github.com/elastic/kibana/assets/1965714/a2985314-5a08-45fb-9bce-8a4283464cd8) ![image](https://github.com/elastic/kibana/assets/1965714/97981e0c-3149-4629-83ec-3c718a393635) </details> <details> <summary>Security Posture Management</summary> ![image](https://github.com/elastic/kibana/assets/1965714/93314f9f-6797-4871-927a-ffe11f11f32f) </details> <details> <summary>Rapid7 Threat Command</summary> ![image](https://github.com/elastic/kibana/assets/1965714/d31578c6-711a-4d52-9b85-2f60267e41ba) </details> <details> <summary>Lateral Movement Detection</summary> ![image](https://github.com/elastic/kibana/assets/1965714/6720eceb-9e42-4024-8ab5-efef6553c3b7) </details> ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

nicpenning added the bug Fixes for quality problems that affect the customer experience label Jun 26, 2023

botelastic bot added the needs-team Issues missing a team label label Jun 26, 2023

wayneseymour added the Team:Fleet Team label for Observability Data Collection Fleet team label Jun 30, 2023

botelastic bot removed the needs-team Issues missing a team label label Jun 30, 2023

criamico self-assigned this Jul 3, 2023

jlind23 added the impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. label Jul 4, 2023

jlind23 added enhancement New value added to drive a business result and removed impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. labels Oct 12, 2023

jlind23 changed the title ~~[Integrations] Ingest Pipelines not showing under Integration Assets tab~~ [Integrations] Show Ingest Pipelines under Integration Assets tab Oct 12, 2023

criamico mentioned this issue Oct 18, 2023

[Integration]: On installing integration wrong number of Assets count shows under Assets tab. #126336

Open

criamico removed their assignment Mar 4, 2024

jen-huang removed the bug Fixes for quality problems that affect the customer experience label Mar 6, 2024

jen-huang changed the title ~~[Integrations] Show Ingest Pipelines under Integration Assets tab~~ [Integrations] Show Ingest Pipelines (and other assets) under Integration Assets tab Mar 6, 2024

kpollich assigned kaanyalti Mar 26, 2024

kpollich added the QA:Needs Validation Issue needs to be validated by QA label Mar 28, 2024

kpollich mentioned this issue Apr 8, 2024

Integration package - installed assets not showing ml_module, ingest_pipeline #178672

Closed

jen-huang assigned jen-huang and unassigned kaanyalti Apr 9, 2024

jen-huang mentioned this issue Apr 30, 2024

[Fleet] Show all integration assets on detail page #182180

Merged

2 tasks

jen-huang closed this as completed in #182180 May 1, 2024

jen-huang closed this as completed in 980d8bf May 1, 2024

jen-huang added QA:Ready for Testing Code is merged and ready for QA to validate and removed QA:Needs Validation Issue needs to be validated by QA labels May 17, 2024

This was referenced Jun 20, 2024

[Question] [Fleet]: Component Template and Index Template is display under Assets Tab. #186517

Closed

[Fleet]: The ingest pipeline counts on the Integration Overview and Integration Assets tab pages is not same. #186518

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Integrations] Show Ingest Pipelines (and other assets) under Integration Assets tab #160555

[Integrations] Show Ingest Pipelines (and other assets) under Integration Assets tab #160555

nicpenning commented Jun 26, 2023 •

edited by jlind23

Loading

elasticmachine commented Jun 30, 2023

jlind23 commented Jul 3, 2023

criamico commented Jul 3, 2023 •

edited

Loading

jlind23 commented Jul 3, 2023

criamico commented Jul 3, 2023

jlind23 commented Jul 3, 2023

ShourieG commented Jul 3, 2023

criamico commented Jul 3, 2023 •

edited

Loading

ShourieG commented Jul 3, 2023 •

edited

Loading

criamico commented Jul 3, 2023

nicpenning commented Jul 12, 2023

nicpenning commented Oct 10, 2023

criamico commented Oct 10, 2023

nimarezainia commented Oct 11, 2023

nicpenning commented Oct 11, 2023 •

edited

Loading

criamico commented Oct 11, 2023

nimarezainia commented Oct 12, 2023

jlind23 commented Oct 12, 2023 •

edited

Loading

jen-huang commented Apr 9, 2024

nimarezainia commented Apr 10, 2024

jen-huang commented Apr 29, 2024

nicpenning commented May 1, 2024 •

edited

Loading

[Integrations] Show Ingest Pipelines (and other assets) under Integration Assets tab #160555

[Integrations] Show Ingest Pipelines (and other assets) under Integration Assets tab #160555

Comments

nicpenning commented Jun 26, 2023 • edited by jlind23 Loading

elasticmachine commented Jun 30, 2023

jlind23 commented Jul 3, 2023

criamico commented Jul 3, 2023 • edited Loading

jlind23 commented Jul 3, 2023

criamico commented Jul 3, 2023

jlind23 commented Jul 3, 2023

ShourieG commented Jul 3, 2023

criamico commented Jul 3, 2023 • edited Loading

ShourieG commented Jul 3, 2023 • edited Loading

criamico commented Jul 3, 2023

nicpenning commented Jul 12, 2023

nicpenning commented Oct 10, 2023

criamico commented Oct 10, 2023

nimarezainia commented Oct 11, 2023

nicpenning commented Oct 11, 2023 • edited Loading

criamico commented Oct 11, 2023

nimarezainia commented Oct 12, 2023

jlind23 commented Oct 12, 2023 • edited Loading

jen-huang commented Apr 9, 2024

nimarezainia commented Apr 10, 2024

jen-huang commented Apr 29, 2024

nicpenning commented May 1, 2024 • edited Loading

🎉

nicpenning commented Jun 26, 2023 •

edited by jlind23

Loading

criamico commented Jul 3, 2023 •

edited

Loading

criamico commented Jul 3, 2023 •

edited

Loading

ShourieG commented Jul 3, 2023 •

edited

Loading

nicpenning commented Oct 11, 2023 •

edited

Loading

jlind23 commented Oct 12, 2023 •

edited

Loading

nicpenning commented May 1, 2024 •

edited

Loading