[Citrix ADC] Add Integration Package with System Data Stream #4458

harnish-crest · 2022-10-13T13:48:24Z

Enhancement

What does this PR do?

Added 1 data stream (System metrics).
Added data collection logic for the data streams.
Added the ingest pipeline for the data streams.
Mapped fields according to the ECS schema and added Fields metadata in the appropriate yaml files.
Added dashboards and visualizations.
Added system test cases for the data stream.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Clone integrations repo.
Install elastic-package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/citrix_adc directory.
Run the following command to run tests.
elastic-package test

Related issues

Screenshots

elasticmachine · 2022-10-13T13:51:50Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-12-13T09:40:37.672+0000
Duration: 15 min 30 sec

Test stats 🧪

Test	Results
Failed	0
Passed	25
Skipped	0
Total	25

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

yug-rajani · 2022-10-13T13:53:56Z

/test

elasticmachine · 2022-10-13T14:08:42Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elasticmachine · 2022-10-13T14:08:59Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`4/4`)	💚
Files	100.0% (`4/4`)	💚
Classes	100.0% (`4/4`)	💚
Methods	100.0% (`39/39`)	💚
Lines	97.639% (`827/847`)	❕
Conditionals	100.0% (`0/0`)	💚

kush-elastic

LGTM

cla-checker-service · 2022-11-08T06:23:35Z

💚 CLA has been signed

packages/citrix_adc/_dev/build/docs/README.md

muthu-mps · 2022-12-12T10:28:27Z

packages/citrix_adc/data_stream/system/elasticsearch/ingest_pipeline/default.yml

+      - EEE MMM  d HH:mm:ss yyyy
+      - EEE MMM  d HH:mm:ss.SSSSSS yyyy
+  - script:
+      description: Convert megabyte to byte.


Why do we need to convert the MB to bytes ?

Kibana provides functionality to automatically convert bytes into human-readable units, to provide support for this functionality in the dashboard we have converted MB into bytes.

I am just wondering why are we converting the MB to bytes which adds extra overhead at the ingest pipeline and then showing the values as MB in dashboards ? Is it not possible to show the MB directly in Kibana without any conversion ?

packages/citrix_adc/manifest.yml

muthu-mps · 2022-12-12T10:30:48Z

In the dashboards can we remove the time when the application was started.
The Kibana system memory shows 16 MB and used memory shows 218MB. Is it representing the data correctly ?

harnish-crest · 2022-12-12T10:51:39Z

In the dashboards can we remove the time when the application was started. The Kibana system memory shows 16 MB and used memory shows 218MB. Is it representing the data correctly ?

The reason behind providing this visualization is, for e.g If a user identifies that CPU/Memory Utilization keeps on increasing, they'll know that this Netscaler instance has been running since it keeps increasing so they can reboot it or something. So all the necessary information would be available on the dashboards themselves, User won't need to go in discover to identify since what date the Netscaler appliance is configured.

muthu-mps · 2022-12-12T11:03:20Z

In the dashboards can we remove the time when the application was started. The Kibana system memory shows 16 MB and used memory shows 218MB. Is it representing the data correctly ?

The reason behind providing this visualization is, for e.g If a user identifies that CPU/Memory Utilization keeps on increasing, they'll know that this Netscaler instance has been running since it keeps increasing so they can reboot it or something. So all the necessary information would be available on the dashboards themselves, User won't need to go in discover to identify since what date the Netscaler appliance is configured.

Can you please provide an example where we need to reboot the instance based on utilisation ?

harnish-crest · 2022-12-12T11:58:14Z

In the dashboards can we remove the time when the application was started. The Kibana system memory shows 16 MB and used memory shows 218MB. Is it representing the data correctly ?

The reason behind providing this visualization is, for e.g If a user identifies that CPU/Memory Utilization keeps on increasing, they'll know that this Netscaler instance has been running since it keeps increasing so they can reboot it or something. So all the necessary information would be available on the dashboards themselves, User won't need to go in discover to identify since what date the Netscaler appliance is configured.

Can you please provide an example where we need to reboot the instance based on utilisation ?

We don’t have any concrete examples, but this was the thought process we were having while working on this dashboard. If you don’t see any value getting provided to the end user, we can remove it.

muthu-mps · 2022-12-13T06:05:49Z

In the dashboards can we remove the time when the application was started. The Kibana system memory shows 16 MB and used memory shows 218MB. Is it representing the data correctly ?

The reason behind providing this visualization is, for e.g If a user identifies that CPU/Memory Utilization keeps on increasing, they'll know that this Netscaler instance has been running since it keeps increasing so they can reboot it or something. So all the necessary information would be available on the dashboards themselves, User won't need to go in discover to identify since what date the Netscaler appliance is configured.

Can you please provide an example where we need to reboot the instance based on utilisation ?

We don’t have any concrete examples, but this was the thought process we were having while working on this dashboard. If you don’t see any value getting provided to the end user, we can remove it.

Please remove the visualisation.

muthu-mps · 2022-12-12T10:59:54Z

packages/citrix_adc/data_stream/system/elasticsearch/ingest_pipeline/default.yml

+      - EEE MMM  d HH:mm:ss yyyy
+      - EEE MMM  d HH:mm:ss.SSSSSS yyyy
+  - script:
+      description: Convert megabyte to byte.


I am just wondering why are we converting the MB to bytes which adds extra overhead at the ingest pipeline and then showing the values as MB in dashboards ? Is it not possible to show the MB directly in Kibana without any conversion ?

harnish-crest · 2022-12-14T10:21:31Z

I am just wondering why are we converting the MB to bytes which adds extra overhead at the ingest pipeline and then showing the values as MB in dashboards ? Is it not possible to show the MB directly in Kibana without any conversion ?

As per my understanding, currently, there is no direct way to convert MB to other formats, the current possible approach is to convert Mb to bytes so that the bytes can be automatically converted to human-readable format(KB, MB, TB, ...) using the Kibana visualization itself, let me know if you have any other suggestions on the same.

muthu-mps · 2022-12-14T14:27:39Z

I am just wondering why are we converting the MB to bytes which adds extra overhead at the ingest pipeline and then showing the values as MB in dashboards ? Is it not possible to show the MB directly in Kibana without any conversion ?

As per my understanding, currently, there is no direct way to convert MB to other formats, the current possible approach is to convert Mb to bytes so that the bytes can be automatically converted to human-readable format(KB, MB, TB, ...) using the Kibana visualization itself, let me know if you have any other suggestions on the same.

For now this is fine. But we may need to look for an option to do it directly instead of applying convertions.

elasticmachine · 2022-12-15T06:03:10Z

Package citrix_adc - 0.4.0 containing this change is available at https://epr.elastic.co/search?package=citrix_adc

harnish-crest added 2 commits October 13, 2022 19:15

Add system data stream

fd62c33

update changelog.yml

c93bec4

harnish-crest marked this pull request as draft October 13, 2022 13:51

yug-rajani requested review from lalit-satapathy and a team October 13, 2022 13:51

yug-rajani assigned harnish-crest Oct 13, 2022

yug-rajani requested a review from kush-elastic October 13, 2022 13:51

harnish-crest mentioned this pull request Oct 13, 2022

[Citrix ADC] Add Integration Package with lbvserver Data Stream #4418

Merged

4 tasks

This was referenced Oct 13, 2022

[Citrix ADC] Add Integration Package with Interface Data Stream #4322

Merged

[Citrix ADC] Add Integration Package with Service Data Stream #4364

Merged

yug-rajani requested a review from muthu-mps October 13, 2022 13:53

yug-rajani linked an issue Oct 13, 2022 that may be closed by this pull request

Create package to monitor server workers using Citrix ADC (Netscaler) #3746

Closed

5 tasks

yug-rajani added enhancement New feature or request Integration:Citrix-Netscaler Team:Service-Integrations Label for the Service Integrations team labels Oct 13, 2022

kush-elastic approved these changes Oct 19, 2022

View reviewed changes

harnish-crest mentioned this pull request Oct 19, 2022

[Citrix ADC] Add Integration Package with VPN Data Stream #4473

Merged

4 tasks

harnish-crest force-pushed the package_citrix_adc_system branch from 470d424 to 06cffd2 Compare November 8, 2022 06:31

update visualization titles and pipeline

c9abfda

harnish-crest force-pushed the package_citrix_adc_system branch from 06cffd2 to c9abfda Compare November 8, 2022 06:56

muthu-mps mentioned this pull request Nov 11, 2022

Create package to monitor server workers using Citrix ADC (Netscaler) #3746

Closed

5 tasks

resolve merge conflicts

b50575e

harnish-crest marked this pull request as ready for review December 7, 2022 11:58

muthu-mps reviewed Dec 12, 2022

View reviewed changes

remove visualization and minor changes in README.md

389f7c2

muthu-mps approved these changes Dec 14, 2022

View reviewed changes

kush-elastic merged commit c2fbb2c into elastic:main Dec 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Citrix ADC] Add Integration Package with System Data Stream #4458

[Citrix ADC] Add Integration Package with System Data Stream #4458

harnish-crest commented Oct 13, 2022 •

edited

Loading

elasticmachine commented Oct 13, 2022 •

edited

Loading

Build stats

Test stats 🧪

yug-rajani commented Oct 13, 2022

elasticmachine commented Oct 13, 2022

elasticmachine commented Oct 13, 2022 •

edited

Loading

kush-elastic left a comment

cla-checker-service bot commented Nov 8, 2022 •

edited

Loading

muthu-mps Dec 12, 2022

harnish-crest Dec 12, 2022

muthu-mps Dec 12, 2022

muthu-mps commented Dec 12, 2022 •

edited

Loading

harnish-crest commented Dec 12, 2022

muthu-mps commented Dec 12, 2022

harnish-crest commented Dec 12, 2022

muthu-mps commented Dec 13, 2022

muthu-mps Dec 12, 2022

harnish-crest commented Dec 14, 2022

muthu-mps commented Dec 14, 2022

elasticmachine commented Dec 15, 2022

[Citrix ADC] Add Integration Package with System Data Stream #4458

[Citrix ADC] Add Integration Package with System Data Stream #4458

Conversation

harnish-crest commented Oct 13, 2022 • edited Loading

What does this PR do?

Checklist

How to test this PR locally

Related issues

Screenshots

elasticmachine commented Oct 13, 2022 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

yug-rajani commented Oct 13, 2022

elasticmachine commented Oct 13, 2022

🚀 Benchmarks report

elasticmachine commented Oct 13, 2022 • edited Loading

🌐 Coverage report

kush-elastic left a comment

Choose a reason for hiding this comment

cla-checker-service bot commented Nov 8, 2022 • edited Loading

muthu-mps Dec 12, 2022

Choose a reason for hiding this comment

harnish-crest Dec 12, 2022

Choose a reason for hiding this comment

muthu-mps Dec 12, 2022

Choose a reason for hiding this comment

muthu-mps commented Dec 12, 2022 • edited Loading

harnish-crest commented Dec 12, 2022

muthu-mps commented Dec 12, 2022

harnish-crest commented Dec 12, 2022

muthu-mps commented Dec 13, 2022

muthu-mps Dec 12, 2022

Choose a reason for hiding this comment

harnish-crest commented Dec 14, 2022

muthu-mps commented Dec 14, 2022

elasticmachine commented Dec 15, 2022

harnish-crest commented Oct 13, 2022 •

edited

Loading

elasticmachine commented Oct 13, 2022 •

edited

Loading

elasticmachine commented Oct 13, 2022 •

edited

Loading

cla-checker-service bot commented Nov 8, 2022 •

edited

Loading

muthu-mps commented Dec 12, 2022 •

edited

Loading