Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Ingest pipelines] add community id processor #103863

Merged
merged 23 commits into from Jul 21, 2021
Merged

[Ingest pipelines] add community id processor #103863

merged 23 commits into from Jul 21, 2021

Conversation

sabarasaba
Copy link
Member

@sabarasaba sabarasaba commented Jun 30, 2021 •
edited

Fixes: #86321

As discussed with @dborodyansky, the two fields that depend on each other (iana_number and transport) are disabled accordingly depending on which one was set instead of being hidden. I've left all the fields on a single row except for the ip and port variants since it seemed to me that it made more sense that way given their correlation.

Release Note

The Ingest Node Pipelines UI added support to configure a community id processor. This processor computes the Community ID for network flow data as defined in the Community ID Specification.

Default description

Screenshot 2021-07-01 at 10 27 28

New processor fields

Screenshot 2021-07-19 at 09 10 54

@sabarasaba sabarasaba self-assigned this Jun 30, 2021
@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

merge conflict between base and head

@sabarasaba sabarasaba added Feature:Ingest Node Pipelines Ingest node pipelines management release_note:feature Makes this part of the condensed release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.15.0 v8.0.0 labels Jul 13, 2021
@sabarasaba sabarasaba marked this pull request as ready for review July 14, 2021 06:50
@sabarasaba sabarasaba requested a review from a team as a code owner July 14, 2021 06:50
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-stack-management (Team:Stack Management)

yuliacech
yuliacech reviewed Jul 15, 2021
View reviewed changes
Copy link
Contributor

@yuliacech yuliacech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sabarasaba,
thank you so much for working on this! Code changes LGTM, but I'd like to suggest getting a review from a docs writer, for example to check that id is spelled ID everywhere (following how it's spelled in the docs). Also I think it would be useful to add default values to the field's help text, same as you did for the target field.

sabarasaba and others added 5 commits July 16, 2021 09:19
@sabarasaba @yuliacech
Update x-pack/plugins/ingest_pipelines/public/application/components/…
40eecaa 
…pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
@sabarasaba @yuliacech
Update x-pack/plugins/ingest_pipelines/public/application/components/…
d9906cc 
…pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
@sabarasaba @yuliacech
Update x-pack/plugins/ingest_pipelines/public/application/components/…
64b70f9 
…pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
@sabarasaba @yuliacech
Update x-pack/plugins/ingest_pipelines/public/application/components/…
19d500f 
…pipeline_editor/components/processor_form/processors/community_id.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
@sabarasaba @yuliacech
Update x-pack/plugins/ingest_pipelines/public/application/components/…
0dabf68 
…pipeline_editor/components/processor_form/processors/community_id.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

jrodewig
jrodewig suggested changes Jul 16, 2021
View reviewed changes
Copy link
Contributor

@jrodewig jrodewig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sabarasaba. I left a few comments related to defaults, field name references, and the docs link. I'll take another after those are addressed.

...application/components/pipeline_editor/components/processor_form/processors/community_id.tsx Outdated Show resolved Hide resolved
...application/components/pipeline_editor/components/processor_form/processors/community_id.tsx Outdated Show resolved Hide resolved
...application/components/pipeline_editor/components/processor_form/processors/community_id.tsx Outdated Show resolved Hide resolved
...blic/application/components/pipeline_editor/components/shared/map_processor_type_to_form.tsx Show resolved Hide resolved
@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

jrodewig
jrodewig approved these changes Jul 19, 2021
View reviewed changes
Copy link
Contributor

@jrodewig jrodewig left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for docs review. Thanks @sabarasaba.

yuliacech
yuliacech approved these changes Jul 20, 2021
View reviewed changes
Copy link
Contributor

@yuliacech yuliacech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job, @sabarasaba!
I re-tested locally and everything looks great 👍 Thank you so much for addressing my comments in the initial review!

@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
ingestPipelines 495 496 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
ingestPipelines 695.2KB 706.0KB +10.8KB

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @sabarasaba

@sabarasaba sabarasaba merged commit d8238ca into elastic:master Jul 21, 2021
@sabarasaba sabarasaba mentioned this pull request Jul 21, 2021
Merged
sabarasaba added a commit to sabarasaba/kibana that referenced this pull request Jul 21, 2021
@sabarasaba @kibanamachine @yuliacech
[Ingest pipelines] add community id processor (elastic#103863)
2ec2b9b 
* wip: community_id processor

* fix up validation conditions for fields

* add tests

* Fix layout and tests

* Fix copy and validations

* Remove unused prop

* fix test matchers

* lowercase copy

* Convert hardcoded values to variables

* Remove extra dollar sign

* fix copy variable

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/processor_form/processors/community_id.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/processor_form/processors/community_id.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* No need to whitelist known fields

* CR changes

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
jloleysens added a commit to jloleysens/kibana that referenced this pull request Jul 21, 2021
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into reporting/onl…
5160a33 
…y-show-migrate-to-authzd-users

* 'master' of github.com:elastic/kibana: (48 commits)
  [Canvas] Expression shape (elastic#103219)
  [FTR] Skips Vega tests
  [Sample data] Use Lens in ecommerce data (elastic#106039)
  [APM] Backends inventory & overview page routes (elastic#106223)
  [TSVB] Add more functional tests for Gauge and TopN (elastic#105361)
  Add toggle to enable/disable rule install from SOs (elastic#106189)
  Improve unit test coverage of FS API calls (elastic#106242)
  Remove recursive plugin status in meta field (elastic#106286)
  [Ingest pipelines] add community id processor (elastic#103863)
  [XY axis] Fixes the values inside bar charts (elastic#106198)
  [data.search] Set default expiration to 1m if search sessions are disabled (elastic#105329)
  set the doc title when navigating to reporting and unset when navigating away (elastic#106253)
  [Lens] Display legend inside chart (elastic#105571)
  [RAC] [TGrid] Migrate the TGrid's rendering to `EuiDataGrid` (elastic#106199)
  [Security Solutions] Removes the elastic legacy client from lists and security_solution plugins (elastic#106130)
  [Enterprise Search] Require security plugin in 8.0 (elastic#106307)
  [DOCS] Updates screenshots in Dev Tools docs (elastic#105859)
  [DOCS] Updates text and screenshots in tags doc (elastic#105853)
  [Alerting] Allow rule types to extract/inject saved object references on rule CRU (elastic#101896)
  Jest and Storybook fixes (elastic#104991)
  ...

# Conflicts:
#	x-pack/plugins/reporting/public/plugin.ts
sabarasaba added a commit that referenced this pull request Jul 21, 2021
@sabarasaba @kibanamachine @yuliacech
[Ingest pipelines] add community id processor (#103863) (#106376)
a867d44 
* wip: community_id processor

* fix up validation conditions for fields

* add tests

* Fix layout and tests

* Fix copy and validations

* Remove unused prop

* fix test matchers

* lowercase copy

* Convert hardcoded values to variables

* Remove extra dollar sign

* fix copy variable

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/shared/map_processor_type_to_form.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/processor_form/processors/community_id.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* Update x-pack/plugins/ingest_pipelines/public/application/components/pipeline_editor/components/processor_form/processors/community_id.tsx

Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

* No need to whitelist known fields

* CR changes

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jrodewig jrodewig jrodewig approved these changes

@yuliacech yuliacech yuliacech approved these changes

Assignees

@sabarasaba sabarasaba

Labels
Feature:Ingest Node Pipelines Ingest node pipelines management release_note:feature Makes this part of the condensed release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.15.0 v8.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Ingest Node Pipelines] Add support for community_id processor
5 participants
@sabarasaba @kibanamachine @elasticmachine @yuliacech @jrodewig