Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lyve cloud integration #4702

Merged
merged 18 commits into from
Dec 15, 2022
Merged

Lyve cloud integration #4702

merged 18 commits into from
Dec 15, 2022

Conversation

leon-seagate
Copy link
Contributor

  • Enhancement

What does this PR do?

  • Added new directory under packages for new cloud provider - a.k.a Seagate's Lyve Cloud.
  • Added data collection logic for the data streams.
  • Added the ingest pipeline for the data streams.
  • Mapped fields according to the ECS schema and added Fields metadata in the appropriate yaml files.
  • Added dashboards and visualizations.
  • Added system test cases for the data stream.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Screenshots

image

@leon-seagate
added lyve cloud plugin
e3e9b37 
Signed-off-by: leon-seagate <leon.markovich@seagate.com>
@leon-seagate
readme changes
b04ccde 
Signed-off-by: leon-seagate <leon.markovich@seagate.com>
@cla-checker-service
Copy link

cla-checker-service bot commented Nov 23, 2022 •
edited
Loading

💚 CLA has been signed

@elasticmachine
Copy link

elasticmachine commented Nov 23, 2022 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-12-15T14:20:34.956+0000

  • Duration: 14 min 31 sec

Test stats 🧪

Test Results
Failed 0
Passed 5
Skipped 0
Total 5

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@leon-seagate
Copy link
Contributor Author

Hey @marc-gr @jamiehynds, can you please verify CLA check ? .

@gtback
Copy link
Member

gtback commented Nov 29, 2022

cla/check

@marc-gr
Copy link
Contributor

marc-gr commented Nov 29, 2022

/test

@marc-gr marc-gr self-requested a review November 29, 2022 14:55
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@marc-gr
Copy link
Contributor

marc-gr commented Nov 30, 2022

A new CODEOWNERS entry is required (https://github.com/elastic/integrations/blob/main/.github/CODEOWNERS#L115) for the package such as /packages/lyve_cloud @elastic/security-external-integrations

marc-gr
marc-gr requested changes Nov 30, 2022
View reviewed changes
Copy link
Contributor

@marc-gr marc-gr left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Looking good 👍 , left a couple comments.

packages/lyve_cloud/_dev/build/build.yml Outdated Show resolved Hide resolved
packages/lyve_cloud/_dev/build/docs/README.md Outdated Show resolved Hide resolved
packages/lyve_cloud/data_stream/audit/elasticsearch/ingest_pipeline/audit_lc.yml Outdated Show resolved Hide resolved
packages/lyve_cloud/_dev/build/docs/README.md Outdated Show resolved Hide resolved
@leon-seagate
changing readme after ecs requestes
9e820f3 
Signed-off-by: leon-seagate <leon.markovich@seagate.com>
@leon-seagate
added sample for readne and changed remove proccesors to one list
a9b42ac 
Signed-off-by: leon-seagate <leon.markovich@seagate.com>
@leon-seagate
added exported fileds, readme notes, set remove proccesor as an arra…
6dc277f 
…y for all remove files

Signed-off-by: leon-seagate <leon.markovich@seagate.com>
leon-seagate
leon-seagate commented Dec 5, 2022
View reviewed changes
Copy link
Contributor Author

@leon-seagate leon-seagate left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marc-gr Hi, I've changed your requests :)

packages/lyve_cloud/_dev/build/docs/README.md Outdated Show resolved Hide resolved
packages/lyve_cloud/_dev/build/docs/README.md Outdated Show resolved Hide resolved
@marc-gr
Copy link
Contributor

marc-gr commented Dec 12, 2022

/test

marc-gr
marc-gr requested changes Dec 12, 2022
View reviewed changes
Copy link
Contributor

@marc-gr marc-gr left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some formatting nits and a couple more things, I think after these changes and some README work adding more details (following https://github.com/elastic/integrations/pull/3433/files#diff-827a4446b988cc0aca46047deee0e42fd15c9383ced59cbe3a1e5f496685b07c) we are good to go 👍

packages/lyve_cloud/data_stream/audit/elasticsearch/ingest_pipeline/audit_lc.yml Outdated Show resolved Hide resolved
packages/lyve_cloud/data_stream/audit/elasticsearch/ingest_pipeline/audit_lc.yml Outdated Show resolved Hide resolved
packages/lyve_cloud/data_stream/audit/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved
packages/lyve_cloud/_dev/build/docs/README.md Outdated Show resolved Hide resolved
packages/lyve_cloud/_dev/build/docs/README.md Outdated Show resolved Hide resolved
packages/lyve_cloud/data_stream/audit/_dev/test/pipeline/test-audit-events.json-expected.json Outdated Show resolved Hide resolved
packages/lyve_cloud/data_stream/audit/_dev/test/pipeline/test-audit-events.json-expected.json Outdated Show resolved Hide resolved
packages/lyve_cloud/data_stream/audit/_dev/test/pipeline/test-audit-events.json-expected.json Outdated Show resolved Hide resolved
packages/lyve_cloud/_dev/build/docs/README.md Outdated Show resolved Hide resolved
packages/lyve_cloud/kibana/dashboard/lyve_cloud-42d12a00-5852-11ed-a702-0d7be6ba8b3f.json Outdated Show resolved Hide resolved
@marc-gr
Copy link
Contributor

marc-gr commented Dec 12, 2022

PS: still need to add the codeowners entry as mentioned in #4702 (comment)

leon-seagate and others added 8 commits December 12, 2022 12:52
@leon-seagate @marc-gr
Update packages/lyve_cloud/data_stream/audit/elasticsearch/ingest_pip…
a73d931 
…eline/audit_lc.yml


cleaner remove proccesor

Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com>
@leon-seagate @marc-gr
Update packages/lyve_cloud/data_stream/audit/elasticsearch/ingest_pip…
d5891b9 
…eline/audit_lc.yml


cleaner readme

Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com>
@leon-seagate @marc-gr
Update packages/lyve_cloud/data_stream/audit/elasticsearch/ingest_pip…
40e8440 
…eline/audit_lc.yml

Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com>
@leon-seagate @marc-gr
Update packages/lyve_cloud/data_stream/audit/elasticsearch/ingest_pip…
f38ed36 
…eline/audit_lc.yml

Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com>
@leon-seagate @marc-gr
Apply suggestions from code review
d910902 
embed the query filter with the dashboard and upgraded version number

Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com>
@leon-seagate
add related.user and removed names from events with changed to provid…
0307a59 
…er format

Signed-off-by: root <leon.markovich@seagate.com>
@leon-seagate
added codeowners, changed empty braces to full in examples, added rel…
c1bb462 
…ated fields, fixed readme after jamies notes

Signed-off-by: root <leon.markovich@seagate.com>
@leon-seagate
added geoip for client
b6c7536 
Signed-off-by: root <leon.markovich@seagate.com>
@marc-gr
Copy link
Contributor

marc-gr commented Dec 14, 2022

/test

@leon-seagate
removed the esc.geo
ff23e95 
Signed-off-by: root <leon.markovich@seagate.com>
@marc-gr
Copy link
Contributor

marc-gr commented Dec 15, 2022

/test

1 similar comment
@marc-gr
Copy link
Contributor

marc-gr commented Dec 15, 2022

/test

@elasticmachine
Copy link

elasticmachine commented Dec 15, 2022 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (1/1) 💚
Files 100.0% (2/2) 💚
Classes 100.0% (2/2) 💚
Methods 86.364% (19/22) 👎 -12.187
Lines 87.556% (197/225) 👎 -6.783
Conditionals 100.0% (0/0) 💚

@marc-gr
Copy link
Contributor

marc-gr commented Dec 15, 2022

/test

@marc-gr
Copy link
Contributor

marc-gr commented Dec 15, 2022

/test

@marc-gr
Copy link
Contributor

marc-gr commented Dec 15, 2022

/test

@marc-gr
Copy link
Contributor

marc-gr commented Dec 15, 2022

/test

@marc-gr marc-gr merged commit 6ed21fe into elastic:main Dec 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marc-gr marc-gr marc-gr approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@leon-seagate @elasticmachine @gtback @marc-gr