Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[O365]Moving edge processing to ingest pipelines #983

Merged
merged 12 commits into from
Jun 9, 2021

Conversation

P1llus
Copy link
Member

@P1llus P1llus commented May 11, 2021

What does this PR do?

This PR removes all edge processing in favor of ingest pipelines

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.

Related issues

@elasticmachine
Copy link

elasticmachine commented May 11, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #983 updated

  • Start Time: 2021-06-09T07:55:07.088+0000

  • Duration: 17 min 23 sec

  • Commit: 663d9fb

Test stats 🧪

Test Results
Failed 0
Passed 25
Skipped 0
Total 25

Trends 🧪

Image of Build Times

Image of Tests

@P1llus P1llus requested a review from adriansr May 25, 2021 11:25
@P1llus P1llus marked this pull request as ready for review May 25, 2021 11:28
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

copy_from: o365audit.ObjectId
if: ctx.event?.code == "AzureActiveDirectory"
## AzureActiveDirectory Schema new user
- set:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a suggestion, I'm OK merging as is.

It looks likeevent.code and event.action are used several times to set ECS categorization. I'm wondering if we could generalize to a script that takes parameters. Get all the logic to make those decisions in one place.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback @leehinman. There is an ongoing discussion on exactly this, also for google_workspace, around things like performance and which way would be better.

If its okay with you I would like to do the same here, in which I leave it like this for now, as I feel it is better for performance, while keeping an eye on the overall discussion. If it turns out that there is no difference or the other way does not have any high impact on benchmarks I will go back and implement it on both.

@P1llus
Copy link
Member Author

P1llus commented Jun 1, 2021

run tests

@marc-gr
Copy link
Contributor

marc-gr commented Jun 8, 2021

/test

@marc-gr marc-gr mentioned this pull request Jun 9, 2021
43 tasks
@marc-gr marc-gr merged commit 8f67eb2 into elastic:master Jun 9, 2021
james-elastic pushed a commit to james-elastic/integrations that referenced this pull request Jun 30, 2021
* adding pipeline tests

* stashing changes

* stashing changes

* stashing changes again

* stashing changes, need to pipe config tenant objects to pipeline still

* first finalized version, ready for review

* finalized version ready for review

* update changelog and manifest

* regenerating test files and merging with master

* Fix config and test files

Co-authored-by: Marc Guasch <marc.guasch@elastic.co>
eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022
* adding pipeline tests

* stashing changes

* stashing changes

* stashing changes again

* stashing changes, need to pipe config tenant objects to pipeline still

* first finalized version, ready for review

* finalized version ready for review

* update changelog and manifest

* regenerating test files and merging with master

* Fix config and test files

Co-authored-by: Marc Guasch <marc.guasch@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Convert o365's edge processing to Ingest Node pipeline
4 participants