Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Azure] Add Microsoft Graph Activity Logs datastream #9314

Merged
merged 62 commits into from Mar 15, 2024
Merged

[Azure] Add Microsoft Graph Activity Logs datastream #9314

merged 62 commits into from Mar 15, 2024

Conversation

kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Mar 8, 2024 •
edited

Proposed commit message

  • Add Microsoft Graph Activity Logs datastream.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

elastic-package build && elastic-package stack up -d -v && eval "$(elastic-package stack shellinit)" && elastic-package test pipeline --generate -v

Related issues

Screenshots

graph_activity_logs

kcreddy added 30 commits June 7, 2022 19:47
@kcreddy
auth0: updated title
ff492e4
@kcreddy
Merge branch 'main' of https://github.com/kcreddy/integrations
17e28c9
@kcreddy
cisco_merki: updated title
0e9e1a9
@kcreddy
azure: updated title
0ef03b1
@kcreddy
cef: updated title
8718cc0
@kcreddy
cisco_secure_endpoint: updated title
36bf06d
@kcreddy
crowdStrike: updated title
67ca3a3
@kcreddy
httpjson: updated title
414262b
@kcreddy
cyberarkpas: updated title
a519be2
@kcreddy
Fireeye: updated title
7bc4125
@kcreddy
Fortinet: updated title
179b373
@kcreddy
santa: updated title
ceae4c3
@kcreddy
google_workspace: updated title
7261205
@kcreddy
iptables: updated title
b45b6f9
@kcreddy
m365_defender: updated title
47259a8
@kcreddy
o365: updated title
73a1d9c
@kcreddy
okta: updated title
9be35b3
@kcreddy
panw_cortex_xdr: updated title
104cd81
@kcreddy
panw: updated title
2dc7e3e
@kcreddy
pfSense: updated title
7ca05cd
@kcreddy
sophos: updated title
62918d1
@kcreddy
suricata: updated title
446b2e9
@kcreddy
zeek: updated title
78e4c1e
@kcreddy
Merge remote-tracking branch 'upstream/main'
130713a
@kcreddy
update description to align with standard wording
051e840
@kcreddy
Merge remote-tracking branch 'upstream/main'
392e0c0
@kcreddy
change version
6367d9e
@kcreddy
change from minor to patch
e13887f
@kcreddy
Merge remote-tracking branch 'upstream/main'
686cb32
@kcreddy
fix version
9864132
@kcreddy kcreddy self-assigned this Mar 8, 2024
@kcreddy kcreddy added Integration:Azure enhancement New feature or request Team:Security-Service Integrations Security Service Integrations Team labels Mar 8, 2024
@elasticmachine
Copy link

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@kcreddy kcreddy marked this pull request as ready for review March 8, 2024 14:16
@kcreddy kcreddy requested review from a team as code owners March 8, 2024 14:16
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@jamiehynds
Copy link

jamiehynds commented Mar 8, 2024 •
edited

fyi @aarju - any feedback you have around dashboards, docs, mappings, etc very welcome :)

@aarju
Copy link

aarju commented Mar 13, 2024

fyi @aarju - any feedback you have around dashboards, docs, mappings, etc very welcome :)

@jamiehynds I don't have any feedback at this time, but I'm looking forward to testing out this integration and I may have some feedback after using it with some live data.

efd6
efd6 reviewed Mar 14, 2024
View reviewed changes
packages/azure/data_stream/graphactivitylogs/agent/stream/azure-eventhub.yml.hbs Outdated Show resolved Hide resolved
packages/azure/data_stream/graphactivitylogs/agent/stream/azure-eventhub.yml.hbs Outdated Show resolved Hide resolved
packages/azure/data_stream/graphactivitylogs/agent/stream/azure-eventhub.yml.hbs Outdated Show resolved Hide resolved
packages/azure/data_stream/graphactivitylogs/agent/stream/log.yml.hbs Outdated Show resolved Hide resolved
packages/azure/data_stream/graphactivitylogs/fields/fields.yml Outdated Show resolved Hide resolved
.../azure/data_stream/graphactivitylogs/elasticsearch/ingest_pipeline/azure-shared-pipeline.yml Outdated Show resolved Hide resolved
packages/azure/data_stream/graphactivitylogs/elasticsearch/ingest_pipeline/default.yml Show resolved Hide resolved
packages/azure/data_stream/graphactivitylogs/elasticsearch/ingest_pipeline/default.yml Outdated
Comment on lines 22 to 26
source: ctx.message = ctx.message.replace(params.empty_field_name, '')
params:
empty_field_name: '"":"",'
ignore_failure: true
tag: script-message-emptyfields
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a surprising order of fields.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is. I took a base ingest pipeline template and modified for this usecase.
This processor is present in most of the package's datastreams. I wonder if its even required here. Might as well remove it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this processor in the new commit.

packages/azure/data_stream/graphactivitylogs/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved
packages/azure/data_stream/graphactivitylogs/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved
@elasticmachine
Copy link

💚 Build Succeeded

History

cc @kcreddy

@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate passed Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
89.0% 89.0% Coverage on New Code
0.0% 0.0% Duplication on New Code

See analysis details on SonarQube

@kcreddy kcreddy requested a review from efd6 March 14, 2024 07:57
@kcreddy kcreddy merged commit 6d5ef11 into elastic:main Mar 15, 2024
5 checks passed
@elasticmachine
Copy link

Package azure - 1.10.0 containing this change is available at https://epr.elastic.co/search?package=azure

@kcreddy
Copy link
Contributor Author

kcreddy commented Mar 15, 2024

@jamiehynds I don't have any feedback at this time, but I'm looking forward to testing out this integration and I may have some feedback after using it with some live data.

@aarju This feature is now available. Please feel free to test and provide feedback. Thanks 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@efd6 efd6 efd6 approved these changes

Assignees

@kcreddy kcreddy

Labels
enhancement New feature or request Integration:Azure Team:Security-Service Integrations Security Service Integrations Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[New Data] Azure Graph Activity Logs
5 participants
@kcreddy @elasticmachine @jamiehynds @aarju @efd6