Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Provider_name and Data[x] notation to the field mapping #1350

Closed
fukusuket opened this issue May 19, 2024 · 4 comments · Fixed by #1352
Closed

Support for Provider_name and Data[x] notation to the field mapping #1350

fukusuket opened this issue May 19, 2024 · 4 comments · Fixed by #1352
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@fukusuket
Copy link
Collaborator

fukusuket commented May 19, 2024

Currently, the field mapping feature does not support following point:

  • Provider_name matching
    • e.g. Since the Application log has multiple Provider_names, we need to convert the field value only when the Provider_name matches.(in addition to Channel, EventID)
  • Data[x] notation filed mapping

We want to support following rule/field conversion, so I'll implement above point.

@fukusuket fukusuket added the enhancement New feature or request label May 19, 2024
@fukusuket fukusuket added this to the v2.16.0 milestone May 19, 2024
@fukusuket fukusuket self-assigned this May 19, 2024
@fukusuket fukusuket changed the title Support for Provider_name and Data[x] notation to the field mapping function Support for Provider_name and Data[x] notation to the field mapping May 19, 2024
@fukusuket
Copy link
Collaborator Author

fukusuket commented May 20, 2024

Specification memo:

  • If possible, make Provider_Name optional
  • If possible, allow multiple values ​​to be set for Provider_Name(It is preferable to have one yaml file)
  • Do not output the value before field value conversion to ExtraFieldInfo (csv/json)

@fukusuket
Copy link
Collaborator Author

  • Do not output the value before field value conversion to ExtraFieldInfo (csv/json)

I have looked into the feasibility of implementation and it does not appear to be easy to achieve 🤔
The main reasons are the following two points:

  • Under the Data field is an array (not simple key value)
  • The exclusion process for Details and ExtraFieldInfo is a bit complicated

@YamatoSecurity
I think I'm going to create the issue of output to ExtraFieldInfo as a separate issue and hold off on implementation, what do you think?

@YamatoSecurity
Copy link
Collaborator

@fukusuket I see, sure, we can hold off on the implementation for now. I'm thinking it might be better to save the Data fields as different fields instead of in an array for JSON. For example: Data-1, Data-2, etc... This way it will be easier to deal with when importing records into elastic stack, etc...

@fukusuket
Copy link
Collaborator Author

fukusuket commented May 23, 2024

@YamatoSecurity

Do not output the value before field value conversion to ExtraFieldInfo (csv/json)

Sorry many times, I looked into it more and found how to implement this just now(though my current implementation is even more complicated...😇). I'll create PR.

I see, It would be nice if eliminating Array would make it easier to integrate with things like ElasticSearch :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment