Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Data Security Finding event class & Data Security object #953

Merged
merged 22 commits into from
Feb 24, 2024

Conversation

jonrau-at-queryai
Copy link
Contributor

Related Issue:

NaN

Description of changes:

BLUF: Add a new Finding category event class: Data Security Finding, add a new object: Data Security to support normalization and standardization of Data Loss Prevention (DLP), Data Security Posture Management (DSPM), and similar findings

Data Security Finding was modeled on Detection Finding and Datastore Activity events, appropriate objects from each were pulled into the finding along with other top-level objects to support a wide amount of normalization. Several objects were added at the top level such as databucket, database, table, file, actor and others to account for the different places that a finding can trigger from and keep track of who owns the datastores or who triggered the finding.

The Data Security object will contain information about the relevant policy(s), the lifecycle of the data (rest/use/transit), the type of tool, the type of detection technique, as well as string values to include the name of the specific detections (or the regex) and the source of the pattern match.

Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
@jonrau-at-queryai jonrau-at-queryai marked this pull request as ready for review February 2, 2024 14:57
@jonrau-at-queryai
Copy link
Contributor Author

Based on feedback from @zschmerber will make the following changes

  • Add confidentiality to data_security
  • Expand enums in confidentiality to include: Private, Restricted, Compartmentalized, Special Access
  • Change data_classification_category_uid and data_classification_category to data_type_uid and data_type, respectively.

@jonrau-at-queryai jonrau-at-queryai marked this pull request as draft February 2, 2024 17:42
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
@jonrau-at-queryai jonrau-at-queryai marked this pull request as ready for review February 5, 2024 13:49
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
@zschmerber
Copy link
Contributor

zschmerber commented Feb 5, 2024

I am wondering if the file object in this class belongs in here?: https://schema.ocsf.io/1.1.0/objects/evidences?extensions=

I'm not sure you need the evidence object but that is how Detections finding has it so I thought I would pose the question.

@jonrau-at-queryai
Copy link
Contributor Author

I am wondering if the file object in this class belongs in here?: https://schema.ocsf.io/1.1.0/objects/evidences?extensions=

I'm not sure you need the evidence object but that is how Detections finding has it so I thought I would pose the question.

In my initial design that is how I had it, but my thoughts to not use it are twofold:

  • Easier searches: If the finding only ever is 1:1 with certain objects, such as src_endpoint and actor.user it can be easier to search against that as a semi-structured object in upstream sources like Amazon Athena or Snowflake, no need to using CROSS JOIN UNNEST or similar computationally hungry operators
  • Readability: There are other objects such as databucket and database at the base_event / "top-level" of this finding and not under evidence

Of course, there can be a multitude of data sources implicated by a specific finding too. I guess in that case, when it comes to ETL/ELT systems, is the argument that it is easier to add multiple types of evidence or is it easier to create multiple events and use metadata.correlation_uid to have them all tied together under the same source event.

In the case of multiple sources implicated, no issue adding evidence in favor of actor, src_endpoint, dst_endpoint, et al but I'll need to update it to add databucket, database, table and a few other objects.

I guess this is another case of balancing the data engineering effort (mapping) against the data analysis effort (search, detections, etc.)

@pagbabian-splunk
Copy link
Contributor

This looks really good overall - a couple of things though: enums should be _id not _uid: there are a few that need to be tweaked. Also, some of the new enums are fully defined in the dictionary, and then it seems overloaded exactly the same in the class and object. If the values are unlikely to be overridden, they can be fully defined in the dictionary - otherwise only the 0-Unknown and 99-Other as well as the sibling need be defined in the dictionary, and the class/object can extend or override.

Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
@jonrau-at-queryai
Copy link
Contributor Author

This looks really good overall - a couple of things though: enums should be _id not _uid: there are a few that need to be tweaked. Also, some of the new enums are fully defined in the dictionary, and then it seems overloaded exactly the same in the class and object. If the values are unlikely to be overridden, they can be fully defined in the dictionary - otherwise only the 0-Unknown and 99-Other as well as the sibling need be defined in the dictionary, and the class/object can extend or override.

Thanks @pagbabian-splunk , just made the required changes. Forgot that the objects and event would inherit from dictionary.json

@floydtree floydtree added enhancement New feature or request findings Issues related to Findings Category v1.2.0 Changes marked for version v1.2.0 of OCSF labels Feb 6, 2024
@floydtree
Copy link
Contributor

@jonrau-at-queryai One quick note, can you add relevant entries in the unreleased section in CHANGELOG.md file? You can refer to the previous entries for reference.

Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
CHANGELOG.md Outdated Show resolved Hide resolved
@jonrau-at-queryai
Copy link
Contributor Author

@jonrau-at-queryai One quick note, can you add relevant entries in the unreleased section in CHANGELOG.md file? You can refer to the previous entries for reference.

Done!

Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
dictionary.json Outdated Show resolved Hide resolved
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Update `pattern_match` in dictionary and data security object
@jonrau-at-queryai
Copy link
Contributor Author

@floydtree @mikeradka put in both changes you folks wanted

CHANGELOG.md Outdated Show resolved Hide resolved
dictionary.json Outdated Show resolved Hide resolved
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
`analytic` update for `data_security` object
objects/data_security.json Outdated Show resolved Hide resolved
…lytic`

Signed-off-by: Jonathan Rau <139361268+jonrau-at-queryai@users.noreply.github.com>
Copy link
Contributor

@floydtree floydtree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! This is great.

Copy link
Contributor

@zschmerber zschmerber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

Copy link
Contributor

@Aniak5 Aniak5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@floydtree floydtree dismissed mikeradka’s stale review February 24, 2024 19:56

comments are addressed

@floydtree floydtree merged commit c86a51a into ocsf:main Feb 24, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request findings Issues related to Findings Category v1.2.0 Changes marked for version v1.2.0 of OCSF
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants