Support more ways to define observables

[rmouritzen-splunk]

Support more ways to define observables

This issue proposes 2 changes:

• Add two new ways to define observables.
• Add a new observable type registry. Instead we are adding duplicate observable type_id detection to the ocsf-server and ocsf-validator.

New ways to define observables

Currently observables can be defined on types in the attribute dictionary and in object definitions. We want to add the capability to also define these observables on attributes in new ways:

1. Define observable on dictionary attributes.
2. Define observable on class and object attributes.
3. Define observables in class definitions by attribute paths. This allows class-specific definitions for attributes that could be nested in a sub-object or sub-array in a new section using a path-based syntax. For definitions that navigate to or through an array, the definition applies to each element of the array.

The class path-based definition will look like this (or something equivalent):

{
  "class": "foo",
  // ... other event class fields
  "observables" {
    "device.file.path": 42
  }
}

With these additions, it is possible that observable definitions could overlap. In cases where a specific observable matches more than one type_id, the most general one should be used. The following list ranks from most to least specific:

1. Observable by attribute's type: a scalar type (defined in types section of attribute dictionary) or object type (defined in the object's definition). Note that attribute scalar types and object types cannot collide.
2. Observable by object (for attributes who's type is an object).
3. Observable by dictionary attribute (defined in the attributes section of the attribute dictionary).
4. Observable by class or object attribute (defined for in a class or object's attributes property for specific attributes).
5. Observable by class-specific path (defined in class definition).

Observable registry

We also want to add a new observable type registry. A new observable-types.json will be supported to unify these definitions. Extensions can extend these by placing an observable-types.json in the extension directory, as with attributes extensions using an extension-specific dictionary.json. The OCSF Server (and any future tools that read the metaschema) will log problems like missing missing type_id values and colliding type_id values. The OCSF Server (and future tools) may optionally stop these sort of errors, though there should always be a way to continue on in case there metaschema is in a temporarily inconsistent state.

This idea doesn't help avoid observable type_id collisions. As mentioned above, we will instead detect collisions as a validation.