Add optional schema property to graph object metadata

[ndowmon]

Added

• Added optional schema property to StepGraphObjectMetadata. This allows
  developers to provide the property schema to expect on entities,
  relationships, and mapped relationships. This serves two uses:
  1. Schemas can be used at runtime or test-time to verify that an entity has
     the correct properties
  2. The j1-integration document command could automatically produce consumer
     documentation about the properties that an entity / relationship is
     expected to have

[austinkelleher]

Awesome! Would you mind adding this to the development documentation please? I'd also love to see a task created to update the integration-template!

[austinkelleher]

Would love to see a ticket in the SDK describing this!

[ndowmon]

#612

[ndowmon]

Awesome! Would you mind adding this to the development documentation please? I'd also love to see a task created to update the integration-template!

I'd like to make some minor updates to toMatchDirectRelationshipSchema, toMatchGraphObjectSchema, and add the toMatchStepMetadata matcher we discussed previously. Then go back and update docs/testing and integration-template

[ndowmon]

Do we want to make some of these required? It could prevent us from bugs like what we saw in graph-aws

[ndowmon]

e.g. properties, required, additionalProperties

[ndowmon]

export interface GraphObjectSchema extends IntegrationEntitySchema {
  $schema?: string;
  $id?: string;
  description?: string;
- additionalProperties?: boolean;
+ additionalProperties: boolean;
+ properties: Required<IntegrationEntitySchema>['properties'];
+ required: Required<IntegrationEntitySchema>['required'];
}

[aiwilliams]

Considering the form of https://github.com/JupiterOne/data-model/blob/178d8838d6004a45ff97b229fd8a0dafd3b50e6f/src/index.ts#L6:

export type IntegrationEntitySchema = {
  $ref?: string;
  allOf?: IntegrationEntitySchema[];
  properties?: {
    [propertyName: string]: any;
  };
  required?: string[];
};

Nothing is required there, which makes sense, allowing for various combinations of those properties.

For this GraphObjectSchema, I think we could make description required. I'm confused about what $schema does here in the context of the metadata for an entity. Will $id ever be provided, perhaps we could remove that? We could probably use some documentation on these properties.

What do you think about naming this EntitySchemaExtension?

[ndowmon]

TBH I think it's bad that we're not using the published NPM types for JSON Schema.

yarn add --dev @types/json-schema

import { JSONSchema7 } from 'json-schema'

Perhaps this type should actually be something like the following?

type GraphObjectSchema = Required<Pick<JSONSchema7, 'additionalProperties' | 'required' | 'properties'>>;

[ndowmon]

Personally I don't see why we would want to document a description if we're just using this to describe step entity metadata.

[ndowmon]

Also I don't love EntitySchemaExtension because I would really like to see this used on relationships as well, in cases where relationships will have properties on them (example, permissions relationships or firewall rule relationships)

[aiwilliams]

Ohh, nice find on the @types/json-schema! I wonder if I totally overlooked that or if it's relatively new. I think it would be wise to move to use those types.

[ndowmon]

Tracking:

• Use @types/json-schema for GraphObjectSchema #620
• Use @types/json-schema IntegrationEntitySchema data-model#147

[aiwilliams]

Considering the form of https://github.com/JupiterOne/data-model/blob/178d8838d6004a45ff97b229fd8a0dafd3b50e6f/src/index.ts#L6:

export type IntegrationEntitySchema = {
  $ref?: string;
  allOf?: IntegrationEntitySchema[];
  properties?: {
    [propertyName: string]: any;
  };
  required?: string[];
};

Nothing is required there, which makes sense, allowing for various combinations of those properties.

For this GraphObjectSchema, I think we could make description required. I'm confused about what $schema does here in the context of the metadata for an entity. Will $id ever be provided, perhaps we could remove that? We could probably use some documentation on these properties.

What do you think about naming this EntitySchemaExtension?