[OAS] Support lazy runtime types #184000

jloleysens · 2024-05-22T10:20:46Z

Summary

Add the ability to declare recursive schemas. Updates @kbn/config-schema to support recursive types. This design follows the underlying pattern provided by Joi: https://joi.dev/api/?v=17.13.0#linkref:

      const id = 'recursive';
      const recursiveSchema: Type<RecursiveType> = schema.object(
        {
          name: schema.string(),
          self: schema.lazy<RecursiveType>(id),
        },
        { meta: { id } }
      );

Since using the .link API we are also using .id which enables us to leverage this mechanism OOTB with joi-to-json for OAS generation (thus we could delete a lot of code there).

I chose to avoid using id originally because I thought it would be simpler if we control more of the conversion in config-schema's case but for recursive schemas and references I think this is a favourable trade off. Open to other ideas!

jloleysens · 2024-05-22T10:21:28Z

/ci

jloleysens · 2024-05-22T10:22:36Z

packages/kbn-config-schema/src/types/lazy.test.ts

+  const id = 'recursive';
+  const object: Type<RecursiveType> = schema.object(
+    {
+      name: schema.string(),
+      self: lazy<RecursiveType>(id),
+    },
+    { meta: { id } }
+  );


It's a bit verbose, I would have liked to do something like Zod but with the way Joi.link works this just seemed simplest.

packages/kbn-config-schema/src/types/type.ts

…andling, mostly deleting code

jloleysens · 2024-05-22T12:10:36Z

/ci

jloleysens · 2024-05-22T12:14:56Z

packages/kbn-config-schema/src/types/object_type.ts

      .object()
      .keys(schemaKeys)
      .default()
      .optional()
      .unknown(unknowns === 'allow')
      .options({ stripUnknown: { objects: unknowns === 'ignore' } });

+    if (options.meta?.id) {
+      schema = schema.id(options.meta.id);


Note:

To keep things simple, id is only supported for object types, for now

Since using the joi's built-in .id() mechanism we can actually delete quite a bit of code from the downstream OAS generation because joi-to-json understands it and correctly extracts references for us

…fix'

jloleysens · 2024-05-22T12:21:01Z

/ci

jloleysens · 2024-05-22T12:22:11Z

packages/kbn-router-to-openapispec/src/__snapshots__/generate_oas.test.ts.snap

Added a test/snapshot, but mostly I was able to delete code from kbn-router-to-openapispec

jloleysens · 2024-05-22T12:23:05Z

...er-to-openapispec/src/oas_converter/kbn_config_schema/post_process_mutations/context.test.ts

-it('can convert and record refs', () => {
-  const ctx = createCtx({ refs: true });
-  const obj = schema.object({}, { meta: { id: 'foo' } });
-  const parsed = joi2JsonInternal(obj.getSchema());
-  const result = ctx.processRef(parsed);
-  expect(result).toEqual({ $ref: '#/components/schemas/foo' });
-  expect(ctx.sharedSchemas.get('foo')).toMatchObject({ type: 'object', properties: {} });
-  expect(metaFields.META_FIELD_X_OAS_REF_ID in ctx.sharedSchemas.get('foo')!).toBe(false);
-});


This is handled by joi-to-json now.

elasticmachine · 2024-05-22T13:46:03Z

Pinging @elastic/kibana-core (Team:Core)

pgayvallet

LGTM, great to finally have recursive types/schemas being supported by config-schema!

packages/kbn-router-to-openapispec/src/oas_converter/kbn_config_schema/parse.ts

pgayvallet · 2024-05-22T14:52:11Z

packages/kbn-config-schema/src/types/lazy.test.ts

+
+describe('lazy', () => {
+  const id = 'recursive';
+  const object: Type<RecursiveType> = schema.object(


As discussed: the explicit Type<RecursiveType> typing can be removed 🚀

kibana-ci · 2024-05-23T07:50:39Z

💚 Build Succeeded

Buildkite Build
Commit: acb66f1

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`enterpriseSearch`	2320	2321	+1
`serverlessSearch`	322	323	+1
`synthetics`	978	979	+1
total			+3

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/config-schema`	139	141	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`enterpriseSearch`	2.7MB	2.7MB	-39.0B
`serverlessSearch`	474.1KB	474.1KB	-39.0B
`synthetics`	1.0MB	1.0MB	+104.0B
total			+26.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`@kbn/config-schema`	19	20	+1

Unknown metric groups

API count

id	before	after	diff
`@kbn/config-schema`	142	144	+2

History

💛 Build #211538 was flaky a0a3885
💚 Build #211500 succeeded e681cd0
💔 Build #211466 failed 843e989

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jloleysens

## Summary Per the title. Related #184000

## Summary Close elastic#182910 Add the ability to declare recursive schemas. Updates `@kbn/config-schema` to support recursive types. This design follows the underlying pattern provided by Joi: https://joi.dev/api/?v=17.13.0#linkref: ```ts const id = 'recursive'; const recursiveSchema: Type<RecursiveType> = schema.object( { name: schema.string(), self: schema.lazy<RecursiveType>(id), }, { meta: { id } } ); ``` Since using the `.link` API we are also using `.id` which enables us to leverage this mechanism OOTB with `joi-to-json` for OAS generation (thus we could delete a lot of code there). I chose to avoid using `id` originally because I thought it would be simpler if we control more of the conversion in config-schema's case but for recursive schemas and references I think this is a favourable trade off. Open to other ideas! --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## Summary Per the title. Related elastic#184000

support lazy runtime types

843e989

jloleysens self-assigned this May 22, 2024

jloleysens added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc release_note:skip Skip the PR/issue when compiling release notes v8.15.0 Feature:OAS Work or issues related to Core-provided mechanisms for generating OAS labels May 22, 2024

jloleysens commented May 22, 2024

View reviewed changes

jloleysens added 6 commits May 22, 2024 14:05

refactor to use built in Joi ID

1f2c7c3

fix use of IDs

3998f93

large ish refactor to rather use the built in joi-to-json reference h…

f09817b

…andling, mostly deleting code

refactored tests after changes

5e6cc11

added parser test

7b3abff

updated unit test after removing ref ID from meta

d7d8a80

comments

41fe990

jloleysens commented May 22, 2024

View reviewed changes

kibanamachine and others added 2 commits May 22, 2024 12:17

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

6510265

…fix'

added OAS test

e681cd0

jloleysens commented May 22, 2024

View reviewed changes

jloleysens marked this pull request as ready for review May 22, 2024 13:46

jloleysens requested a review from a team as a code owner May 22, 2024 13:46

jloleysens added 2 commits May 22, 2024 16:14

do not export lazy function directly

beea249

fix other use of "lazy"

a0a3885

pgayvallet approved these changes May 22, 2024

View reviewed changes

jloleysens added 2 commits May 22, 2024 17:39

remove explicit typing

b42e0e7

address nit

acb66f1

jloleysens enabled auto-merge (squash) May 23, 2024 06:40

jloleysens merged commit 1cc878f into elastic:main May 23, 2024
17 checks passed

kibanamachine added the backport:skip This commit does not require backporting label May 23, 2024

jloleysens mentioned this pull request May 24, 2024

[config-schema] Add docs for new lazy type #184124

Merged

jloleysens deleted the kbn-config/support-recursive-types branch May 24, 2024 11:40

jloleysens added a commit that referenced this pull request May 27, 2024

[config-schema] Add docs for new lazy type (#184124)

099e79a

## Summary Per the title. Related #184000

rshen91 pushed a commit to rshen91/kibana that referenced this pull request May 30, 2024

[config-schema] Add docs for new lazy type (elastic#184124)

0e4fd8b

## Summary Per the title. Related elastic#184000

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[OAS] Support lazy runtime types #184000

[OAS] Support lazy runtime types #184000

jloleysens commented May 22, 2024 •

edited

Loading

jloleysens commented May 22, 2024

jloleysens May 22, 2024

jloleysens commented May 22, 2024

jloleysens May 22, 2024 •

edited

Loading

jloleysens commented May 22, 2024

jloleysens May 22, 2024

jloleysens May 22, 2024

elasticmachine commented May 22, 2024

pgayvallet left a comment

pgayvallet May 22, 2024

kibana-ci commented May 23, 2024

API count

[OAS] Support lazy runtime types #184000

[OAS] Support lazy runtime types #184000

Conversation

jloleysens commented May 22, 2024 • edited Loading

Summary

jloleysens commented May 22, 2024

jloleysens May 22, 2024

Choose a reason for hiding this comment

jloleysens commented May 22, 2024

jloleysens May 22, 2024 • edited Loading

Choose a reason for hiding this comment

jloleysens commented May 22, 2024

jloleysens May 22, 2024

Choose a reason for hiding this comment

jloleysens May 22, 2024

Choose a reason for hiding this comment

elasticmachine commented May 22, 2024

pgayvallet left a comment

Choose a reason for hiding this comment

pgayvallet May 22, 2024

Choose a reason for hiding this comment

kibana-ci commented May 23, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

API count

History

jloleysens commented May 22, 2024 •

edited

Loading

jloleysens May 22, 2024 •

edited

Loading