[Security Solution] OpenAPI docs bundler #171526
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maximpn
added
release_note:skip
maximpn
force-pushed
the
openapi-bundler
branch
8 times, most recently
from
97bc7ab
to
29bb05d
Compare
maximpn
force-pushed
the
openapi-bundler
branch
2 times, most recently
from
6037a09
to
d5d7094
Compare
maximpn
changed the title
maximpn
force-pushed
the
openapi-bundler
branch
2 times, most recently
from
3cf55ec
to
e06785d
Compare
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
maximpn
force-pushed
the
openapi-bundler
branch
from
8a24b10
to
5d0228a
Compare
maximpn
force-pushed
the
openapi-bundler
branch
from
5448bc7
to
5e7930d
Compare
maximpn
force-pushed
the
openapi-bundler
branch
from
ebbbf73
to
482bab3
Compare
banderror
reviewed
Nov 28, 2023
Comment on lines
+105
to
+134
| And the target spec will look like | ||
|
|
||
| ```yaml | ||
| openapi: 3.0.3 | ||
| info: | ||
| title: Bundled specs file. See individual paths.verb.tags for details | ||
| version: not applicable | ||
| paths: | ||
| /api/path/to/endpoint: | ||
| get: | ||
| operationId: MyGetEndpoint | ||
| responses: | ||
| '200': | ||
| description: Successful response | ||
| content: | ||
| application/json: | ||
| schema: | ||
| type: object | ||
| post: | ||
| operationId: MyPostEndpoint | ||
| responses: | ||
| '200': | ||
| description: Successful response | ||
| content: | ||
| application/json: | ||
| schema: | ||
| type: object | ||
| components: | ||
| schemas: {} | ||
| ``` |
There was a problem hiding this comment.
Could you please update this according to the changed format of the bundle?
Comment on lines
+185
to
+203
| The target spec will look like | ||
|
|
||
| ```yaml | ||
| openapi: 3.0.3 | ||
| info: | ||
| title: Bundled specs file. See individual paths.verb.tags for details | ||
| version: not applicable | ||
| paths: | ||
| /api/path/to/endpoint: | ||
| get: | ||
| operationId: MyGetEndpoint | ||
| responses: | ||
| '200': | ||
| description: Successful response | ||
| content: | ||
| application/json: | ||
| schema: | ||
| type: object | ||
| ``` |
Comment on lines
+241
to
+259
| The target spec will look like | ||
|
|
||
| ```yaml | ||
| openapi: 3.0.3 | ||
| info: | ||
| title: Bundled specs file. See individual paths.verb.tags for details | ||
| version: not applicable | ||
| paths: | ||
| /api/path/to/endpoint: | ||
| get: | ||
| operationId: MyGetEndpoint | ||
| responses: | ||
| '200': | ||
| description: Successful response | ||
| content: | ||
| application/json: | ||
| schema: | ||
| type: object | ||
| ``` |
Comment on lines
+300
to
+323
| The target spec will look like | ||
|
|
||
| ```yaml | ||
| openapi: 3.0.0 | ||
| info: | ||
| title: Bundled specs file. See individual paths.verb.tags for details | ||
| version: not applicable | ||
| paths: | ||
| /api/path/to/endpoint: | ||
| patch: | ||
| operationId: MyPatchEndpoint | ||
| requestBody: | ||
| required: true | ||
| content: | ||
| application/json: | ||
| schema: | ||
| type: object | ||
| properties: | ||
| param1: | ||
| type: string | ||
| enum: [val1, val2, val3] | ||
| param2: | ||
| type: number | ||
| ``` |
Comment on lines
+351
to
+377
| The target spec will look like | ||
|
|
||
| ```yaml | ||
| openapi: 3.0.0 | ||
| info: | ||
| title: Bundled specs file. See individual paths.verb.tags for details | ||
| version: not applicable | ||
| paths: | ||
| /api/path/to/endpoint: | ||
| patch: | ||
| operationId: MyPatchEndpoint | ||
| requestBody: | ||
| required: true | ||
| content: | ||
| application/json: | ||
| schema: | ||
| type: object | ||
| properties: | ||
| param1: | ||
| type: string | ||
| enum: [val1, val2, val3] | ||
| param2: | ||
| type: number | ||
| required: | ||
| - param1 | ||
| - param2 | ||
| ``` |
Comment on lines
+413
to
+436
| The target spec will look like | ||
|
|
||
| ```yaml | ||
| openapi: 3.0.0 | ||
| info: | ||
| title: Bundled specs file. See individual paths.verb.tags for details | ||
| version: not applicable | ||
| paths: | ||
| /api/path/to/endpoint: | ||
| patch: | ||
| operationId: MyPatchEndpoint | ||
| requestBody: | ||
| required: true | ||
| content: | ||
| application/json: | ||
| schema: | ||
| type: object | ||
| properties: | ||
| param1: | ||
| type: string | ||
| enum: [val1, val2, val3] | ||
| param2: | ||
| type: number | ||
| ``` |
Comment on lines
+475
to
+493
| The target spec will look like | ||
|
|
||
| ```yaml | ||
| openapi: 3.0.3 | ||
| info: | ||
| title: Bundled specs file. See individual paths.verb.tags for details | ||
| version: not applicable | ||
| paths: | ||
| /api/path/to/endpoint: | ||
| post: | ||
| operationId: MyPostEndpoint | ||
| responses: | ||
| '200': | ||
| description: Successful response | ||
| content: | ||
| application/json: | ||
| schema: | ||
| type: object | ||
| ``` |
banderror
approved these changes
Nov 28, 2023
There was a problem hiding this comment.
Fantastic, incredible work @maximpn! 💪
Thank you for working on changing the bundle format (given the very limited time you had before your time off), and adding the e2e test coverage for the tool. Thanks to your hard work we're now in a good spot in terms of docs generation, and we'll push it through the finish line together with @marshallmain (thankfully, there's not much stuff left to do).
Hope you will have a great time off, and see you in Prague! 👋
Comment on lines
+17
to
+30
| describe('OpenAPI Bundler', () => { | ||
| afterEach(() => { | ||
| removeTargetFile(); | ||
| }); | ||
|
|
||
| it('bundles two simple specs', async () => { | ||
| await bundleFolder('two_simple_specs'); | ||
| await expectBundleToMatchFile('two_simple_specs', 'expected.yaml'); | ||
| }); | ||
|
|
||
| it('bundles one file with a local reference', async () => { | ||
| await bundleFolder('spec_with_local_ref'); | ||
| await expectBundleToMatchFile('spec_with_local_ref', 'expected.yaml'); | ||
| }); |
There was a problem hiding this comment.
Amazing, thank you for adding the tests!
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
History
To update your PR or re-run it, just comment with: cc @maximpn |
Merged
2 tasks
spong
added a commit
that referenced
this pull request
Jan 11, 2024
…ature registration (#174317) ## Summary Resolves #172509 Adds ability to register feature capabilities through the assistant server so they no longer need to be plumbed through the `ElasticAssistantProvider`, which now also makes them available server side. Adds new `/internal/elastic_assistant/capabilities` route and `useCapabilities()` UI hook for fetching capabilities. ### OpenAPI Codegen Implemented using the new OpenAPI codegen and bundle packages: * Includes OpenAPI codegen script and CI action as detailed in: #166269 * Includes OpenAPI docs bundling script as detailed in: #171526 To run codegen/bundling locally, cd to `x-pack/plugins/elastic_assistant/` and run any of the following commands: ```bash yarn openapi:generate yarn openapi:generate:debug yarn openapi:bundle ``` > [!NOTE] > At the moment `yarn openapi:bundle` will output an empty bundled schema since `get_capabilities_route` is an internal route, this is to be expected. Also, if you don't see the file in your IDE, it's probably because `target` directories are ignored, so you may need to manually find/open the bundled schema at it's target location: `/x-pack/plugins/elastic_assistant/target/openapi/elastic_assistant.bundled.schema.yaml` ### Registering Capabilities To register a capability on plugin start, add the following in the consuming plugin's `start()`: ```ts plugins.elasticAssistant.registerFeatures(APP_UI_ID, { assistantModelEvaluation: config.experimentalFeatures.assistantModelEvaluation, assistantStreamingEnabled: config.experimentalFeatures.assistantStreamingEnabled, }); ``` ### Declaring Feature Capabilities Feature capabilities are declared in `x-pack/packages/kbn-elastic-assistant-common/impl/capabilities/index.ts`: ```ts /** * Interfaces for features available to the elastic assistant */ export type AssistantFeatures = { [K in keyof typeof assistantFeatures]: boolean }; export const assistantFeatures = Object.freeze({ assistantModelEvaluation: false, assistantStreamingEnabled: false, }); ``` ### Using Capabilities Client Side And can be fetched client side using the `useCapabilities()` hook ala: ```ts // Fetch assistant capabilities const { data: capabilities } = useCapabilities({ http, toasts }); const { assistantModelEvaluation: modelEvaluatorEnabled, assistantStreamingEnabled } = capabilities ?? assistantFeatures; ``` ### Using Capabilities Server Side Or server side within a route (or elsewhere) via the `assistantContext`: ```ts const assistantContext = await context.elasticAssistant; const pluginName = getPluginNameFromRequest({ request, logger }); const registeredFeatures = assistantContext.getRegisteredFeatures(pluginName); if (!registeredFeatures.assistantModelEvaluation) { return response.notFound(); } ``` > [!NOTE] > Note, just as with [registering arbitrary tools](#172234), features are registered for a specific plugin, where the plugin name that corresponds to your application is defined in the `x-kbn-context` header of requests made from your application, which may be different than your plugin's registered `APP_ID`. Perhaps this separation of concerns from one plugin to another isn't necessary, but it was easy to add matching the behavior of registering arbitrary tools. We can remove this granularity in favor of global features if desired. ### Test Steps * Verify `/internal/elastic_assistant/capabilities` route is called on security solution page load in dev tools, and that by default the `Evaluation` UI in setting does is not displayed and `404`'s if manually called. * Set the below experimental feature flag in your `kibana.dev.yml` and observe the feature being enabled by inspecting the capabilities api response, and that the evaluation feature becomes available: ``` xpack.securitySolution.enableExperimental: [ 'assistantModelEvaluation'] ``` * Run the `yarn openapi:*` codegen scripts above and ensure they execute as expected (code is generated/bundled) ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
eokoneyo
pushed a commit
to eokoneyo/kibana
that referenced
this pull request
Jan 12, 2024
…ature registration (elastic#174317) ## Summary Resolves elastic#172509 Adds ability to register feature capabilities through the assistant server so they no longer need to be plumbed through the `ElasticAssistantProvider`, which now also makes them available server side. Adds new `/internal/elastic_assistant/capabilities` route and `useCapabilities()` UI hook for fetching capabilities. ### OpenAPI Codegen Implemented using the new OpenAPI codegen and bundle packages: * Includes OpenAPI codegen script and CI action as detailed in: elastic#166269 * Includes OpenAPI docs bundling script as detailed in: elastic#171526 To run codegen/bundling locally, cd to `x-pack/plugins/elastic_assistant/` and run any of the following commands: ```bash yarn openapi:generate yarn openapi:generate:debug yarn openapi:bundle ``` > [!NOTE] > At the moment `yarn openapi:bundle` will output an empty bundled schema since `get_capabilities_route` is an internal route, this is to be expected. Also, if you don't see the file in your IDE, it's probably because `target` directories are ignored, so you may need to manually find/open the bundled schema at it's target location: `/x-pack/plugins/elastic_assistant/target/openapi/elastic_assistant.bundled.schema.yaml` ### Registering Capabilities To register a capability on plugin start, add the following in the consuming plugin's `start()`: ```ts plugins.elasticAssistant.registerFeatures(APP_UI_ID, { assistantModelEvaluation: config.experimentalFeatures.assistantModelEvaluation, assistantStreamingEnabled: config.experimentalFeatures.assistantStreamingEnabled, }); ``` ### Declaring Feature Capabilities Feature capabilities are declared in `x-pack/packages/kbn-elastic-assistant-common/impl/capabilities/index.ts`: ```ts /** * Interfaces for features available to the elastic assistant */ export type AssistantFeatures = { [K in keyof typeof assistantFeatures]: boolean }; export const assistantFeatures = Object.freeze({ assistantModelEvaluation: false, assistantStreamingEnabled: false, }); ``` ### Using Capabilities Client Side And can be fetched client side using the `useCapabilities()` hook ala: ```ts // Fetch assistant capabilities const { data: capabilities } = useCapabilities({ http, toasts }); const { assistantModelEvaluation: modelEvaluatorEnabled, assistantStreamingEnabled } = capabilities ?? assistantFeatures; ``` ### Using Capabilities Server Side Or server side within a route (or elsewhere) via the `assistantContext`: ```ts const assistantContext = await context.elasticAssistant; const pluginName = getPluginNameFromRequest({ request, logger }); const registeredFeatures = assistantContext.getRegisteredFeatures(pluginName); if (!registeredFeatures.assistantModelEvaluation) { return response.notFound(); } ``` > [!NOTE] > Note, just as with [registering arbitrary tools](elastic#172234), features are registered for a specific plugin, where the plugin name that corresponds to your application is defined in the `x-kbn-context` header of requests made from your application, which may be different than your plugin's registered `APP_ID`. Perhaps this separation of concerns from one plugin to another isn't necessary, but it was easy to add matching the behavior of registering arbitrary tools. We can remove this granularity in favor of global features if desired. ### Test Steps * Verify `/internal/elastic_assistant/capabilities` route is called on security solution page load in dev tools, and that by default the `Evaluation` UI in setting does is not displayed and `404`'s if manually called. * Set the below experimental feature flag in your `kibana.dev.yml` and observe the feature being enabled by inspecting the capabilities api response, and that the evaluation feature becomes available: ``` xpack.securitySolution.enableExperimental: [ 'assistantModelEvaluation'] ``` * Run the `yarn openapi:*` codegen scripts above and ensure they execute as expected (code is generated/bundled) ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
semd
pushed a commit
to semd/kibana
that referenced
this pull request
Jan 12, 2024
…ature registration (elastic#174317) ## Summary Resolves elastic#172509 Adds ability to register feature capabilities through the assistant server so they no longer need to be plumbed through the `ElasticAssistantProvider`, which now also makes them available server side. Adds new `/internal/elastic_assistant/capabilities` route and `useCapabilities()` UI hook for fetching capabilities. ### OpenAPI Codegen Implemented using the new OpenAPI codegen and bundle packages: * Includes OpenAPI codegen script and CI action as detailed in: elastic#166269 * Includes OpenAPI docs bundling script as detailed in: elastic#171526 To run codegen/bundling locally, cd to `x-pack/plugins/elastic_assistant/` and run any of the following commands: ```bash yarn openapi:generate yarn openapi:generate:debug yarn openapi:bundle ``` > [!NOTE] > At the moment `yarn openapi:bundle` will output an empty bundled schema since `get_capabilities_route` is an internal route, this is to be expected. Also, if you don't see the file in your IDE, it's probably because `target` directories are ignored, so you may need to manually find/open the bundled schema at it's target location: `/x-pack/plugins/elastic_assistant/target/openapi/elastic_assistant.bundled.schema.yaml` ### Registering Capabilities To register a capability on plugin start, add the following in the consuming plugin's `start()`: ```ts plugins.elasticAssistant.registerFeatures(APP_UI_ID, { assistantModelEvaluation: config.experimentalFeatures.assistantModelEvaluation, assistantStreamingEnabled: config.experimentalFeatures.assistantStreamingEnabled, }); ``` ### Declaring Feature Capabilities Feature capabilities are declared in `x-pack/packages/kbn-elastic-assistant-common/impl/capabilities/index.ts`: ```ts /** * Interfaces for features available to the elastic assistant */ export type AssistantFeatures = { [K in keyof typeof assistantFeatures]: boolean }; export const assistantFeatures = Object.freeze({ assistantModelEvaluation: false, assistantStreamingEnabled: false, }); ``` ### Using Capabilities Client Side And can be fetched client side using the `useCapabilities()` hook ala: ```ts // Fetch assistant capabilities const { data: capabilities } = useCapabilities({ http, toasts }); const { assistantModelEvaluation: modelEvaluatorEnabled, assistantStreamingEnabled } = capabilities ?? assistantFeatures; ``` ### Using Capabilities Server Side Or server side within a route (or elsewhere) via the `assistantContext`: ```ts const assistantContext = await context.elasticAssistant; const pluginName = getPluginNameFromRequest({ request, logger }); const registeredFeatures = assistantContext.getRegisteredFeatures(pluginName); if (!registeredFeatures.assistantModelEvaluation) { return response.notFound(); } ``` > [!NOTE] > Note, just as with [registering arbitrary tools](elastic#172234), features are registered for a specific plugin, where the plugin name that corresponds to your application is defined in the `x-kbn-context` header of requests made from your application, which may be different than your plugin's registered `APP_ID`. Perhaps this separation of concerns from one plugin to another isn't necessary, but it was easy to add matching the behavior of registering arbitrary tools. We can remove this granularity in favor of global features if desired. ### Test Steps * Verify `/internal/elastic_assistant/capabilities` route is called on security solution page load in dev tools, and that by default the `Evaluation` UI in setting does is not displayed and `404`'s if manually called. * Set the below experimental feature flag in your `kibana.dev.yml` and observe the feature being enabled by inspecting the capabilities api response, and that the evaluation feature becomes available: ``` xpack.securitySolution.enableExperimental: [ 'assistantModelEvaluation'] ``` * Run the `yarn openapi:*` codegen scripts above and ensure they execute as expected (code is generated/bundled) ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
CoenWarmer
pushed a commit
to CoenWarmer/kibana
that referenced
this pull request
Feb 15, 2024
…ature registration (elastic#174317) ## Summary Resolves elastic#172509 Adds ability to register feature capabilities through the assistant server so they no longer need to be plumbed through the `ElasticAssistantProvider`, which now also makes them available server side. Adds new `/internal/elastic_assistant/capabilities` route and `useCapabilities()` UI hook for fetching capabilities. ### OpenAPI Codegen Implemented using the new OpenAPI codegen and bundle packages: * Includes OpenAPI codegen script and CI action as detailed in: elastic#166269 * Includes OpenAPI docs bundling script as detailed in: elastic#171526 To run codegen/bundling locally, cd to `x-pack/plugins/elastic_assistant/` and run any of the following commands: ```bash yarn openapi:generate yarn openapi:generate:debug yarn openapi:bundle ``` > [!NOTE] > At the moment `yarn openapi:bundle` will output an empty bundled schema since `get_capabilities_route` is an internal route, this is to be expected. Also, if you don't see the file in your IDE, it's probably because `target` directories are ignored, so you may need to manually find/open the bundled schema at it's target location: `/x-pack/plugins/elastic_assistant/target/openapi/elastic_assistant.bundled.schema.yaml` ### Registering Capabilities To register a capability on plugin start, add the following in the consuming plugin's `start()`: ```ts plugins.elasticAssistant.registerFeatures(APP_UI_ID, { assistantModelEvaluation: config.experimentalFeatures.assistantModelEvaluation, assistantStreamingEnabled: config.experimentalFeatures.assistantStreamingEnabled, }); ``` ### Declaring Feature Capabilities Feature capabilities are declared in `x-pack/packages/kbn-elastic-assistant-common/impl/capabilities/index.ts`: ```ts /** * Interfaces for features available to the elastic assistant */ export type AssistantFeatures = { [K in keyof typeof assistantFeatures]: boolean }; export const assistantFeatures = Object.freeze({ assistantModelEvaluation: false, assistantStreamingEnabled: false, }); ``` ### Using Capabilities Client Side And can be fetched client side using the `useCapabilities()` hook ala: ```ts // Fetch assistant capabilities const { data: capabilities } = useCapabilities({ http, toasts }); const { assistantModelEvaluation: modelEvaluatorEnabled, assistantStreamingEnabled } = capabilities ?? assistantFeatures; ``` ### Using Capabilities Server Side Or server side within a route (or elsewhere) via the `assistantContext`: ```ts const assistantContext = await context.elasticAssistant; const pluginName = getPluginNameFromRequest({ request, logger }); const registeredFeatures = assistantContext.getRegisteredFeatures(pluginName); if (!registeredFeatures.assistantModelEvaluation) { return response.notFound(); } ``` > [!NOTE] > Note, just as with [registering arbitrary tools](elastic#172234), features are registered for a specific plugin, where the plugin name that corresponds to your application is defined in the `x-kbn-context` header of requests made from your application, which may be different than your plugin's registered `APP_ID`. Perhaps this separation of concerns from one plugin to another isn't necessary, but it was easy to add matching the behavior of registering arbitrary tools. We can remove this granularity in favor of global features if desired. ### Test Steps * Verify `/internal/elastic_assistant/capabilities` route is called on security solution page load in dev tools, and that by default the `Evaluation` UI in setting does is not displayed and `404`'s if manually called. * Set the below experimental feature flag in your `kibana.dev.yml` and observe the feature being enabled by inspecting the capabilities api response, and that the evaluation feature becomes available: ``` xpack.securitySolution.enableExperimental: [ 'assistantModelEvaluation'] ``` * Run the `yarn openapi:*` codegen scripts above and ensure they execute as expected (code is generated/bundled) ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
maximpn
added a commit
that referenced
this pull request
Feb 23, 2024
## Summary This PR moves `AlertIds` schema definition to one schema to avoid duplication. ## Details OAS should as simple and straightforward as possible. Having the same entities defined in different schemas may spawn ambiguity issues. On top of that [OAS docs bundler](#171526) requires unique names for shared schemas. `AlertIds` definition is duplicated in `x-pack/plugins/security_solution/common/api/endpoint/model/schema/common.schema.yaml` and `x-pack/plugins/security_solution/common/api/detection_engine/alert_assignees/set_alert_assignees_route.schema.yaml`. To get rid of this duplication `AlertIds` definition has been moved in a common `Shared Alert Schema Primitives` and referenced accordingly in the schema files it's used. Additionally `NonEmptyString` and `UUID` schemas were moved from rule schema to a common `Shared Primitives Schema` as it's not only related to the rule schema.
fkanout
pushed a commit
to fkanout/kibana
that referenced
this pull request
Mar 4, 2024
…77425) ## Summary This PR moves `AlertIds` schema definition to one schema to avoid duplication. ## Details OAS should as simple and straightforward as possible. Having the same entities defined in different schemas may spawn ambiguity issues. On top of that [OAS docs bundler](elastic#171526) requires unique names for shared schemas. `AlertIds` definition is duplicated in `x-pack/plugins/security_solution/common/api/endpoint/model/schema/common.schema.yaml` and `x-pack/plugins/security_solution/common/api/detection_engine/alert_assignees/set_alert_assignees_route.schema.yaml`. To get rid of this duplication `AlertIds` definition has been moved in a common `Shared Alert Schema Primitives` and referenced accordingly in the schema files it's used. Additionally `NonEmptyString` and `UUID` schemas were moved from rule schema to a common `Shared Primitives Schema` as it's not only related to the rule schema.
6 tasks
maximpn
added a commit
that referenced
this pull request
May 8, 2024
…#181944) **Addresses:** elastic/security-team#7981 **Relates to:** #171526 ## Summary This PR adjusts OpenAPI Bundler (`kbn-openapi-bundler` package) based on Docs Engineering team requirements. Main requirements include producing one valid OpenAPI spec bundle file. After adjustments OpenAPI Bundler one valid OpenAPI spec bundle file per API version e.g. `2023-10-31-my-oas.bundled.schema.yaml`. ## Details This PR further improves #171526 to satisfy Docs Engineering team requirements and includes the following adjustments and improvements - [x] Produce one valid OpenAPI spec file per API version - [x] Make `outputFilePath` independent from `rootDir` `outputFilePath` can be an absolute or relative to node.js working directory path. Additionally it can contain `{version}` placeholder to adjust where API version is placed in the result bundle file name. API prepends the result bundle file name if `{version}` is omitted. - [x] Reduce folded `allOf` items Inlined schemas lead to `allOf` folder into each other as well as having multiple object schemas which could be merged. Docs Engineering team stated potential problems related to folded `allOf` items. - [x] Prevent bundling incompatible source OpenAPI schemas like `3.0.3` and `3.1.0` - [x] Make sure to bundle path item's `summary`, `description` and `properties` - [x] Make sure to bundle all shared components (not only `schemas`) Besides the changes above PR contains minor bug fixing and improvements. **Note:** It intentionally doesn't include bundling configuration changes like integration it into PR's build pipeline and focuses only on functional changes. Bundling configuration will be addressed separately to avoid spreading reviewers focus.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses: https://github.com/elastic/security-team/issues/7981
Summary
This PR adds an OpenAPI spec bundler to simplify integration with the Docs Engineering team. The bundler produces a single bundled file by dereferencing and inlining some of external references and bundling them and paths into a single file.
Details
Currently we maintain a number of schema files inside
x-pack/plugins/security_solution/common/api/**.schema.yamland it might be hard for external teams to keep track of all the changes in our schemas. By creating a singular schema file, we provide a clear integration point for others.The bundler addresses the following issues
RuleActionAlertsFilterorRuleActionParamsare exposed directly from the Alerting framework and might be considered implementation details, we don't want to document interfaces that are not designed to be public so hiding them is a good option)x-modifyproperty (Instead of exposingx-modify: partialwe need to make the exported data structure partial and instead of exposingx-modify: requiredwe need to make the exported data structure required)x-codegen-enabledandx-modifyin common_attributes.schema.yamlwhich might make the final documentation hardly usable from the UX perspective, so we can inline them)and lives in a new
@kbn/openapi-bundlerpackage underpackages/kbn-openapi-bundlerfolder.Related changes
version: 2023-10-31has been changed to explicit string typeversion: '2023-10-31'for all specs undersecurity_solution/common/apifolder. Implicit type causesjs-yamlparsing it as aDataJS object leading to serializing it like2023-10-31T00:00:00.000Z.ListRequestQueryschema insecurity_solution/common/api/endpoint/actions/list.schema.yamlhas been renamed toEndpointActionListRequestQueryto avoid conflicts withListRequestQueryinsecurity_solution/common/api/endpoint/metadata/list_metadata.schema.yaml. While it's not an issue to have completely different schemas sharing the same name in different files it may be an indication of pitfalls in the API design. I'd say it's an open question if such cases need to be always resolved automatically or reviewed manually. At this moment the bundler can't resolve such conflicts.How to test?
There is a a new JS script added to Security Solution plugin located at
x-pack/plugins/security_solution/scripts/openapi/bundle.jswith a corresponding entry inpackage.jsonnamedopenapi:bundle.To test the PR change directory to Security Solution plugin's root folder and run the bundler like below
cd x-pack/plugins/security_solution yarn openapi:bundleIt should produce a bundled OpenAPI spec at
x-pack/plugins/security_solution/target/openapi/security_solution.bundled.schema.yaml.Open issues
3.0and3.1specs (Maybe convert automatically to3.1?). Folder like OpenAPI bundling format implemented here allows to mix OpenAPI3.0and3.1specs.Improvements