Skip to content

perf(kb): lazy-load KB schemas per namespace/endpoint (#251)#266

Merged
margaretjgu merged 5 commits into
mainfrom
perf/kb-lazy-load
May 1, 2026
Merged

perf(kb): lazy-load KB schemas per namespace/endpoint (#251)#266
margaretjgu merged 5 commits into
mainfrom
perf/kb-lazy-load

Conversation

@margaretjgu
Copy link
Copy Markdown
Member

@margaretjgu margaretjgu commented May 1, 2026

Summary

closes #251

  • elastic stack kb previously loaded all 48 namespace files (500 endpoint definitions) and constructed Zod schemas for all of them on every invocation, including --help
  • Added src/kb/api-manifest.ts with lightweight metadata for all 500 endpoints (no Zod construction)
  • Rewrote src/kb/apis.ts as a lazy-loading barrel with loadKbApi() / loadKbApisInFile() dynamic import helpers
  • Added registerKbCommandsLazy() to src/kb/register.ts: sniffs process.argv to identify the invoked endpoint, loads only that namespace file, registers everything else as lightweight stubs
  • Kept registerKbCommands(definitions) for test use
  • elastic stack kb --help now runs in ~0.4s

Test plan

  • All 21 CLI smoke tests pass including elastic stack kibana --help and elastic kb --help deprecation redirect
  • All 20 KB unit tests pass
  • elastic stack kb alerting get-alerting-rule-id --help shows correct flags (lazy load works)
  • elastic stack kb --help lists all namespace groups

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ COPYPASTE jscpd yes no no 5.93s
✅ REPOSITORY gitleaks yes no no 97.46s
✅ REPOSITORY git_diff yes no no 0.47s
✅ REPOSITORY secretlint yes no no 13.86s
✅ REPOSITORY trivy yes no no 15.12s
✅ TYPESCRIPT eslint 4 0 0 6.46s

See detailed reports in MegaLinter artifacts
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

Copy link
Copy Markdown
Member

@JoshMock JoshMock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

@margaretjgu margaretjgu merged commit 1197e58 into main May 1, 2026
18 checks passed
@margaretjgu margaretjgu deleted the perf/kb-lazy-load branch May 1, 2026 19:40
flobernd added a commit that referenced this pull request May 13, 2026
- Add `codegen:kibana` target invoking `npx tsx cli/kibana/index.ts` and
  copying `output/kibana/apis/*.ts` into `src/kb/apis/`. The hand-written
  lazy loader at `src/kb/apis.ts` is intentionally not overwritten.
- Run `scripts/build-api-manifest.mjs` after ES codegen and
  `scripts/build-kb-manifest.mts` after Kibana codegen so the per-endpoint
  manifests stay in sync with the generated namespace files.
- Stop copying `output/es/index.ts` over `src/es/apis.ts` (it became a
  hand-written lazy loader in #218).
- Fix `build-kb-manifest.mts` to load definitions directly from each
  `src/kb/apis/*.ts` file (the previous `allKbApis` import was dead since
  the lazy-loader refactor in #266). Manifest regen also picks up the one
  endpoint that had been missing as a result.
- Workflow: append the kibana step, expand diff-detection and `add-paths`
  to include `src/kb`, refresh the PR body to list the kibana outputs.
- CONTRIBUTING: document the new target.

Refs: #79
flobernd added a commit that referenced this pull request May 13, 2026
- Add `codegen:kibana` target invoking `npx tsx cli/kibana/index.ts` and
  copying `output/kibana/apis/*.ts` into `src/kb/apis/`. The hand-written
  lazy loader at `src/kb/apis.ts` is intentionally not overwritten.
- Run `scripts/build-api-manifest.mjs` after ES codegen and
  `scripts/build-kb-manifest.mts` after Kibana codegen so the per-endpoint
  manifests stay in sync with the generated namespace files.
- Stop copying `output/es/index.ts` over `src/es/apis.ts` (it became a
  hand-written lazy loader in #218).
- Fix `build-kb-manifest.mts` to load definitions directly from each
  `src/kb/apis/*.ts` file (the previous `allKbApis` import was dead since
  the lazy-loader refactor in #266). Manifest regen also picks up the one
  endpoint that had been missing as a result.
- Workflow: append the kibana step, expand diff-detection and `add-paths`
  to include `src/kb`, refresh the PR body to list the kibana outputs.
- CONTRIBUTING: document the new target.

Refs: #79
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(kb): lazy-load KB schemas per namespace/endpoint to cut startup heap

2 participants