feat: faster, lazy-friendly runtime loading #4181

RomainMuller · 2023-07-17T10:35:07Z

Instead of eagerly loading each type from the assemblies in order to annotate the constructors with jsii fully qualified type names (FQNs), leverage the jsii.rtti data that is injected by the jsii compiler since release 1.19.0.

This should make the jsii runtimes friendlier to large libraries that include lazy-loading provisions, such as the aws-cdk-lib.

In addition to this, JSII_RUNTIME_PACKAGE_CACHE was flipped from opt-in to opt-out (set it to any value other than enabled to disable), and the @jsii/runtime entry point now sets --preserve-symlinks so that we can symbolically link packages from the cache instead of copying them around, which is significantly faster.

Finally, the jsii kernel had not opted out of the assembly validation feature, which is redundant in the majority of scenarios, and is quite time-consuming (~500ms for aws-cdk-lib)... So also opting out, but allowing users to opt-back-in via an environment variable.

Example on a "simple" repro via aws-cdk-lib:

Before:

[@jsii/kernel:timing] tar.extract(<redacted>/.venv-vanilla/lib/python3.11/site-packages/aws_cdk/_jsii/aws-cdk-lib@2.87.0.jsii.tgz) => <redacted>/jsii-kernel-Xxp43L/node_modules/aws-cdk-lib: 1.909s
[@jsii/kernel:timing] loadAssemblyFromPath(<redacted>/jsii-kernel-Xxp43L/node_modules/aws-cdk-lib): 383.8ms
[@jsii/kernel:timing] require(<redacted>/jsii-kernel-Xxp43L/node_modules/aws-cdk-lib): 630.081ms
[@jsii/kernel:timing] registerAssembly({ name: aws-cdk-lib, types: 10957 }): 8.452ms
[@jsii/kernel:timing] load({
  "name": "aws-cdk-lib",
  "version": "2.87.0",
  "tarball": "<redacted>/.venv-vanilla/lib/python3.11/site-packages/aws_cdk/_jsii/aws-cdk-lib@2.87.0.jsii.tgz",
  "api": "load"
}): 2.933s

After:

[@jsii/kernel:timing] tar.extract(<redacted>/.venv-lazy/lib/python3.11/site-packages/aws_cdk/_jsii/aws-cdk-lib@2.87.0.jsii.tgz) => <redacted>/jsii-kernel-eAOMah/node_modules/aws-cdk-lib: 12.247ms
[@jsii/kernel:timing] loadAssemblyFromPath(<redacted>/jsii-kernel-eAOMah/node_modules/aws-cdk-lib): 388.388ms
[@jsii/kernel:timing] require(<redacted>/jsii-kernel-eAOMah/node_modules/aws-cdk-lib/lazy-index.js): 132.801ms
[@jsii/kernel:timing] registerAssembly({ name: aws-cdk-lib, types: 10957 }): 0.009ms
[@jsii/kernel:timing] load({
  "name": "aws-cdk-lib",
  "version": "2.87.0",
  "tarball": "<redacted>/.venv-lazy/lib/python3.11/site-packages/aws_cdk/_jsii/aws-cdk-lib@2.87.0.jsii.tgz",
  "api": "load"
}): 537.449ms

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Instead of eagerly loading each type from the assemblies in order to annotate the constructos with jsii fully qualified type names (FQNs), leverage the `jsii.rtti` data that is injected by the `jsii` compiler since release `1.19.0`.

# Conflicts: # packages/@jsii/kernel/src/kernel.ts # packages/@jsii/kernel/src/objects.ts

packages/@jsii/kernel/src/kernel.ts

mergify · 2023-07-25T10:36:08Z

Thank you for contributing! ❤️ I will now look into making sure the PR is up-to-date, then proceed to try and merge it!

mergify · 2023-07-25T10:36:43Z

Merging (with squash)...

In #4181, a faster method to load modules was introduced: symlinking instead of recursing through the directory tree, mostly affecting the load times of large modules. Since Windows Vista, non-Administrator users on Windows aren't allowed to create symlinks anymore, so this new loading method fails for users working in corporate Windows environments. Catch the error and fall back to the slower copying method if that happens. Fixes #4208.

In #4181, a faster method to load modules was introduced: symlinking instead of recursing through the directory tree, mostly affecting the load times of large modules. Since Windows Vista, non-Administrator users on Windows aren't allowed to create symlinks anymore, so this new loading method fails for users working in corporate Windows environments. Catch the error and fall back to the slower copying method if that happens. Fixes #4208. --- By submitting this pull request, I confirm that my contribution is made under the terms of the [Apache 2.0 license]. [Apache 2.0 license]: https://www.apache.org/licenses/LICENSE-2.0

The package cache mechanism that was turned on by default in #4181 works in theory under parallelism, but not in practice. Typically the CDK CLI will prevent CDK apps from running in parallel, but Python testing frameworks like `tox` use subprocess parallelism to speed up test runs, leading to the jsii imports being executed at the same time. Since jsii is sync, the locking needs to be sync. The sync locking feature of the `lockfile` library doesn't have wait support (for good reason), and so when a lock is already acquired by another process it quickly burns through its 12 retries in a hot loop, and then exits with an error. Two changes to address this: - (Ab)use `Atomics.wait()` to get a synchronous sleeping primitive; since `lockSync` doesn't support synchronous sleep, we build our own retry loop with synchronous sleep around `lockSync`. - Since the extracted directory is immutable: if the marker file in the extracted directory exists, we can treat it as evidence that the directory has been completely written and we can skip trying to vy for exclusive access to write it. This avoids all lock contention after the very first CDK app execution. Fixes #4207.

The package cache mechanism that was turned on by default in #4181 works in theory under parallelism, but not in practice. Typically the CDK CLI will prevent CDK apps from running in parallel, but Python testing frameworks like `tox` use subprocess parallelism to speed up test runs, leading to the jsii imports being executed at the same time. Since jsii is sync, the locking needs to be sync. The sync locking feature of the `lockfile` library doesn't have wait support (for good reason), and so when a lock is already acquired by another process it quickly burns through its 12 retries in a hot loop, and then exits with an error. Two changes to address this: - (Ab)use `Atomics.wait()` to get a synchronous sleeping primitive; since `lockSync` doesn't support synchronous sleep, we build our own retry loop with synchronous sleep around `lockSync`. - Since the extracted directory is immutable: if the marker file in the extracted directory exists, we can treat it as evidence that the directory has been completely written and we can skip trying to vy for exclusive access to write it. This avoids all lock contention after the very first CDK app execution. Fixes #4207. --- By submitting this pull request, I confirm that my contribution is made under the terms of the [Apache 2.0 license]. [Apache 2.0 license]: https://www.apache.org/licenses/LICENSE-2.0

mergify bot added the contribution/core This is a PR that came from AWS. label Jul 17, 2023

also fix that exports is not being used

565e04a

RomainMuller marked this pull request as ready for review July 17, 2023 12:41

RomainMuller added 2 commits July 17, 2023 14:51

Merge remote-tracking branch 'origin/main' into rmuller/lazy-friend

f3ce4b0

default-enable the package cache, set --preserve-symlinks

19c7871

RomainMuller changed the title ~~feat: leverage runtime type information in jsii runtime~~ feat: faster, lazy-friendly runtime loading Jul 17, 2023

RomainMuller added 4 commits July 18, 2023 13:02

Merge remote-tracking branch 'origin/main' into rmuller/lazy-friend

fc2ba71

# Conflicts: # packages/@jsii/kernel/src/kernel.ts # packages/@jsii/kernel/src/objects.ts

undo unnecessary change

cfe40a6

add some info in CONTRIBUTING.md

d5ae229

opt out of assembly validation by default

4b6dac1

DanielMSchmidt mentioned this pull request Jul 24, 2023

feat(provider-generator): use lazy loading of generated provider namespaces hashicorp/terraform-cdk#3012

Closed

3 tasks

Merge remote-tracking branch 'origin/main' into rmuller/lazy-friend

8ab3813

RomainMuller force-pushed the rmuller/lazy-friend branch from 8ed8f68 to 8ab3813 Compare July 24, 2023 09:51

mrgrain requested changes Jul 24, 2023

View reviewed changes

packages/@jsii/kernel/src/kernel.ts Show resolved Hide resolved

RomainMuller mentioned this pull request Jul 25, 2023

Imports of large Python library (aws-cdk-lib) extremely slow #3389

Open

5 tasks

mrgrain approved these changes Jul 25, 2023

View reviewed changes

mergify bot added the pr/ready-to-merge This PR is ready to be merged. label Jul 25, 2023

mergify bot merged commit ef6e5b1 into main Jul 25, 2023

mergify bot deleted the rmuller/lazy-friend branch July 25, 2023 10:36

mergify bot removed the pr/ready-to-merge This PR is ready to be merged. label Jul 25, 2023

qiv-paul mentioned this pull request Aug 2, 2023

New feature from PR #4181 causes "EPERM: operation not permitted, symlink" on Windows #4208

Closed

sirrus233 mentioned this pull request Aug 3, 2023

jsii release 1.86.0 is breaking our tox tests #4207

Closed

rix0rrr mentioned this pull request Aug 8, 2023

fix(kernel): fast module loading fails on Windows (EPERM) #4212

Merged

rix0rrr mentioned this pull request Aug 9, 2023

fix(kernel): package cache fails under parallelism #4215

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: faster, lazy-friendly runtime loading #4181

feat: faster, lazy-friendly runtime loading #4181

RomainMuller commented Jul 17, 2023 •

edited

Loading

mergify bot commented Jul 25, 2023

mergify bot commented Jul 25, 2023

feat: faster, lazy-friendly runtime loading #4181

feat: faster, lazy-friendly runtime loading #4181

Conversation

RomainMuller commented Jul 17, 2023 • edited Loading

mergify bot commented Jul 25, 2023

mergify bot commented Jul 25, 2023

RomainMuller commented Jul 17, 2023 •

edited

Loading