fix: support hoisted jsii dependencies

[MrArnoldPalmer]

Removes copy of the project directory and instead only outputs
transliterated assemblies to the working directory. This avoids
potential problems when generating docs for packages that may have some
dependencies that are symlinks, such as those in a monorepo.

Pending aws/jsii#3437

fixes #390

[MrArnoldPalmer]

NOTE: Originally this was targeting allowing users to transliterate assemblies for projects that may have hoisted dependencies, IE a dependency is within a parent directory's node_modules somewhere. However, just changing to using the package directory as the working directory and outputting the transliterated assemblies elsewhere isn't good enough, as the assembliesDir defaults to the project directory and wont search parent directories for dependencies/.jsii assemblies.

This however should still fix the resolution of symlinked dependencies that may exist in a packages node_modules such as those within a lerna managed monorepo. This case is slightly harder to setup a succint test for but open to suggestions if reviewers feel strongly about adding one.

[agdimech]

This PR also improves performance by ~ 6x :)

One of my builds is down from 6 mins to 1 min with this PR.

In terms of the failing tests, this is caused by a transient error and jest timeout flakiness. Ran locally and all tests passed:

Test Suites: 14 passed, 14 total
Tests:       109 passed, 109 total
Snapshots:   132 passed, 132 total
Time:        506.923 s

[MrArnoldPalmer]

yeah tests are fine for me as well but I haven't bothered trying to fix the flakiness in a different PR yet.

[agdimech]

@Chriscbr - are we able to merge this or do tests have to pass here?

[MrArnoldPalmer]

@agdimech tests definitely have to pass, there is a some non-zero chance I've broken something (maybe a race condition causing the discrepancy in local vs CI execution).

[agdimech]

Couldn't figure out how to contribute to this PR so I opened a fresh one which should fix the unit tests: #714

Feel free to take the fix and apply it here and close mine or vise versa.

[MrArnoldPalmer]

@agdimech ty! I didn't realize it was purely a timeout as I tried previously upping it and still seeing failures. I merged in your commit here just to prevent re-review etc. Should auto-merge when successful.

[agdimech]

Sorry, I guess that didn’t fix it :(

I’ve seen this combination of errors locally to which I bumped the timeout and it resolved. Guess my hardware might be a bit beefier then the hw used as part of the build.

We can try bumping this to something like 120 although there may be a nasty timing issue contributing to this. What is strange is I would have thought that the test duration would have reduced with this PR, given the lack of fs.copy.

[agdimech]

I just had a look at the other open PR: https://github.com/cdklabs/jsii-docgen/runs/6928391566?check_suite_focus=true

the same issue is showing up there related to:
ENOENT: no such file or directory, chdir '/tmp/EtnTZW' -> '/tmp/wryVeX . What is really strange is it is failing when trying to cd back to the previous cwd as per:

jsii-docgen/src/docgen/view/documentation.ts

Line 382 in dd39bac

process.chdir(cwd);

Is it possible for this scenario to occur?

A) Simple Formatter Test Runs from cwd tmp/wryVeX and commences the work function .
B) Another test starts and create a workDir at tmp/wryVeX. This test completes and rms tmp/wryVeX
C) Simple Formatter Test completes and tries to cd back to cwd which no longer exists, hence the failure.

[RichiCoder1]

I'm manually applying this patch to a local installation and I hope it moves forward if only because the massive perf boost.

[agdimech]

And thoughts on my previous comments?

I don’t think these build failures are related to this PR as per: https://github.com/cdklabs/jsii-docgen/runs/7122708633?check_suite_focus=true

[agdimech]

Any idea how to proceed with this?

[RichiCoder1]

Plusing this again. Fix both perf issues and support for Yarn/PNP which hoist dependencies.

[agdimech]

This can be closed as #981 has been merged and released as of version ^7.2.1 :)

[MrArnoldPalmer]

closing in favor of #981