Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Generate package-locks for demo serving (#1220)
This change implements a package lock generator. This allows modules in a demo to be loaded using versions based on the resolved package lock result. The package lock is generated by copying the `package.json` file, placing it an isolated temporary directory and running `npm install` in a separate process. Copying the `package.json` file is required to support both `dependencies` and `devDependencies`, which is a common use case wrt demos. This is an expensive operation which depends on the transitive dependency size. As such, there are number of performance improvements built in: * **Persistent backing** - generated package locks are permanently stored in Firestore. This is required to ensure load consistency since this is persisted across instances & deployments. As a result, the package-lock generation performance penalty will only be seen on the _first request for that package version_. To prevent users from ever seeing this latency, requests can be made to generate the package lock before any user needs it. * **In-memory cache** - a per-instance least recently used cache is used to improve performance of package lock reads. * **Data structure** - package locks are not stored in the same format that they are generated. The initial representation is very large and instead they are converted to a flat object with only package names & versions. This implies that conflicting versions present in `package-lock.json` are not preserved. This results in an ~100x compression vs original `package-lock.json` files (eg 432KB package-lock.json is now 6KB). * **Compression** - before storing into the in-memory cache, gzip compression is used to reduce memory and increase the number of items that can be cached. (For the 432KB package-lock.json, this is now 2.2KB). To use the package lock, the HTML/JS rewriter now inserts the root package string as a query parameter to subsequent requests. These requests can then simply use the same package-lock to resolve versions.
- Loading branch information