Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sh/archiver to jszip swap #975

Merged
merged 20 commits into from May 31, 2023
Merged

Sh/archiver to jszip swap #975

merged 20 commits into from May 31, 2023

Conversation

shetzel
Copy link
Contributor

@shetzel shetzel commented May 18, 2023

What does this PR do?

Replaces the archiver library with jszip.

What issues does this PR fix or reference?

@W-13196750@

QA Notes:
Be sure to test using both NodeJS v18.16.0 and v18.15.0, but primarily 18.16.0.

  1. conversion of static resources
  2. deploys, both source and metadata format with static resources
  3. retrieves
  4. package version create with static resources

@shetzel shetzel requested a review from a team as a code owner May 18, 2023 18:12
@@ -28,12 +28,12 @@
"@salesforce/core": "^3.36.2",
"@salesforce/kit": "^1.9.2",
"@salesforce/ts-types": "^1.7.2",
"archiver": "^5.3.1",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

be sure to take the types out of devDeps

const zip = JSZip();

const zipDirRecursive = (dir: string): void => {
const list = fs.readdirSync(dir);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

readdir has an option for withFileTypes that will give you the isDirectory value without having to do another fs operation inside the loop.


public constructor(rootDestination?: SourcePath) {
super(rootDestination);
void pipeline(this.zip, this.getOutputStream());
const destination = rootDestination ? `for: ${rootDestination}` : 'in memory';
this.logger.debug(`generating zip ${destination}`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the logger is only used in ZipWriter, why not create it on the lowest level so all the other writers don't suffer the perf impact of unused loggers.

then, if someone else needs it, they could hoist it higher in the hierarchy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also used in StandardWriter

? this.addToZip(streamAsBuffer, writeInfo.output)
: // these will be zero-length files, which archiver supports via stream but not buffer
this.addToZip(writeInfo.source, writeInfo.output);
return this.addToZip(streamAsBuffer, writeInfo.output);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool!

Copy link
Contributor

@mshanemc mshanemc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commits seem fine, no obvious perf impacts (looking especially at the sourceToZip across both OS).

https://forcedotcom.github.io/source-deploy-retrieve/perf-Linux/
https://forcedotcom.github.io/source-deploy-retrieve/perf-Windows/

@mshanemc
Copy link
Contributor

QA notes:
✅ deploy a static resource (single file) to org
✅ deploy a static resource (single file, but empty) to org

✅ deploy a zip containing one empty text file
✅ deploy an empty static resource folder as a zip (same error as current SDR/PDR combo)
| Error testBasic The specified Static Resource is not a valid zip file

📓 Scenario: static resource has 1 zip file in it. That file is forceIgnored. we got an issue about this scenario once, and I saw the code change related to it.

current SDR/PDR throws this error before sending anything to the API.
Error (1): Metadata API request failed: force-app/main/default/staticresources/testBasic.resource-meta.xml: Expected source files for type 'StaticResource'

👎🏻 this PR sends the bad zip file to the API, get a deployment failure, and shows this error. I slightly prefer the old way (fail faster, especially if it's a large deployment)
Error (1): Metadata API request failed: force-app/main/default/staticresources/testBasic.resource-meta.xml: Expected source files for type 'StaticResource'

Still net improvement over archiver and happy to ship it.

@mshanemc mshanemc merged commit 03ff2df into main May 31, 2023
38 of 66 checks passed
@mshanemc mshanemc deleted the sh/archiver-to-jszip-swap branch May 31, 2023 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants