Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: run prettier in parallel in generate-clients #1949

Merged
merged 5 commits into from
Jan 22, 2021

Conversation

trivikr
Copy link
Member

@trivikr trivikr commented Jan 22, 2021

Issue #, if available:
Fixes: #1947

Description of changes:
runs prettier in parallel in generate-clients script by using MIT licensed third-party @mixer/parallel-prettier package

Metrics to generate all clients:

Existing setup takes ~360 seconds
$ time yarn generate-clients
...
...
Done in 359.39s.
433.54s user 13.15s system 124% cpu 5:59.52 total
Disabling prettier takes ~17 seconds
$ time yarn generate-clients
...
...
Done in 17.10s.
6.29s user 1.55s system 45% cpu 17.262 total
Using @mixer/parallel-prettier takes ~83 seconds
$ time yarn generate-clients
...
...
Done in 83.25s.
10.36s user 2.17s system 15% cpu 1:23.38 total

This improves time taken to generate all clients by ~76% on a 16 core CPU.

System Information
  • Operating System: Amazon Linux 2 x86_64
  • Host type: c5.4xlarge (16 Virtual CPUs, 32.0 GiB Memory, 160 GiB SSD (EBS) Storage)
  • Node.js version: v14.15.4

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@codecov-io
Copy link

codecov-io commented Jan 22, 2021

Codecov Report

Merging #1949 (d5bb407) into master (f2a47e8) will increase coverage by 0.52%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1949      +/-   ##
==========================================
+ Coverage   79.30%   79.83%   +0.52%     
==========================================
  Files         368      368              
  Lines       15132    15553     +421     
  Branches     3222     3368     +146     
==========================================
+ Hits        12001    12417     +416     
- Misses       3131     3136       +5     
Impacted Files Coverage Δ
packages/util-waiter/src/utils/validate.ts 81.81% <0.00%> (-18.19%) ⬇️
lib/storage/src/data-chunk/readable-helper.ts 86.84% <0.00%> (-5.47%) ⬇️
packages/s3-request-presigner/src/getSignedUrl.ts 92.85% <0.00%> (-2.98%) ⬇️
packages/util-dynamodb/src/convertToAttr.ts 98.70% <0.00%> (-1.30%) ⬇️
packages/util-waiter/src/poller.ts 100.00% <0.00%> (ø)
packages/util-waiter/src/waiter.ts 100.00% <0.00%> (ø)
protocol_tests/aws-ec2/endpoints.ts 81.48% <0.00%> (ø)
protocol_tests/aws-json/endpoints.ts 81.48% <0.00%> (ø)
protocol_tests/aws-query/endpoints.ts 81.48% <0.00%> (ø)
packages/util-dynamodb/src/marshall.ts 100.00% <0.00%> (ø)
... and 156 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9e4b2d6...d5bb407. Read the comment docs.

@aws-sdk-js-automation
Copy link

AWS CodeBuild CI Report

  • CodeBuild project: sdk-staging-test
  • Commit ID: d5bb407
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@trivikr trivikr merged commit 878617a into aws:master Jan 22, 2021
@trivikr trivikr deleted the parallel-prettier branch January 22, 2021 18:32
@github-actions
Copy link

github-actions bot commented Feb 6, 2021

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Running prettify on generate-clients takes ~80% of the time
4 participants