support: Upgrade aws-sdk to v3 #5863

mudana-grune · 2022-05-18T10:01:00Z

https://youtrack.weseek.co.jp/issue/GW-7807

Upgrade package to aws-sdk v3
Make async / await function available
Change syntax to typescript

Testing Steps

Configure AWS

Configure AWS in File upload settings (admin/app)

Upload attachment from wiki page

Upload attachment from wiki page and make sure attachment displayed in the preview

Check url of the attchment (make sure the url already points to the specified url bucket and make sure we can download the attachment)
Download and remove attachment from attachment data (make sure download and delete atachment success without error)

Upload Attachment from user profile picture setting

Make sure upload and delete profile picture is success in Set Profile Picture
Check profile picture source and make sure the url already points to specified bucket url

https://youtrack.weseek.co.jp/issue/GW-7758 - Install @aws-sdk/client-s3 - Convert file-uploader aws to typescript (on progress) - Install @aws-sdk/s3-request-presigner to get signedUrl

https://youtrack.weseek.co.jp/issue/GW-7758 - Add delete file and upload file - Add forcePathStyle in aws config

https://youtrack.weseek.co.jp/issue/GW-7631 - Remove aws.js - Remove Expires options in getSignedUrl function param - Update getSignedUrl, add expiresIn option

https://youtrack.weseek.co.jp/issue/GW-7631 - Implement deleteFileByFilePath

…ptize-aws

hakumizuki · 2022-05-30T07:01:20Z

packages/app/src/server/service/file-uploader/aws.ts

@@ -207,7 +212,7 @@ module.exports = function(crowi) {

    let stream;
    try {
-      stream = s3.getObject(params).createReadStream();
+      stream = s3.send(new GetObjectCommand(params));


The return type of S3Client.send is not Readable.

I think s3.send(new GetObjectCommand(params)).Body is what we want here. Please check and change if necessary.

See:

export interface GetObjectOutput { Body?: Readable | ReadableStream | Blob; ... }

Updated to return stream.Body

hakumizuki · 2022-05-30T08:55:56Z

I have also improved type system code in this commit. Please check this as well.

…ptize-aws

https://youtrack.weseek.co.jp/issue/GW-7807 - Change return type of findDeliveryFile method - Remove GetObjectCommandOutput import

https://youtrack.weseek.co.jp/issue/GW-7807 - Update return type of findDeliveryFilemethod

hakumizuki

Overall, most of your code looks good.
To close this PR, please make sure all the apis works correctly.

hakumizuki · 2022-05-30T11:46:10Z

packages/app/src/server/service/file-uploader/aws.ts

    }
    catch (err) {
      logger.error(err);
      throw new Error(`Coudn't get file from AWS for the Attachment (${attachment._id.toString()})`);
    }

    // return stream.Readable
-    return stream;
+    return stream.Body;


You should get the Body attribute at L216 so the variable name matches to its value.

!! Important !!

Please make sure the findDeliveryFile method works correctly.
Especially, please test that the app.get('/download/:id([0-9a-z]{24})' , loginRequired, attachment.api.download); endpoint works correctly. Please write down what and how you checked this as well.

Extract Body attribute from GetObjectCommandOutput as type for stream variable type

Upload , delete and get attachment already woking properly

Download testing

Open attachment data

Attachment list will shown

Click download button

Files will be downloaded properly

オリジナルのファイル名でダウンロードできないといけないのでは？

routes/attachment.js には

'Content-Disposition': `attachment;filename*=UTF-8''${encodeURIComponent(attachment.originalName)}`

をセットしてる箇所があるので。

https://youtrack.weseek.co.jp/issue/GW-7807 - Extract Body attribute from GetObjectCommandOutput Interface as type

…ptize-aws

reg-suit · 2022-05-31T05:56:56Z

✨✨ That's perfect, there is no visual difference! ✨✨

Check out the report here.

mudana-grune and others added 5 commits April 5, 2022 17:40

Install @aws-sdk/client-s3

b25b4ba

https://youtrack.weseek.co.jp/issue/GW-7758 - Install @aws-sdk/client-s3 - Convert file-uploader aws to typescript (on progress) - Install @aws-sdk/s3-request-presigner to get signedUrl

implement aws sdk v3

9b4e359

https://youtrack.weseek.co.jp/issue/GW-7758 - Add delete file and upload file - Add forcePathStyle in aws config

implement aws sdk v3

8c6b240

https://youtrack.weseek.co.jp/issue/GW-7631 - Remove aws.js - Remove Expires options in getSignedUrl function param - Update getSignedUrl, add expiresIn option

Implement deleteFileByFilePath

51387e8

https://youtrack.weseek.co.jp/issue/GW-7631 - Implement deleteFileByFilePath

Merge branch 'master' into feat/gw7631-upgrade-aws-to-v3-and-typescri…

e74b078

…ptize-aws

github-actions bot added the type/feature label May 18, 2022

mudana-grune self-assigned this May 18, 2022

mudana-grune requested a review from hakumizuki May 18, 2022 10:03

improve writing style

0617aca

mudana-grune temporarily deployed to VRT May 18, 2022 10:19 Inactive

Merge branch 'master' into feat/gw7807-upgrade-aws-to-v3-and-typescri…

bff8d51

…ptize-aws

hakumizuki temporarily deployed to VRT May 24, 2022 05:45 Inactive

hakumizuki suggested changes May 30, 2022

View reviewed changes

Improved type system & Ensure return value compatibility

b1f22ab

hakumizuki temporarily deployed to VRT May 30, 2022 08:58 Inactive

Merge branch 'master' into feat/gw7807-upgrade-aws-to-v3-and-typescri…

87f0c65

…ptize-aws

hakumizuki temporarily deployed to VRT May 30, 2022 09:14 Inactive

Code improvement of findDeliveryFile

bcd2315

https://youtrack.weseek.co.jp/issue/GW-7807 - Change return type of findDeliveryFile method - Remove GetObjectCommandOutput import

mudana-grune temporarily deployed to VRT May 30, 2022 11:25 Inactive

Code improvement

8725e4b

https://youtrack.weseek.co.jp/issue/GW-7807 - Update return type of findDeliveryFilemethod

mudana-grune temporarily deployed to VRT May 30, 2022 11:42 Inactive

hakumizuki suggested changes May 30, 2022

View reviewed changes

Code improvement

3800929

https://youtrack.weseek.co.jp/issue/GW-7807 - Extract Body attribute from GetObjectCommandOutput Interface as type

mudana-grune temporarily deployed to VRT May 31, 2022 04:25 Inactive

Merge branch 'master' into feat/gw7807-upgrade-aws-to-v3-and-typescri…

84652bc

…ptize-aws

hakumizuki self-requested a review May 31, 2022 05:45

hakumizuki approved these changes May 31, 2022

View reviewed changes

hakumizuki temporarily deployed to VRT May 31, 2022 05:52 Inactive

yuki-takei changed the title ~~feat: gw7807 upgrade aws to v3 and typescriptize aws~~ support: Upgrade aws-sdk to v3 May 31, 2022

yuki-takei added type/support and removed type/feature labels May 31, 2022

yuki-takei merged commit a550bc3 into master May 31, 2022

yuki-takei deleted the feat/gw7807-upgrade-aws-to-v3-and-typescriptize-aws branch May 31, 2022 07:00

github-actions bot mentioned this pull request May 31, 2022

Release v5.0.8 #5932

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support: Upgrade aws-sdk to v3 #5863

support: Upgrade aws-sdk to v3 #5863

mudana-grune commented May 18, 2022

hakumizuki May 30, 2022

mudana-grune May 30, 2022 •

edited

hakumizuki commented May 30, 2022

hakumizuki left a comment

hakumizuki May 30, 2022

hakumizuki May 30, 2022

mudana-grune May 31, 2022

mudana-grune May 31, 2022

yuki-takei May 31, 2022

reg-suit bot commented May 31, 2022

support: Upgrade aws-sdk to v3 #5863

support: Upgrade aws-sdk to v3 #5863

Conversation

mudana-grune commented May 18, 2022

Testing Steps

Configure AWS

Upload attachment from wiki page

Upload Attachment from user profile picture setting

hakumizuki May 30, 2022

Choose a reason for hiding this comment

mudana-grune May 30, 2022 • edited

Choose a reason for hiding this comment

hakumizuki commented May 30, 2022

hakumizuki left a comment

Choose a reason for hiding this comment

hakumizuki May 30, 2022

Choose a reason for hiding this comment

hakumizuki May 30, 2022

Choose a reason for hiding this comment

mudana-grune May 31, 2022

Choose a reason for hiding this comment

mudana-grune May 31, 2022

Choose a reason for hiding this comment

Download testing

yuki-takei May 31, 2022

Choose a reason for hiding this comment

reg-suit bot commented May 31, 2022

mudana-grune May 30, 2022 •

edited