file size affecting upload order when uploading array of files/images to s3 - NextJS

[hampak]

Hello, I'm currently accepting multiple images/files from the frontend and then passing them to the backend(route handler). I store the files in an array - in the order which I would like to upload. However, the code I have right now doesn't seem to upload them "one by one".

For example, if I have an array of files like this (assume that the name of the files are also the size of the files): [7mb, 24kb, 100kb], I would loop over this array and expect that the createdAt date is the order of 7mb -> 24kb -> 100kb. So for example, the last modified date in s3 would be something like this:

7mb -> 2023-10-3 21:00:04
24kb -> 2023-10-3 21:01:10
100kb -> 2023-10-3 21:01:50

However, the code I have right now would upload them like this: 24kb -> 100kb -> 7mb which is the order of file size (because the small file takes less time to upload even though it's at the end of the array).

const uploadImages = async (file: any) => {

      const imageName = randomBytes(16).toString("hex")

      const command = new PutObjectCommand({
        Bucket: process.env.AWS_BUCKET_NAME,
        Key: imageName,
        ContentType: file.type
      })

      const preSignedUrl = await getSignedUrl(client, command, {
        expiresIn: 1000
      })

      const upload = await fetch(preSignedUrl, {
        method: "PUT",
        body: file,
        headers: { "Content-Type": file.type }
      })

      if (upload.ok) return new Response("OK")
    }

const photoUrls = photosImagesArray.map((file) => {
  uploadImages(file)
})

Q. Would there be some kind of way to wait until the upload of the previous file/image to be completed before starting the upload process of the next one?
Q. If not, should I use Promise in this case?

Much help would be appreciated! I'm using Next 13.5 btw

[ajredniwja]

Hey @hampak map is not waiting for uploadImages. You can try something like this:

const uploadImages = async(file) => {
    const imageName = randomBytes(16).toString("hex")

    const command = new PutObjectCommand({
        Bucket: process.env.AWS_BUCKET_NAME,
        Key: imageName,
        ContentType: file.type
    })

    const preSignedUrl = await getSignedUrl(client, command, {
        expiresIn: 1000
    })

    const upload = await fetch(preSignedUrl, {
        method: "PUT",
        body: file,
        headers: { "Content-Type": file.type }
    })

    if (upload.ok) return new Response("OK")
};

const uploadAllImages = async(photosImagesArray) => {
    for (let i = 0; i < photosImagesArray.length; i++) {
        await uploadImages(photosImagesArray[i]);
        console.log(`Image at index ${i} has been uploaded.`);
    }
};

uploadAllImages(photosImagesArray);

@github-actions proposed-answer

[github-actions[bot]]

Hello! A team member has marked the above comment as the likely answer to this discussion thread.

• If you agree, please upvote that comment, or click on Mark as answer. I will automatically mark the comment as the answer next time I check.

• If this answer does not help you, please downvote the answer instead and let us know why it was not helpful. I will add a label to this discussion to gain attention from the team.

[hampak]

@ajredniwja Thank you very much for your suggestion, will try it out :)

[hampak]

@ajredniwja Tested it and it works, thank you very much :)

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.