Skip to content
Multipart + Presigned URL upload to AWS S3/Minio via the browser
Branch: master
Clone or download
prestonlimlianjie Merge pull request #2 from prestonlimlianjie/bug-fix
fixes math.round bug described in issue #1
Latest commit 3615aee Feb 18, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
backend
frontend fixes math.round bug Feb 18, 2019
.gitignore first commit Dec 13, 2018
README.md Update README.md Dec 13, 2018

README.md

Multipart + Presigned URL upload to AWS S3/Minio via the browser

Motivation

I created this demo repo because documentation for multipart uploading of large files using presigned URLs was very scant.

I wanted to create a solution to allow users to upload files directly from the browser to AWS S3 (or any S3-compliant storage server). This worked great when I used AWS SDK's getSignedUrl API to generate a temporary URL that the browser could upload the file to.

However, I hit a snag when dealing with files > 5GB because the pre-signed URL only allows for a maximum file size of 5GB to be uploaded at one go. As such, this repo demonstrates the use of multipart + presigned URLs to upload large files to an AWS S3-compliant storage service.

Components used in this demo

  • Frontend Server: React (Next.js)
  • Backend Server: Node.js (Express), using the AWS JS SDK
  • Storage Server: Minio (but this can easily be switched out to AWS S3)

How to run

  • Clone the repo and change directory into the repo
  • Open three different terminal windows.

Storage Server

In window 1, run:

# Set up the Minio server (ignore this if you are using AWS S3)
# Minio docs: https://docs.minio.io/docs/minio-quickstart-guide
minio server /data

Note: Set the S3-compliant bucket policy as appropriate to allow the right access

Backend Server

Replace the following code in backend/server.js with your AWS S3 or S3-compliant storage server config.

const s3  = new AWS.S3({
  accessKeyId: '<ACCESS_KEY_ID>' , // Replace with your access key id
  secretAccessKey: '<SECRET_ACCESS_KEY>' , // Replace with your secret access key
  endpoint: 'http://127.0.0.1:9000' ,
  s3ForcePathStyle: true, // needed with minio?
  signatureVersion: 'v4'
});

Note: If you are using AWS S3, follow the docs on the AWS website to instantiate a new AWS S3 client.

In window 2, run:

cd backend
npm install
node server.js

Frontend Server

In window 3, run:

cd frontend
npm install
npm run dev

Upload File

Go to http://localhost:3000 in your browser window and upload a file.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.