Skip to content

Latest commit

 

History

History
153 lines (124 loc) · 5.41 KB

s3.md

File metadata and controls

153 lines (124 loc) · 5.41 KB
Raw

S3

Action uses Amazon S3 for three purposes:

  1. Serving manually managed assets, such as logos
  2. Serving builds bundled with webpack
  3. Storing and serving user-generated assets

Action uses a single S3 bucket. Within the bucket, the directory structure is as follows:

   S3 Bucket
   =========
   |
   |--- static/                     1) Manually managed assets
   |
   |--- :instance/
      |--- build/                   2) Builds bundled with webpack
      |  |--- :vX.Y.Z/
      |
      |--- store/                   3) User-generated assets
         |--- :model/
            |--- :id/
               |--- :field/
                  |--- :asset.ext

Deploying Builds to S3

Action builds are transferred to S3 automatically using S3Plugin for webpack and the command $ npm run build:client-deploy. The S3 configuration is managed using a set of environment variables:

AWS_ACCESS_KEY_ID="XXXXXXXXXXXXXXXXXXXX" AWS_REGION="us-east-1" # for example AWS_S3_BUCKET="some-bucket-name" AWS_SECRET_ACCESS_KEY="YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY" CDN_BASE_URL="//some.url.com/instance" # (development|staging|production)

It's recommended that you store these environment variables locally (never check in secret information!) and sequence your builds thusly:

$ cp ~/environments/development.env .env
$ npm run build:deploy

You should see output like this:

> parabol-action@0.13.6 build:deploy /Users/jordanhusney/Source/Repositories/github.com/Parabol/action.git
> rimraf build && npm run build:server && npm run build:client-deploy
...
Uploading [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 100% 0.0s
Hash: 1f0b7bfe7fb69b086386
Version: webpack 2.1.0-beta.25
Time: 113568ms

How User Assets Are Uploaded & Served

A client is able to directly upload and access assets securely on an instance within an S3 bucket. "Directly," means that assets do not need to pass through the server in order to be uploaded or downloaded from S3, increasing speed and decreasing bandwidth. Convenience functions to facilitate access to S3 are defined within server/utils/s3.js.

To illustrate how this scheme works, we will document the call chain necessary to upload and later access a user avatar image.

Example: uploading and accessing a user avatar image

Let's assume the client has an ECMAScript File object that holds the user's filename and data for an avatar image. We'll refer to this data as file.jpg. The client wants to upload this image to S3.

  1. A PUT URL is requested from the server by calling the createUserPicturePutUrl mutation (which calls s3SignPutUrl on the server). This URL is signed with the server's S3 secret. It must be used within the expiration window (60 seconds, by default). The URL generated by the Action server's graphql model is unique and is formatted to contain the proper :model/:id/:field/:fileid.ext components. The extension is extracted from the user's filename.

  2. The client then HTTP PUTs user's image file data to S3 using the URL returned from the Action using the fetch API.

  3. If the file upload is successful, the user's profile is updated with the URL of the new avatar image on S3 by calling the updateUserProfile mutation.

  4. If the previous version of the user profile image was also hosted on S3, the server attempts to asynchronously delete the old image.

Controlling access to uploaded assets

By default, assets uploaded to S3 use the authenticated-read S3 ACL.

This means, in order for a client to access these assets they must request a signed GetObject URL from the server. In some instances, it will make sense for the server to generate these URLs when retrieved from the server's database. In other instances, it'll make sense to generate these URLs shortly before the client attempts to download the asset.

In other cases, such as the hosting of avatar images, it will make sense to create the objects on S3 with a more permissive ACL so the object can be accessed publicly, without authentication. In this instance the public-read ACL may be used.

Securing the S3 Bucket and instance itself

The AWS IAM user accessing S3 should only have permission to access its instance's assets within the bucket and nothing else via an IAM policy.

Below is an example policy configuration that achieves this:

{
 "Version":"2012-10-17",
 "Statement": [
   {
     "Sid": "AllowUserToSeeBucketListInTheConsole",
     "Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::*"]
   },
  {
     "Sid": "AllowRootListing",
     "Action": ["s3:ListBucket"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::action-files.parabol.co"],
     "Condition":{"StringEquals":{"s3:prefix":[""],"s3:delimiter":["/"]}}
    },
   {
     "Sid": "AllowListingOfInstanceFolder",
     "Action": ["s3:ListBucket"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::action-files.parabol.co"],
     "Condition":{"StringLike":{"s3:prefix":["development/*"]}}
   },
   {
     "Sid": "AllowAllS3ActionsInInstanceFolder",
     "Effect": "Allow",
     "Action": ["s3:*"],
     "Resource": ["arn:aws:s3:::action-files.parabol.co/development/*"]
   }
 ]
}