Crystal on AWS

See Also: This article describes how to use a remote version of Crystal to open a remote project on AWS S3. If you instead want to run a local version of Crystal that opens a project on AWS S3, see Projects on AWS S3 instead.

Crystal can serve an archived website directly from AWS, making it accessible to anyone with a web browser; no need to install Crystal locally. This uses AWS Lambda to run Crystal and Amazon S3 to store the project.

Prerequisites

1. An AWS account. Create one if you don't have one already.

2. A .crystalproj uploaded to S3. Create an S3 bucket and upload your project directory to it. See Projects on AWS S3 for details on uploading, including the recommended Pack16 format conversion.

Tip: After uploading, copy the S3 URL of your .crystalproj from the S3 Console; you'll need it in the next step. It looks like: s3://my-bucket/My Site.crystalproj

Deploy

Option 1: Quick Deploy (recommended)

Click one of the buttons below to open the AWS CloudFormation Console with the Crystal template pre-loaded:

Stable (latest release)
Dev (main branch, may be unstable)

Then:

1. Stack name: Choose a name for this deployment (e.g. crystal-my-site). Each archived site you deploy needs its own stack name.

2. Project S3 URL: Paste the S3 URL of your .crystalproj (e.g. s3://my-bucket/My Site.crystalproj). You can copy this directly from the S3 Console.

3. (Optional) Under Password Protection, enter a Username and Password to restrict access to the site. Leave both blank for public access.

4. Leave the Advanced Options at their defaults.

5. Scroll down and check the box "I acknowledge that AWS CloudFormation might create IAM resources with custom names." This is required because the template creates an IAM role that allows the Lambda function to read your project from S3.

6. Click Create stack.

7. Wait for the stack status to reach CREATE_COMPLETE (typically 2-4 minutes).

8. Go to the Outputs tab. The SiteUrl1 value is the public URL where your archived site is now accessible.

Option 2: Manual Deploy

If you prefer to inspect the template before deploying, or if the Quick Deploy links above don't work for your region:

1. Download the CloudFormation template: crystal.cloudformation.yaml

2. Open the AWS CloudFormation Console.

3. Click Create stack > With new resources (standard).

4. Select Upload a template file, choose the downloaded crystal.cloudformation.yaml, and click Next.

5. Follow the same steps as Quick Deploy above (fill in stack name, Project S3 URL, acknowledge IAM, create).

(Optional) Add a Custom Domain

By default, your site is served at an auto-generated URL like https://abc123.lambda-url.us-east-2.on.aws/. To use your own domain (e.g. mysite.example.com), you can add a custom domain using AWS CloudShell, a browser-based terminal built into the AWS Console that requires no local setup.

Adding a custom domain

Prerequisites: A domain you own, with access to manage its DNS records at your registrar.

1. Open AWS CloudShell.

2. In the top-right corner of the page, select the region matching the one your Crystal stack is deployed in (e.g. "US East (Ohio) us-east-2").

3. Paste and run this command, replacing <stack-name> and <domain> with your values:

   curl -sL https://raw.githubusercontent.com/davidfstr/Crystal-Web-Archiver/main/src/crystal_on_aws/custom-domain-add.sh | bash -s <stack-name> <domain>
   

   For example:

   curl -sL https://raw.githubusercontent.com/davidfstr/Crystal-Web-Archiver/main/src/crystal_on_aws/custom-domain-add.sh | bash -s crystal-my-site mysite.example.com
   

4. The script will display two CNAME records to add at your domain registrar. Add both records, then press Enter in CloudShell to continue.

5. Wait for the script to finish (typically 5-10 minutes). When complete, your site will be live at https://mysite.example.com/.

Removing a custom domain

1. Open AWS CloudShell and select the correct region.

2. Paste and run:

   curl -sL https://raw.githubusercontent.com/davidfstr/Crystal-Web-Archiver/main/src/crystal_on_aws/custom-domain-remove.sh | bash -s <stack-name>
   

3. The script will list the resources to be deleted and ask for confirmation. After completion, it will remind you to remove the CNAME records at your registrar.

(Optional) Password-Protect the Site

You can restrict access to your archived site by requiring visitors to enter a username and password (using HTTP Basic Auth). Since the site is served over HTTPS, credentials are encrypted in transit.

1. Open your stack in the CloudFormation Console.
2. Click Update > Use current template > Next.
3. Under Password Protection, enter a Username and Password.
4. Click through Next > Next > check the IAM acknowledgement > Submit.

Visitors will now be prompted for credentials when they access the site. To remove password protection, update the stack and clear both fields.

Managing Your Deployment

Updating to a new Crystal version

When a new version of Crystal is released, the public container image (latest tag) is updated automatically. To pick up the new version:

1. Open your stack in the CloudFormation Console.
2. Click Update > Use current template > Next.
3. Change the Image Refresh Token field (under Advanced Options) to any new value (e.g. 2 or today's date).
4. Click through Next > Next > check the IAM acknowledgement > Submit.

The stack update will pull the latest Crystal image. This typically takes 2-4 minutes.

Changing the project URL

To point the deployment at a different .crystalproj:

1. Open your stack in the CloudFormation Console.
2. Click Update > Use current template > Next.
3. Change the Project S3 URL to the new S3 URL.
4. Click through to submit.

Deleting the deployment

1. If you added a custom domain, remove it first.
2. Open your stack in the CloudFormation Console.
3. Click Delete.
4. Confirm the deletion. All AWS resources created by the stack will be removed.

Note: This does not delete your .crystalproj from S3. Your archived project data is preserved.

What Gets Created

The CloudFormation template creates the following AWS resources:

Resource	Purpose
Lambda Function	Runs Crystal to serve your archived site
Function URL	Provides a public HTTPS endpoint for the Lambda function
IAM Role	Grants the Lambda function permission to read your project from S3 and write logs to CloudWatch
URL Parser (helper Lambda)	Parses the S3 URL you provide into bucket name and key path (needed because CloudFormation's built-in string functions are limited)
Image Copier (helper Lambda)	Copies Crystal's public container image to a private ECR repository in your account (needed because Lambda can only run images from private ECR)
ECR Repository	Stores the private copy of the Crystal container image (crystal-on-aws)
CloudWatch Log Groups	Stores logs from the Lambda functions (auto-expires after 7 days)

When a custom domain is added, the custom-domain-add.sh script additionally creates:

Resource	Purpose
CloudFront Distribution	Routes traffic from your custom domain to the Lambda Function URL
ACM Certificate	Provides HTTPS for your custom domain

Cost

Crystal on AWS is designed to be inexpensive for typical archival use (low-traffic sites served to occasional visitors).

The main cost components are:

• Lambda: Billed per request and per GB-second of compute time. A typical page load takes a few hundred milliseconds. Lambda has a generous free tier (1M requests/month).
• S3: Billed for storage and GET requests. Crystal reads objects on-demand as visitors browse.
• ECR: Billed for storage of the private container image copy (~350 MB, about $0.04/month).
• CloudWatch Logs: Log groups are configured with 7-day retention to limit costs.
• CloudFront (custom domain only): Billed per request and per GB of data transfer. Has a free tier (1 TB/month).

For a low-traffic archived site, expect costs on the order of a few cents per month beyond S3 storage for the project itself. The main ongoing cost is typically S3 storage for your .crystalproj.

Local Development

This section covers workflows for developers working on the Crystal on AWS codebase itself, using scripts in src/crystal_on_aws/.

Pushing Crystal code changes

Use deploy-image.sh to build a new container image and push it to ECR. It automatically refreshes all Lambda functions whose stacks reference the pushed image.

# Build and push to your private ECR; refresh all stacks using it
src/crystal_on_aws/deploy-image.sh

# Build and push to public ECR with the "dev" tag; refresh all stacks using it
src/crystal_on_aws/deploy-image.sh --public --dev

# Build and push to public ECR with the release tag + "latest"; refresh all stacks using it
src/crystal_on_aws/deploy-image.sh --public --release

For local development it is frequently useful to build/push Crystal code changes to a private ECR image. After your initial push, use switch-stack-version.sh (mentioned below) to switch one of your existing stacks to use your private ECR version of Crystal.

Pushing CloudFormation template changes

To a privately-deployed stack

There is no deploy-cf-template.sh target for private stacks. Update the template directly via the AWS Console or CLI:

AWS Console: Open the stack → Update → Replace current template → upload src/crystal_on_aws/crystal.cloudformation.yaml → keep all existing parameter values → submit.

CLI:

STACK_NAME=<stack-name>
PARAM_KEYS=$(aws cloudformation describe-stacks \
    --stack-name "${STACK_NAME}" \
    --query 'Stacks[0].Parameters[].ParameterKey' \
    --output text)
PARAMS=()
for KEY in ${PARAM_KEYS}; do
    PARAMS+=("ParameterKey=${KEY},UsePreviousValue=true")
done
aws cloudformation update-stack \
    --stack-name "${STACK_NAME}" \
    --template-body file://src/crystal_on_aws/crystal.cloudformation.yaml \
    --parameters "${PARAMS[@]}" \
    --capabilities CAPABILITY_NAMED_IAM

To the public dev or release template

Use deploy-cf-template.sh to upload the template to S3, where the "Launch Stack" buttons reference it:

src/crystal_on_aws/deploy-cf-template.sh --public --dev      # Updates the Dev launch button
src/crystal_on_aws/deploy-cf-template.sh --public --release  # Updates the Stable launch button

Note: This only uploads the template to S3. It does not update any already-deployed stacks. Each existing stack continues to use its own copy of the template until manually updated.

Switching an existing stack to a different image version

Use switch-stack-version.sh to point an existing CloudFormation stack at a different Crystal image — without rebuilding or redeploying anything:

# Switch to your locally-built private ECR image
src/crystal_on_aws/switch-stack-version.sh <stack-name>

# Switch to the public dev image (main branch)
src/crystal_on_aws/switch-stack-version.sh <stack-name> --public --dev

# Switch to the public release image (latest stable)
src/crystal_on_aws/switch-stack-version.sh <stack-name> --public --release

ImageRefreshToken is always bumped, so the stack will re-pull the image even if the tag hasn't changed.