Optimization: Images Processing

[robertu7]

Background

We have lots of images uploaded by users: avatars, profile covers, article embedded images, and etc. Based on Lighthouse Report, images have big negative impact for performance score:

In general, there are three sides to be optimzed:

Source

• Compressing
• Resizing: different sizes fit different needs
• Formats: WebP, JPEG 2000, etc.
• Progressive

Application

• Lazy Loading Images: Lazy Loading matters-web#1159
• Responsive Images

Proxy

• CDN
• Cache Policy ^[1]

Solutions

Currently, we will do simple image processing (compressing & resizing) in connectors/aws, while image uploading. But there are cons:

1. It's synchronous, uploading is blocked by processing;
2. Growing complexity, we need add more and more codes for above needs;

Lambda To Rescue!

With AWS Lambda, processing image will be asynchronously and separately. There are two ways to implement:

1) Lazy processing

Use Serverless Image Handler, process image on client requests, and cached by CDN.

Pro & Cons:

• Lower S3 cost, only store raw images;
• Higher CDN cost by low hit rate;
• Longer response time, process on-demand;

2) Post-processing

1. The client calls the API to upload image, server forward it to AWS S3 directly, now raw image is accessible.
2. AWS Lambda post-process the raw image from AWS S3, asynchronously, now optimized images are accessible.

Steps

1. Formating: WebP
2. Resizing
   • avatar: raw, 144w
   • embed raw, 1080w, 540w, 360w, 144w
   • profileCover: raw, 1080w, 540w
3. Compressing

Results

# AWS S3 
/matters-server-stage
├── 1080w
│   ├── uuid.jpeg
│   └── uuid.webp
├── 540w
│   ├── uuid.jpeg
│   └── uuid.webp
├── 360w
│   ├── uuid.jpeg
│   └── uuid.webp
├── 144w
│   ├── uuid.jpeg
│   └── uuid.webp
└── uuid.jpeg

Usage

<!-- <ArticleDetail.Content> -->
<figure>
  <picture>
    <source type="image/webp" media="(min-width: 768px)" srcset="https://xxx.cloudfront.net/embed/1080w/uuid.webp" alt="...">
    <source type="image/webp" srcset="https://xxx.cloudfront.net/embed/540w/uuid.webp" alt="...">
    <source media="(min-width: 768px)" srcset="https://xxx.cloudfront.net/embed/1080w/uuid.jpeg" alt="...">
    <img src="https://xxx.cloudfront.net/embed/540w/uuid.jpeg" alt="...">
  </picture>
  <figcaption>...</figcaption>
</figure>

<!-- <ArticleDigest.Cover>, View Mode = default -->
<picture>
  <source type="image/webp" media="(min-width: 768px)" srcset="https://xxx.cloudfront.net/embed/1080w/uuid.webp" alt="...">
  <source type="image/webp" srcset="https://xxx.cloudfront.net/embed/540w/uuid.webp" alt="...">
  <source media="(min-width: 768px)" srcset="https://xxx.cloudfront.net/embed/1080w/uuid.jpeg" alt="...">
  <img src="https://xxx.cloudfront.net/embed/540w/uuid.jpeg" alt="...">
</picture>

<!-- <ArticleDigest.Cover>, View Mode = compact -->
<picture>
  <source type="image/webp" media="(min-width: 768px)" srcset="https://xxx.cloudfront.net/embed/360w/uuid.webp" alt="...">
  <source type="image/webp" srcset="https://xxx.cloudfront.net/embed/144w/uuid.webp" alt="...">
  <source media="(min-width: 768px)" srcset="https://xxx.cloudfront.net/embed/360w/uuid.jpeg" alt="...">
  <img src="https://xxx.cloudfront.net/embed/144w/uuid.jpeg" alt="...">
</picture>

<!-- <UserProfile.Cover> -->
<picture>
  <source type="image/webp" media="(min-width: 768px)" srcset="https://xxx.cloudfront.net/profileCover/1080w/uuid.webp" alt="...">
  <source type="image/webp" srcset="https://xxx.cloudfront.net/profileCover/540w/uuid.webp" alt="...">
  <source media="(min-width: 768px)" srcset="https://xxx.cloudfront.net/profileCover/1080w/uuid.jpeg" alt="...">
  <img src="https://xxx.cloudfront.net/profileCover/540w/uuid.jpeg" alt="...">
</picture>

<!-- <Avatar> -->
<picture>
  <source type="image/webp" srcset="https://xxx.cloudfront.net/avatar/144w/uuid.webp" alt="...">
  <img src="https://xxx.cloudfront.net/profileCover/144w/uuid.jpeg" alt="...">
</picture>

---

^[1] We do had set a long cache TTL in CloudFront, but LightHouse doesn't think so.
^[2] https://css-tricks.com/responsive-images-css/
^[3] https://dev.to/jsco/a-comprehensive-guide-to-responsive-images-picture-srcset-source-etc-4adj
^[4] https://css-tricks.com/using-webp-images/#article-header-id-3

[guoliu]

Great summary! One question I have in mind is, what do we display in editor after user upload image? Do we need to wait for image process to insert the image path, or should we get some placeholder first and replace it later?

[robertu7]

Optimization: Image Processing

[robertu7]

(5) runtime performance profiling

[robertu7]

Fallback image for broken link: https://medium.com/@webcore1/react-fallback-for-broken-images-strategy-a8dfa9c1be1e

[robertu7]

MIgration: https://aws.amazon.com/blogs/big-data/building-scalable-and-responsive-big-data-interfaces-with-aws-lambda/

[robertu7]

Implemented by https://github.com/thematters/serverless-file-post-processing