Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media: Convert HEIF images to WebP/JPEG #13015

Merged
merged 16 commits into from Feb 14, 2023
Merged

Media: Convert HEIF images to WebP/JPEG #13015

merged 16 commits into from Feb 14, 2023

Conversation

swissspidy
Copy link
Collaborator

@swissspidy swissspidy commented Feb 13, 2023

Context

Photos taken on iPhone typically have an .heic file extension. The HEIF/HEIC image format is great because it's super small, but it's not supported by any browsers.

Summary

This change implements HEIC image decoding via a simple web worker a script that loads libheif-js from an external script that then converts the image to WebP via Canvas.

Safari does not support WebP in Canvas though, so there it falls back to JPEG.

Relevant Technical Choices

libheif-js uses LGPL 3.0, which is why we don't bundle it in the plugin but instead load it dynamically.

We can't use it in a worker because of cross-origin isolation limitations.

So the whole decoding happens in the main thread. In my testing it was usually quite fast, so probably not a big deal.

To-do

  • Load libheif-js from CDN

User-facing changes

Screen.Recording.2023-02-13.at.17.56.10.mov

Testing Instructions

  • This is a non-user-facing change and requires no QA

This PR can be tested by following these steps:

  1. Try uploading some HEIC images to the editor via drag & drop
    I used some from https://github.com/alexcorvi/heic2any/tree/master/demo
  2. Verify the images are converted to WebP/JPEG and work fine when previewing the story.

Reviews

Does this PR have a security-related impact?

No

Does this PR change what data or activity we track or use?

No

Does this PR have a legal-related impact?

libheif-js uses LGPL 3.0, which is why we don't bundle it in the plugin but instead load it dynamically.

Checklist

  • This PR addresses an existing issue and I have linked this PR to it in ZenHub
  • I have tested this code to the best of my abilities
  • I have verified accessibility to the best of my abilities (docs)
  • I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
  • This code is covered by automated tests (unit, integration, and/or e2e) to verify it works as intended (docs)
  • I have added documentation where necessary
  • I have added a matching Type: XYZ label to the PR

Fixes #8292

@swissspidy swissspidy added Type: Enhancement New feature or improvement of an existing feature Group: Media labels Feb 13, 2023
@swissspidy swissspidy marked this pull request as ready for review February 14, 2023 15:05
@googleforcreators-bot
Copy link
Collaborator

Plugin builds for a88ab7b are ready 🛎️!

@github-actions
Copy link
Contributor

Size Change: +532 B (0%)

Total Size: 2.7 MB

Filename Size Change
assets/js/web-stories-editor.js 1.44 MB +532 B (0%)
ℹ️ View Unchanged
Filename Size
assets/css/web-stories-block-rtl.css 4.6 kB
assets/css/web-stories-block.css 4.64 kB
assets/css/web-stories-carousel-rtl.css 702 B
assets/css/web-stories-carousel.css 701 B
assets/css/web-stories-dashboard-rtl.css 657 B
assets/css/web-stories-dashboard.css 659 B
assets/css/web-stories-editor-rtl.css 769 B
assets/css/web-stories-editor.css 771 B
assets/css/web-stories-embed-rtl.css 676 B
assets/css/web-stories-embed.css 677 B
assets/css/web-stories-list-styles-rtl.css 2.39 kB
assets/css/web-stories-list-styles.css 2.42 kB
assets/css/web-stories-theme-style-twentyeleven-rtl.css 102 B
assets/css/web-stories-theme-style-twentyeleven.css 102 B
assets/css/web-stories-theme-style-twentyfifteen-rtl.css 251 B
assets/css/web-stories-theme-style-twentyfifteen.css 251 B
assets/css/web-stories-theme-style-twentyfourteen-rtl.css 287 B
assets/css/web-stories-theme-style-twentyfourteen.css 287 B
assets/css/web-stories-theme-style-twentyseventeen-rtl.css 288 B
assets/css/web-stories-theme-style-twentyseventeen.css 288 B
assets/css/web-stories-theme-style-twentysixteen-rtl.css 224 B
assets/css/web-stories-theme-style-twentysixteen.css 224 B
assets/css/web-stories-theme-style-twentyten-rtl.css 143 B
assets/css/web-stories-theme-style-twentyten.css 143 B
assets/css/web-stories-theme-style-twentytwelve-rtl.css 256 B
assets/css/web-stories-theme-style-twentytwelve.css 256 B
assets/css/web-stories-theme-style-twentytwenty-rtl.css 86 B
assets/css/web-stories-theme-style-twentytwenty.css 86 B
assets/css/web-stories-theme-style-twentytwentyone-rtl.css 326 B
assets/css/web-stories-theme-style-twentytwentyone.css 326 B
assets/css/web-stories-widget-rtl.css 482 B
assets/css/web-stories-widget.css 482 B
assets/js/2490.js 93.3 kB
assets/js/2630.js 38.6 kB
assets/js/3780.js 12.9 kB
assets/js/4422.js 49.3 kB
assets/js/7357.js 92 B
assets/js/7418.js 7.95 kB
assets/js/7480.js 12.8 kB
assets/js/9475.js 194 kB
assets/js/chunk-colorthief.js 2.62 kB
assets/js/chunk-ffmpeg.js 5.98 kB
assets/js/chunk-html-to-image.js 4.51 kB
assets/js/chunk-mediainfo.js 96 B
assets/js/chunk-opentype.js 96 B
assets/js/chunk-react-calendar.js 13.4 kB
assets/js/chunk-react-color.js 25.8 kB
assets/js/chunk-selfie-segmentation.js 16.3 kB
assets/js/chunk-web-stories-template-0-metaData.js 546 B
assets/js/chunk-web-stories-template-0.js 11.3 kB
assets/js/chunk-web-stories-template-1-metaData.js 540 B
assets/js/chunk-web-stories-template-1.js 9.56 kB
assets/js/chunk-web-stories-template-10-metaData.js 532 B
assets/js/chunk-web-stories-template-10.js 7.33 kB
assets/js/chunk-web-stories-template-11-metaData.js 539 B
assets/js/chunk-web-stories-template-11.js 9.01 kB
assets/js/chunk-web-stories-template-12-metaData.js 496 B
assets/js/chunk-web-stories-template-12.js 9.66 kB
assets/js/chunk-web-stories-template-13-metaData.js 525 B
assets/js/chunk-web-stories-template-13.js 7.36 kB
assets/js/chunk-web-stories-template-14-metaData.js 582 B
assets/js/chunk-web-stories-template-14.js 7.35 kB
assets/js/chunk-web-stories-template-15-metaData.js 544 B
assets/js/chunk-web-stories-template-15.js 9 kB
assets/js/chunk-web-stories-template-16-metaData.js 588 B
assets/js/chunk-web-stories-template-16.js 10.8 kB
assets/js/chunk-web-stories-template-17-metaData.js 540 B
assets/js/chunk-web-stories-template-17.js 9.18 kB
assets/js/chunk-web-stories-template-18-metaData.js 587 B
assets/js/chunk-web-stories-template-18.js 9.86 kB
assets/js/chunk-web-stories-template-19-metaData.js 501 B
assets/js/chunk-web-stories-template-19.js 9.42 kB
assets/js/chunk-web-stories-template-2-metaData.js 586 B
assets/js/chunk-web-stories-template-2.js 9.23 kB
assets/js/chunk-web-stories-template-20-metaData.js 548 B
assets/js/chunk-web-stories-template-20.js 8.97 kB
assets/js/chunk-web-stories-template-21-metaData.js 536 B
assets/js/chunk-web-stories-template-21.js 9.84 kB
assets/js/chunk-web-stories-template-22-metaData.js 525 B
assets/js/chunk-web-stories-template-22.js 7.78 kB
assets/js/chunk-web-stories-template-23-metaData.js 604 B
assets/js/chunk-web-stories-template-23.js 7.43 kB
assets/js/chunk-web-stories-template-24-metaData.js 517 B
assets/js/chunk-web-stories-template-24.js 11.6 kB
assets/js/chunk-web-stories-template-25-metaData.js 543 B
assets/js/chunk-web-stories-template-25.js 7.03 kB
assets/js/chunk-web-stories-template-26-metaData.js 600 B
assets/js/chunk-web-stories-template-26.js 7.24 kB
assets/js/chunk-web-stories-template-27-metaData.js 542 B
assets/js/chunk-web-stories-template-27.js 7.77 kB
assets/js/chunk-web-stories-template-28-metaData.js 532 B
assets/js/chunk-web-stories-template-28.js 9 kB
assets/js/chunk-web-stories-template-29-metaData.js 561 B
assets/js/chunk-web-stories-template-29.js 9.23 kB
assets/js/chunk-web-stories-template-3-metaData.js 539 B
assets/js/chunk-web-stories-template-3.js 8.35 kB
assets/js/chunk-web-stories-template-30-metaData.js 576 B
assets/js/chunk-web-stories-template-30.js 7.85 kB
assets/js/chunk-web-stories-template-31-metaData.js 503 B
assets/js/chunk-web-stories-template-31.js 10.2 kB
assets/js/chunk-web-stories-template-32-metaData.js 552 B
assets/js/chunk-web-stories-template-32.js 13.2 kB
assets/js/chunk-web-stories-template-33-metaData.js 491 B
assets/js/chunk-web-stories-template-33.js 9.04 kB
assets/js/chunk-web-stories-template-34-metaData.js 570 B
assets/js/chunk-web-stories-template-34.js 7.54 kB
assets/js/chunk-web-stories-template-35-metaData.js 565 B
assets/js/chunk-web-stories-template-35.js 8.86 kB
assets/js/chunk-web-stories-template-36-metaData.js 575 B
assets/js/chunk-web-stories-template-36.js 12.6 kB
assets/js/chunk-web-stories-template-37-metaData.js 529 B
assets/js/chunk-web-stories-template-37.js 6.27 kB
assets/js/chunk-web-stories-template-38-metaData.js 572 B
assets/js/chunk-web-stories-template-38.js 7.89 kB
assets/js/chunk-web-stories-template-39-metaData.js 589 B
assets/js/chunk-web-stories-template-39.js 8.03 kB
assets/js/chunk-web-stories-template-4-metaData.js 564 B
assets/js/chunk-web-stories-template-4.js 12.6 kB
assets/js/chunk-web-stories-template-40-metaData.js 556 B
assets/js/chunk-web-stories-template-40.js 10.2 kB
assets/js/chunk-web-stories-template-41-metaData.js 573 B
assets/js/chunk-web-stories-template-41.js 7.7 kB
assets/js/chunk-web-stories-template-42-metaData.js 521 B
assets/js/chunk-web-stories-template-42.js 6.96 kB
assets/js/chunk-web-stories-template-43-metaData.js 557 B
assets/js/chunk-web-stories-template-43.js 8.69 kB
assets/js/chunk-web-stories-template-44-metaData.js 583 B
assets/js/chunk-web-stories-template-44.js 11 kB
assets/js/chunk-web-stories-template-45-metaData.js 565 B
assets/js/chunk-web-stories-template-45.js 7.48 kB
assets/js/chunk-web-stories-template-46-metaData.js 531 B
assets/js/chunk-web-stories-template-46.js 5.19 kB
assets/js/chunk-web-stories-template-47-metaData.js 591 B
assets/js/chunk-web-stories-template-47.js 9.38 kB
assets/js/chunk-web-stories-template-48-metaData.js 556 B
assets/js/chunk-web-stories-template-48.js 9.04 kB
assets/js/chunk-web-stories-template-49-metaData.js 518 B
assets/js/chunk-web-stories-template-49.js 9.63 kB
assets/js/chunk-web-stories-template-5-metaData.js 556 B
assets/js/chunk-web-stories-template-5.js 9.87 kB
assets/js/chunk-web-stories-template-50-metaData.js 503 B
assets/js/chunk-web-stories-template-50.js 9.07 kB
assets/js/chunk-web-stories-template-51-metaData.js 526 B
assets/js/chunk-web-stories-template-51.js 10.3 kB
assets/js/chunk-web-stories-template-52-metaData.js 601 B
assets/js/chunk-web-stories-template-52.js 10.3 kB
assets/js/chunk-web-stories-template-53-metaData.js 551 B
assets/js/chunk-web-stories-template-53.js 5.76 kB
assets/js/chunk-web-stories-template-54-metaData.js 547 B
assets/js/chunk-web-stories-template-54.js 7.61 kB
assets/js/chunk-web-stories-template-55-metaData.js 574 B
assets/js/chunk-web-stories-template-55.js 7.08 kB
assets/js/chunk-web-stories-template-56-metaData.js 542 B
assets/js/chunk-web-stories-template-56.js 9.8 kB
assets/js/chunk-web-stories-template-57-metaData.js 528 B
assets/js/chunk-web-stories-template-57.js 14.8 kB
assets/js/chunk-web-stories-template-58-metaData.js 554 B
assets/js/chunk-web-stories-template-58.js 5.71 kB
assets/js/chunk-web-stories-template-59-metaData.js 590 B
assets/js/chunk-web-stories-template-59.js 8.9 kB
assets/js/chunk-web-stories-template-6-metaData.js 568 B
assets/js/chunk-web-stories-template-6.js 7.03 kB
assets/js/chunk-web-stories-template-60-metaData.js 509 B
assets/js/chunk-web-stories-template-60.js 9.4 kB
assets/js/chunk-web-stories-template-7-metaData.js 569 B
assets/js/chunk-web-stories-template-7.js 7.43 kB
assets/js/chunk-web-stories-template-8-metaData.js 569 B
assets/js/chunk-web-stories-template-8.js 8.85 kB
assets/js/chunk-web-stories-template-9-metaData.js 580 B
assets/js/chunk-web-stories-template-9.js 8.38 kB
assets/js/chunk-web-stories-templates.js 1.16 kB
assets/js/chunk-web-stories-textset-0.js 5 kB
assets/js/chunk-web-stories-textset-1.js 6.58 kB
assets/js/chunk-web-stories-textset-2.js 7.57 kB
assets/js/chunk-web-stories-textset-3.js 14.6 kB
assets/js/chunk-web-stories-textset-4.js 4.1 kB
assets/js/chunk-web-stories-textset-5.js 5.41 kB
assets/js/chunk-web-stories-textset-6.js 5.21 kB
assets/js/chunk-web-stories-textset-7.js 10 kB
assets/js/generateBlurhash.worker.worker.js 1.1 kB
assets/js/imgareaselect.js 3.77 kB
assets/js/web-stories-activation-notice.js 27.1 kB
assets/js/web-stories-block.js 24.1 kB
assets/js/web-stories-carousel.js 3.5 kB
assets/js/web-stories-dashboard.js 60.8 kB
assets/js/web-stories-embed.js 20 B
assets/js/web-stories-lightbox.js 649 B
assets/js/web-stories-tinymce-button.js 2.85 kB
assets/js/web-stories-widget.js 587 B

compressed-size-action

@swissspidy swissspidy merged commit 0fd6ee8 into main Feb 14, 2023
@swissspidy swissspidy deleted the try/heic branch February 14, 2023 15:21
@swissspidy swissspidy added this to the 1.30.0 milestone Feb 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Group: Media Type: Enhancement New feature or improvement of an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Convert HEIC image files to a web safe image format
2 participants