This experimental WordPress plugin aims to bring improved media capabilities to WordPress and specifically the block editor, all powered by new web platform features like WebAssembly. Highlights include:
- 100% client-side image compression for all new uploads, including for all thumbnails.
- Convert HEIC images (typically from iPhone) to more web-safe formats such as JPEG or AVIF.
- Effortlessly optimize existing images and videos in your content.
- Convert animated GIFs to videos for improved performance.
- Easily upload images from another device (nice AirDrop alternative!).
Want to see it in action? Check out my demo tweet or this interview. Detailed feature descriptions, including for upcoming features, are further below.
Install and activate the latest nightly build on your WordPress website and then start dragging media files into a new post, or optimize images in an existing post.
Note: Requires WordPress 6.5+ and PHP 8.0+.
Use WordPress Playground to try this plugin directly in the browser without installing it on your site:
Note: This is experimental. Playground does not support some of the WebAssembly features, so you will not have the full experience.
You can download the latest nightly build directly from GitHub.
Reminder: this is an experimental WordPress plugin. While you can totally install it on your website, it is recommended to try it on a development environment first.
Afterwards, updates will be available automatically in your WordPress admin.
Important: If you clone this repository, you will first need to build the assets.
Run npm install and npm run build to build all the JavaScript and CSS.
A non-exclusive list of things that have been implemented so far:
- Local poster generation during video upload
- Preview image generation for PDFs
- Converting GIFs to videos
- Video transcoding and compression (e.g. MOV to MP4)
- Image transcoding and compression (e.g. HEIF/HEIC to WebP or AVIF)
- Audio transcoding (e.g. OGG to MP3)
- Video audio track detection and muting
- Optimization of existing media with a comparison screen
- Self-recording using webcam / microphone
- Import external media to library
- Client-side image downsizing and thumbnail generation
- Preferences dialog to change default settings
These features can be tested either by dropping media files into the editor or by editing existing media blocks in the editor.
Note: not all of these have been fully implemented yet.
Poster images are important for UX as they are shown before a video is fully loaded and started playing. During upload, a poster image (thumbnail) of the video is automatically taken and added to the video in both the editor and the media library.
Screen.Recording.2023-10-11.at.20.28.09.mov
In addition to that, poster images are automatically generated for PDFs as well (powered by PDF.js).
Screen.Recording.2023-10-11.at.20.26.38.mov
Videos are the better GIFs, at least when it comes to file types. Animated GIFs should really not be used anymore.
This project automatically converts animated GIFs to an actual video file during upload. The image block, upon receiving the updated media item, transforms into a video block. Note: Right now, that last part (block transformation) is achieved by patching Gutenberg.
Screen.Recording.2023-10-11.at.20.31.20.mov
Uses libheif-js to automatically convert HEIC images (typically taken on iPhone) to a more common format (WebP or JPEG).
This project uses ffmpeg.wasm to unlock video/image/audio transcoding and compression during upload.
For example from MOV to MP4, HEIF to WebP, or OGG to MP3.
During upload, detect whether a video actually has audio. If not, hide/disable “Muted” option in the editor.
Note: Right now, this is achieved by patching Gutenberg.
This allows completely removing the audio channel from a video, so it’s actually muted.
During upload, it’s automatically detected whether the video actually has any audio channels. If not, it’s marked as muted.
Screen.Recording.2023-10-11.at.20.32.24.mov
While the image block in Gutenberg already supports importing external images into the site’s own media library, other blocks don’t. This project changes this, while also hooking the import into the same upload logic so that imported files automatically get compressed and transcoded if needed.
Adds self-recording capabilities to the video/image/audio blocks in Gutenberg so that one can record videos, GIFs (which are basically muted looped videos), audio transcripts, or simply take a still picture using your camera and/or microphone.
This also supports automatically blurring backgrounds.
Screen.Recording.2023-10-11.at.20.37.07.mov
Automatically extracts the dominant color of an image and generates a BlurHash during upload. This can be used to improve perceived performance by using them as placeholders during loading — both in the editor and on the frontend.
Whenever available, the BlurHash is used to generate a blurred placeholder image using just CSS, with a fallback to just a solid color.
Caveat: no placeholder is displayed if the image has alpha transparency, as that would lead to undesired results.
This is more advanced than the Dominant Color Images feature plugin, which does not generate blurred placeholders.
Allows optimizing existing media items in a post by converting them to a more modern format. Either individually or in bulk. After conversion, the old and new versions can optionally be compared with a slider. The file size savings in % are shown next to it.
Single optimization with approval/comparison step:
optimize-existing-media.mov
Bulk optimization without the approval step:
Screen.Recording.2023-11-27.at.15.48.26.mov
When a new image is uploaded, WordPress will detect if it is a “big” image by checking whether it exceeds a certain threshold (currently 2560px width). Images exceeding the threshold will be scaled down on the server to prevent users from using unnecessarily large images in their content. However, the original file is still retained just in case.
While the server-side downsizing is admirable, it’s still wasteful to upload the full original image. Doing this client-side improves upload time and reduces bandwidth consumption.
When uploading an image to WordPress, it generates many thumbnail versions of it. This often leads to very slow upload processes, to the point where WordPress had to build a workaround to resume failed attempts. Besides the performance concern, the thumbnails would also not benefit from the same client-side image optimizations.
To address this, the thumbnail generation is done entirely client-side as well. Less server stress, less failed uploads, and thumbnails with the same traits as the original.
Detect the most relevant part of an image to improve thumbnail cropping. For example using MediaPipe’s object detection solution.
Oftentimes, the MozJPEG encoder provides better or equal results than WebP. It’s appealing to be able to generate lightweight images without the UX downsides of WebP. MozJPEG is typically not available in the server-side image libraries (GD and Imagick) used by WordPress.
Same as above, useful for comparison.
Adds a preferences modal to the block editor where users can choose their preferred image formats (e.g. MozJPEG, WebP) and quality levels.
The default camera and microphone to use for media recording can be set here as well.
Ability to convert all existing images in the media library client-side. Ideally without having to stay on the page.
Use AI to automatically generate video captions for videos. Nice accessibility and UX enhancement.
Use AI to automatically generate alt text for images and videos to improve accessibility.
Show all in-progress items for easier inspection, and allow cancelling individual uploads.
Allows uploading media to the current post from an entirely different device, such as a mobile phone, even when not logged in.