RFC: next/video

[heff]

Goals

This RFC aims to address comments like #20323.

I wish there was something like next/image but for videos.

Non-Goals

1. Not solving Youtube, Vimeo, Wistia and other video platform player embedding - This is already solved by projects like react-player, so it’s not prioritized in this initial proposal shape.
2. Not solving <Audio /> - Audio is more than just “video minus the visual-part”. It brings its own unique uses and issues. It’s not prioritized in this initial shape, but could be a fast follow.

Background

<video> is harder than it looks.

1. <video> has many of the same performance and layout-related issues as <img>, and then many other problems because video is interactive and file sizes are much bigger.
2. Video source files can be hard to work with. Many are not compatible in all (or any!) browsers and mobile devices. Some files may be too big for mobile networks. Some files may play but not be streamable, requiring a full download before playback.
3. Storing and delivering unoptimized video files can quickly get expensive and hit repo or hosting service limits. See Vercel's guidance on the topic.

Open source players like react-player and video.js can solve many of the client-side related issues with video, but they can’t solve server-side aspects. They also can’t bring any of the magic that a JS meta-framework like Next.js can when combining client and server, much like with next/image.

Proposal

There are three parts to this next/video proposal:

1. The <Video /> component
2. The workflow for optimizing video files
3. Proposed features

1. <Video />

import Video from 'next/video';
import myVideo from '/videos/myVideo.mp4';

export default function Page() {
  return <Video src={myVideo} />;
}

The <Video /> component will by default include a best-in-class video player, solving the many cross-browser <video> issues and bringing a default UI that is accessible, responsive, and aesthetically pleasing.

Much like next/image it will add build-time features like a blurred, fast-loading placeholder (or ”poster”) and smart sizing based on the video source file’s aspect ratio.

It will provide a clear path towards customization, from building a custom UI to integrating third-party software like analytics providers.

2. The video source file optimization workflow

/videos

Any video source file added to a new directory named /videos will be automatically uploaded via a watch script, or manually via a CLI command, to a configured third-party service for hosting, optimization, and delivery.

[video-file-name].json

A new file named [video-file-name].json (e.g. myVideo.mp4.json) will be created in the /videos directory for mapping the original video file name to the third-party-hosted version, and for storing metadata about the video, for example processing status and aspect ratio.

/videos/.gitignore

The original video source file will be ignored in the repo via a local .gitignore file, addressing the git repo video storage problems. If storing the video source files in the repo is also desired for any reason, LFS can be enabled and the .gitignore removed.

import myVideo from '/videos/myVideo.mp4'

A module import of import myVideo from '/videos/myVideo.mp4' will return the .json file object, including third-party ID mapping and metadata. This will be possible via a Webpack plugin and extension to next.config.js. The <Video /> component will use the object to retrieve metadata and a URL to the optimized version of the video, or fall back to the local video source during development if the optimized version is still processing.

Custom processing handlers

The optimization workflow is customizable, allowing developers and other video service providers to integrate their own video processing into this workflow.

3. Proposed features

Third-party service provided features

These features would be expected of the configured video service, but may behave differently or be optional depending on the specific service.

• Auto optimization: Automatically optimize video files and deliver via CDN (or multiple CDNs) for better loading performance and quality
• Wider compatibility: Use video source files that aren’t supported natively by all browsers
• AI-powered: Whisper captions support and processing handlers for other AI tooling

<Video /> features

• Posters & Previews: Zero-config placeholder images and timeline hover thumbnails
• Customizable UI: Choose from themes or build your own player controls
• Configurable analytics: Report on video engagement and streaming performance

Workflow features

• Automatically process any video source file added to the /videos directory
• Fall back to the original source file in development while the video is processing, and show process in the video player
• Support local and remote source files

Proof of concept

A prototype of this proposal can be found at github.com/muxinc/next-video. Please try it out and give feedback!

The proof of concept is built using Mux as the video infrastructure. The v1.0 of this feature would include additional services as options.

[bhatvikrant]

I love the idea of having something like this in nextjs. I believe the best case is where this

[rawnly]

Hey @bhatvikrant this just came out : https://next-video.dev/

[bhatvikrant]

Woah, just saw this, it’s pretty cool! Love the direction it’s going in.

[artemis-prime]

As my religious friends say, from your mouth, to God's ears!

[artemis-prime]

@rawnly , does anyone know the connection between the two projects? Is he working with vercel to get it into the core or ? I just filed an issue on their site asking for some comment on the connection (say in the README)

[heff]

@artemis-prime the next-video project (next-video.dev & the github repo) is linked in the RFC as the proof of concept. We hope it can be useful on its own for now and as a conversation piece for how video could ultimately be first-class in next.js. There's been no specific progress on the RFC front specifically yet, but we've been taking feedback and requests on the stand-alone project that will help move things forward either way.

[burkeingraffia]

Having this work with VideoJS (and companies who use it like Brightcove) would be great.

[mmcc]

You could use Video.js as the player fairly easily! Is there something specific you're looking for there?

[jeffreymc2]

I have src file coming from existing s3 endpoints as either m3u8 file > .ts or existing endpoints serving mp4, mov, etc.

I'm getting all sorts of console errors. 500 errors when fetching /video api. And sporadic
Source Not Supported MEDIA_ELEMENT_ERROR: Format error.

My question is: if the video files are already hosted, why do we need to upload again? Is there a way around this to handle multple file types (i.e., m3u8, mov, mp4) just using the existing endpoint urls?

[luwes]

We added a next-video/player only export today, available in the latest release v1.1.0.
Hope this helps!

example:
next-video-demo.vercel.app/player-only
main/examples/default-provider/app/(default)/player-only/page.tsx

[B33fb0n3]

Thanks for sharing this. The method with the next-video/player does not work. When integrating it with the player, all (public and external accessible) URLs will get a CORS error

Access to fetch at 'https://...6ee3.mp4' from origin 'https://...n.de' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

When using the default HTML video element with the same URL, there is no CORS error. So the problem here is on the player side and in depending on the file.

[heff]

Adding some notes here about 1 year later.

The next-video project is going well and steadily growing in weekly downloads. The major focuses right now are player flexibility, player file size, and alternative metadata stores.

We're playing a bit of whac-a-mole with the install wizard as Next.js makes updates, for example the move from webpack to turbopack as the default bundler, which is where some of our auto-processing logic lives. It's not a big problem exactly, but I think it points to some of the value of building video support into Next.js similar to next/image.

That said, if this is ultimately a wontfix, feel free to say so!