diff --git a/.gitignore b/.gitignore index b77727c..663eb41 100644 --- a/.gitignore +++ b/.gitignore @@ -143,3 +143,4 @@ enctests/sources/enc_sources/chimera_cars_srgb_jpg enctests/sources/enc_sources/chimera_wind enctests/results.old tmp +enctests/wedge_results diff --git a/ColorPreservation.md b/ColorPreservation.md index 109a3fc..70cdca6 100644 --- a/ColorPreservation.md +++ b/ColorPreservation.md @@ -13,7 +13,7 @@ We would like ffmpeg to do as little as possible in terms of color space convers The main problem is that ffmpeg by default assumes that any unknown still image format has a color space of [rec601](https://en.wikipedia.org/wiki/Rec._601) which is very unlikely to be the color space your source media was generate in. So unless you tell it otherwise it will attempt to convert from that colorspace producing a color shift. -Separately, all the video formats typically do not use the full numeric range [0-255] but instead the Y' (luminance) channel have a nominal range of [16..235] and the CB and CR channels have a nominal range of [16..240] with 128 as the neutral value. This frequently results in quantisation artifacts for 8-bit encoding (the standard for web playback). This fortunately is something you can change, [see TV vs. Full range](Quickstart.html#tv-vs-full-range-). below. The other option is to use higher bit depth, e.g. 10-bit or 12 bit for formats such as [ProRes](Encoding.html#prores-). +Separately, all the video formats typically do not use the full numeric range [0-255] but instead the Y' (luminance) channel have a nominal range of [16..235] and the CB and CR channels have a nominal range of [16..240] with 128 as the neutral value. This frequently results in quantization artifacts for 8-bit encoding (the standard for web playback). This fortunately is something you can change, [see TV vs. Full range](Quickstart.html#tv-vs-full-range-). below. The other option is to use higher bit depth, e.g. 10-bit or 12 bit for formats such as [ProRes](Encoding.html#prores-). Even if you are sticking to 8-bits encodes, if your source media is able to have a higher bit-depth (e.g. you are able to write out 16-bit PNG's to do the encode) it will help with the accuracy of the RGB to YUV conversion, particularly if you are using libswscale (see below). @@ -27,7 +27,7 @@ For examples comparing these see: [here](https://academysoftwarefoundation.githu ``` This is the most basic colorspace filtering. bt470bg is essentially part of the bt601 spec. See: [https://www.ffmpeg.org/ffmpeg-filters.html#colormatrix](https://www.ffmpeg.org/ffmpeg-filters.html#colormatrix) -e.g. +Example: ``` ffmpeg -y -i ../sourceimages/chip-chart-1080-noicc.png \ - -pix_fmt yuv444p10le -vf "scale=in_range=full:in_color_matrix=bt709:out_range=tv:out_color_matrix=bt709" \ + -pix_fmt yuv444p10le \ + -vf "scale=in_range=full:in_color_matrix=bt709:out_range=tv:out_color_matrix=bt709" \ -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -qscale:v 1 \ -color_range tv -colorspace bt709 -color_primaries bt709 -color_trc iec61966-2-1 \ ./chip-chart-yuvconvert/spline444out_color_matrix.mp4 diff --git a/OtherFfmpegArgs.md b/OtherFfmpegArgs.md index 0cad9c4..cebc795 100644 --- a/OtherFfmpegArgs.md +++ b/OtherFfmpegArgs.md @@ -1,6 +1,6 @@ --- layout: default -title: Useful Ffmpeg Filters. +title: Useful Ffmpeg Filters nav_order: 5.5 parent: Encoding Overview --- diff --git a/Quickstart.md b/Quickstart.md index 66d1ede..b47b709 100644 --- a/Quickstart.md +++ b/Quickstart.md @@ -173,17 +173,6 @@ For more details see: * [Encoding Guide](Encoding.html#range) -# Encoding as RGB. -You do not *have* to encode into YCrCb, h264 does support RGB encoding, which may be preferable in some situations. - -Using the encoder: -``` --c:v libx264rgb -``` -Will skip the conversion completely. Sadly this has no support in web browsers, but is supported by some players (e.g. RV). It is also limited to 8-bit. - -TODO Check about 10-bit encoding. - For more details see: * [Comparing full-range vs. tv range](https://academysoftwarefoundation.github.io/EncodingGuidelines/tests/greyramp-fulltv/compare.html) * [Encoding Guide](Encoding.html#range) diff --git a/RGBEncoding.md b/RGBEncoding.md new file mode 100644 index 0000000..4cfd4cd --- /dev/null +++ b/RGBEncoding.md @@ -0,0 +1,86 @@ +--- +layout: default +nav_order: 6 +title: RGB Encoding. +parent: Encoding Overview +--- + +# RGB Encoding +You do not *have* to encode into YCrCb, there are a number of codecs that support RGB directly. + +It does lower the compression efficiency a little, but eliminates any concern that the YCrCb conversion is affecting the imagery. + +| Codec Family | ffmpeg codec | Bit depth | pix_fmt | Web Support | +|------------|------------|------------|------------| +| [h264](Encodeh264.html) | libx264rgb | 8 | | no | +| [HEVC/H265](EncodeHevc.html) | libx265 | 8 10 12 | rgb rgb-10 rgb-12 | All browsers | +| [HEVC/H265](EncodeHevc.html) | hevc_videotoolbox | 8 10 | rgb rgb-10 rgb-12 | All browsers | +| [VP9](EncodeVP9.html) | libvpx-vp9 | 8 10 12 | rgb rgb-10 rgb-12 | All Browsers | +| [Mjpeg](EncodeMJPEG.html) | mjpeg | 8 | yuv-4:2:2 yuv-4:4:4 | no | +| [DNxHD](EncodeDNxHD.html) | dnxhd | 8 10 | rgb | no | + + +## H264 RGB Encoding + +h264 does support RGB encoding, which may be preferable in some situations. + +Using the encoder: +``` +-c:v libx264rgb +``` + +Has no support in web browsers, but there is limited support in players such as RV. + +## mjpeg RGB Encoding + +For more details see: [Mjpeg](EncodeMJPEG.html). This is an 8-bit only encode. + +``` + -c:v mjpeg -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -pix_fmt rgb24 -q:v 2 -vf "scale=in_range=full:out_range=full" +``` + +## VP9 RGB Encoding + +VP9 has some excellent 10 and 12 bit RGB encodes. + +Example encoding: + + +``` +ffmpeg -r 24 -start_number 1 -i inputfile.%04d.png -frames:v 200 -c:v libvpx-vp9 \ + -crf 22 -pix_fmt gbrp10le -quality good -row-mt 1 -speed 2 -vf "scale=in_range=full:out_range=full" \ + -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb \ + -y outputfile.mp4 +``` + + +## RGB Comparisons + + +| *label* | *Codec Params* | +| 8_bit_libx264rgb | -c:v libx264rgb -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -crf 18 -pix_fmt rgb24 -preset slow -vf "scale=in_range=full:out_range=full" -x264-params keyint=15:no-deblock=1 | +| 8bit_mjpeg | -c:v mjpeg -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -pix_fmt rgb24 -q:v 2 -vf "scale=in_range=full:out_range=full" | +| vp9_10bit_rgb | -c:v libvpx-vp9 -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -crf 22 -pix_fmt gbrp10le -quality good -row-mt 1 -speed 2 -vf "scale=in_range=full:out_range=full" | +| vp9_12bit_rgb | -c:v libvpx-vp9 -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -crf 22 -pix_fmt gbrp12le -quality good -row-mt 1 -speed 2 -vf "scale=in_range=full:out_range=full" | +| vp9_8bit_rgb | -c:v libvpx-vp9 -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -crf 22 -pix_fmt gbrp -quality good -vf "scale=in_range=full:out_range | + +|  This is showing different rgb encoding against file size. | +|  This is showing different rgb encoding against encoding time | +|  This is showing different rgb encoding against mean VMAF | + + + +For more details see: + * [Comparing full-range vs. tv range](https://academysoftwarefoundation.github.io/EncodingGuidelines/tests/greyramp-fulltv/compare.html) + * [Encoding Guide](Encoding.html#range) diff --git a/enctests/bitDepth/BitDepthComparisons.md b/enctests/bitDepth/BitDepthComparisons.md new file mode 100755 index 0000000..2bf5f54 --- /dev/null +++ b/enctests/bitDepth/BitDepthComparisons.md @@ -0,0 +1,91 @@ +--- +layout: default +nav_order: 6.08 +title: Bit Depth Analysis +parent: Test Output +--- + +# Bit Depth Analysis + +Sometimes its useful to know what the actual dynamic range of a particular combination of codec, profile and pixel format is, since it may not necessarily be what you think. i.e. you think you are getting 10-bits (i.e. 1024 values) when in fact you could be getting quite a bit less. + +This work is inspired by [https://github.com/ColorlabMD/Prores-BitDepth](https://github.com/ColorlabMD/Prores-BitDepth). + +These tests generate a luminance YCrCb flat image, where each frame increments the luminance by one using the geq ffmpeg filter. + +For example for the prores_ks codec we would run: + +``` +ffmpeg -y -r 24 -f lavfi -i nullsrc=s=720x480,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v prores_ks -profile:v 4444xq ./colors-prores_ks_10_4444xq.mov +``` + +We then read the resulting movie file, to see if the luminance that we expect. + +The columns are defined as: + * Test Name - which codec we are testing. + * Bit Depth - What bit-depth the codec is being evaluated to (which determines how many unique values to test to). + * Unique values - How many resulting values are unique, but may not be exactly the same value. + * STDDEV > 0.0001 - Standard deviation of the frame is > 0.0001 which means what should be a flat color, isnt. + * Off by 1 - The color is a flat color, but is "off by 1" (i.e. what should be 3 is 4), again not ideal. + * Other invalid - is a flat color, but is some other value than expected. + +## Running the tests + +To run the tests you need three additional python libraries as well as ffmpeg: + +``` +pip install yuvio numpy pyseq +``` + +Warning: The file sizes can get large, so expect it to generate 8GB of data. + +Note, the one thing the ColorlabMD test does that this does not currently do is to also do a comparison of chroma variation. + +## Results + +### Prores_ks + +Tests - prores_ks_10_4444xq, prores_ks_10_proxy, prores_ks_10_hq + +It shows that the ffmpeg_ks encoder is only encoding to the legal 10-bit range (i.e. 877 values), which is pretty much what we expect. + +The tests [https://github.com/ColorlabMD/Prores-BitDepth](https://github.com/ColorlabMD/Prores-BitDepth) tests seem a little unfair on prores_ks, which clearly defines itself as a 10-bit codec, and I suspect that the apple encoding is setting full range values which Prores (in this case) is not. + +Note, Prores_ks will read 12 bit files, just not generate them. + +### Apple Videotoolbox Prores + +Tests - prores_videotoolbox_10_proxy, prores_videotoolbox_10_hq, prores_videotoolbox_10_4444, prores_videotoolbox_12_xq + +Interestingly, we are getting better results for prores_ks for proxy and HQ, in that the values are at least consistent, which for videotoolbox they are often off by 1. + +However, for 4444 and XQ they are good. + +### DNxHD + +All the values are as expected. + +### h264 + +All the values are as expected. + +### x265 HEVC + +x265 HEVC is struggling with flat color with quite a few values not generating a uniform value. + +### VP9 + +All the values are as expected. + +### libsvtav1 + +Something rather odd going on here, the 10-bit encode is generating only 475 unique values rather than 1024. Looking at the values, it seems like there is some sort of rounding going on here. + +### libaom-av1 + +Is slightly better, with 866 unique values, but many of them are off by one or more. At 12-bit the results are similar. + + +Click [here](bitDepthResults.html) to see the full page table. + +{% include_relative bitDepthResults.html %} diff --git a/enctests/bitDepth/bitDepthResults.html b/enctests/bitDepth/bitDepthResults.html new file mode 100755 index 0000000..3d2aa82 --- /dev/null +++ b/enctests/bitDepth/bitDepthResults.html @@ -0,0 +1,25 @@ +
| Test Name | Bit Depth | Unique Values | Range of Valid Values | a.std() > 0.0001 | Off by 1 | Other Invalid | Fffmpeg command |
|---|---|---|---|---|---|---|---|
| prores_ks_10_4444xq | 10 | 1023 / 1024 | [1-1022] | [0, 1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v prores_ks -profile:v 4444xq ./colors-prores_ks_10_4444xq.mov | ||
| prores_ks_10_proxy | 10 | 1016 / 1024 | [4-1019] | [0-3, 1020-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv422p10le -frames:v 1024 -vf geq=N:512:512 -c:v prores_ks -profile:v proxy ./colors-prores_ks_10_proxy.mov | ||
| prores_ks_10_hq | 10 | 1016 / 1024 | [4-1019] | [0-3, 1020-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v prores_ks -profile:v hq ./colors-prores_ks_10_hq.mov | ||
| prores_videotoolbox_10_proxy | 10 | 1015 / 1024 | [4-527, 1019] | [3, 528-1018] | [0-3, 528-1018, 1020-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v prores_videotoolbox -profile:v proxy ./colors-prores_videotoolbox_10_proxy.mov | |
| prores_videotoolbox_10_hq | 10 | 1015 / 1024 | [4-446, 1019] | [3, 447-1018] | [0-3, 447-1018, 1020-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v prores_videotoolbox -profile:v hq ./colors-prores_videotoolbox_10_hq.mov | |
| prores_videotoolbox_10_4444 | 10 | 1023 / 1024 | [1-1022] | [0, 1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v prores_videotoolbox -profile:v 4444 ./colors-prores_videotoolbox_10_4444.mov | ||
| prores_videotoolbox_12_xq | 12 | 4088 / 4096 | [4-4091] | [0-3, 4092-4095] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p12le -frames:v 4096 -vf geq=N:2048:2048 -c:v prores_videotoolbox -profile:v xq ./colors-prores_videotoolbox_12_xq.mov | ||
| dnxhd_10_dnxhr_444 | 10 | 1024 / 1024 | [0-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v dnxhd -profile:v dnxhr_444 ./colors-dnxhd_10_dnxhr_444.mov | |||
| dnxhd_10_dnxhqx_422 | 10 | 1024 / 1024 | [0-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv422p10le -frames:v 1024 -vf geq=N:512:512 -c:v dnxhd -profile:v dnxhr_hqx ./colors-dnxhd_10_dnxhqx_422.mov | |||
| dnxhd_8_422 | 8 | 256 / 256 | [0-255] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=1920x1080,format=yuv422p -frames:v 256 -vf geq=N:128:128 -c:v dnxhd -b:v 36M ./colors-dnxhd_8_422.mov | |||
| dnxhd_8_dnxhqx_422 | 8 | 256 / 256 | [0-255] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv422p -frames:v 256 -vf geq=N:128:128 -c:v dnxhd -profile:v dnxhr_hq ./colors-dnxhd_8_dnxhqx_422.mov | |||
| h264-placebo | 10 | 1024 / 1024 | [0-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v h264 -preset placebo -qp 0 ./colors-h264-placebo.mov | |||
| h264-slower | 10 | 1024 / 1024 | [0-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v h264 -preset slower -crf 15 ./colors-h264-slower.mov | |||
| hevc-slower-444-10 | 10 | 1024 / 1024 | [0-1000, 1002-1003, 1005-1023] | [1001, 1004] | [1001, 1004] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v hevc -profile:v main444-10 -preset slower -crf 10 ./colors-hevc-slower-444-10.mov | |
| hevc-slower-444-12 | 12 | 4096 / 4096 | [0-4, 6, 8, 10 ... 4085, 4087, 4089, 4091-4095] | [5, 7, 9, 11 ... 4083-4084, 4086, 4088, 4090] | [5, 7, 9, 11 ... 4083-4084, 4086, 4088, 4090] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p12le -frames:v 4096 -vf geq=N:2048:2048 -c:v hevc -profile:v main444-12 -preset slower -crf 2 ./colors-hevc-slower-444-12.mov | |
| vp9-slower-444-10 | 10 | 1024 / 1024 | [0-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v libvpx-vp9 -quality good -crf 5 -b:v 0 ./colors-vp9-slower-444-10.mp4 | |||
| vp9-slower-444-12 | 12 | 4096 / 4096 | [0-4095] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p12le -frames:v 4096 -vf geq=N:2048:2048 -c:v libvpx-vp9 -quality good -crf 5 -b:v 0 ./colors-vp9-slower-444-12.mp4 | |||
| libsvtav1-420-8 | 8 | 256 / 256 | [0-255] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p -frames:v 256 -vf geq=N:128:128 -c:v libsvtav1 -preset 9 -crf 3 ./colors-libsvtav1-420-8.mp4 | |||
| libsvtav1-420-10 | 10 | 475 / 1024 | [0, 4, 8, 12 ... 1016, 1018, 1020, 1022] | [1-3, 5-7, 9-11, 13-15 ... 1017, 1019, 1021, 1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v libsvtav1 -preset 9 -crf 3 ./colors-libsvtav1-420-10.mp4 | ||
| libaom-444-10 | 10 | 866 / 1024 | [1-10, 12-14, 16-18, 20-22 ... 993, 997, 1001-1015, 1018-1023] | [0, 684-685] | [111, 115, 119, 123 ... 994, 996, 998-1000, 1016-1017] | [0, 11, 15, 19 ... 990-992, 994-996, 998-1000, 1016-1017] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v libaom-av1 -cpu-used 4 -usage good -crf 20 -row-mt 1 ./colors-libaom-444-10.mp4 |
| libaom-444-12 | 12 | 3805 / 4096 | [4-32, 55-70, 72-79, 88-103 ... 4065-4072, 4087, 4091, 4095] | [0-3] | [33-48, 51-54, 80-87, 113-120 ... 4011-4014, 4040-4047, 4073-4080, 4083-4086] | [0-3, 33-54, 71, 80-87 ... 4064, 4073-4086, 4088-4090, 4092-4094] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv444p12le -frames:v 4096 -vf geq=N:2048:2048 -c:v libaom-av1 -cpu-used 4 -usage good -crf 20 -row-mt 1 ./colors-libaom-444-12.mp4 |
| hevc_videotoolbox_8_main | 8 | 256 / 256 | [0-255] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=yuv420p -frames:v 256 -vf geq=N:128:128 -c:v hevc_videotoolbox -profile:v main -q:v 100 ./colors-hevc_videotoolbox_8_main.mov | |||
| hevc_videotoolbox_10_main10 | 10 | 1024 / 1024 | [0-1023] | ffmpeg -y -r 24 -f lavfi -i nullsrc=s=256x120,format=p010le -frames:v 1024 -vf geq=N:512:512 -c:v hevc_videotoolbox -profile:v main10 -q:v 100 ./colors-hevc_videotoolbox_10_main10.mov |
| Test Name | Bit Depth | Unique Values | Range of Valid Values | a.std() > 0.0001 | Off by 1 | Other Invalid | Fffmpeg command |
|---|---|---|---|---|---|---|---|
| ", test['testname'], " | ", test['bits']," | ", count, "/", test['frames'], " | ", seqToStr(validvalues), " | ", seqToStr(nearlymissing), " | ", seqToStr(nearlymissing1), " | ", seqToStr(missing), " | ",cmd," |
Comparing full range encoding vs. tv range, but also yuv420p vs. yuvj420p. We believe that this is well supported on web browsers, and dont see a downside to it. There may be cases where other applications do not read it. The code to generate these files is here. Full screen view is here
+Comparing full range encoding vs. tv range, but also yuv420p vs. yuvj420p. We believe that this is well supported on web browsers, and dont see a downside to it. There may be cases where other applications do not read it. The code to generate these files is here. Full screen view is here
Comparing full range encoding vs. tv range, but also yuv420p vs. yuvj420p. We believe that this is well supported on web browsers, and dont see a downside to it. There may be cases where other applications do not read it. The code to generate these files is here. Full screen view is here
" % os.path.basename(__file__), + introduction="Comparing full range encoding vs. tv range, but also yuv420p vs. yuvj420p. We believe that this is well supported on web browsers, and dont see a downside to it. There may be cases where other applications do not read it. The code to generate these files is here. Full screen view is here
" % os.path.basename(__file__), videohtml = ' width=920 ')