RGB with alpha channel is always black in ffmpeg/ffplay

[magiblot]

The following script:

BlankClip(color=$FF, pixel_type="RGB32")
Info()
ShowFrameNumber(true)

Looks like this in VirtualDub2 (using AVS+ 3.6.0):

But it produces a black screen in ffplay:

I'm using ffmpeg 4.2.3 on linux, but with @qyot27's patches for Avisynth+ support (backported to Arch Linux's official ffmpeg package as of this morning).

ffplay version n4.2.3 Copyright (c) 2003-2020 the FFmpeg developers
  built with gcc 10.1.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-nvdec --enable-nvenc --enable-omx --enable-shared --enable-version3
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100

I have tried with different bit depths and with packed/planar formats, and only RGB formats with an alpha plane seem affected.

I had difficulties testing YUVA. I wrote the following script which works in ffplay, but throws an error in VirtualDub2:

BlankClip(color=$FF, pixel_type="YV24")
AddAlphaPlane()
Info()
ShowFrameNumber(true)

Error decompressing video frame 23:

Cannot decompress video frame: the video data is too short (921600 bytes, should be 1228800).

[qyot27]

I think this is an issue with ffplay itself (I'm using a git build from June 2nd and can reproduce this on Windows 10). Initially I thought it might even be in SDL2, but using SDL2 through mpv's sdl vo displays bgra correctly (mpv's vo=gpu only displays the checkerboard for alpha-laden formats, which is a not-so-minor annoyance).

You can see that ffmpeg itself is fine with it by bypassing ffplay:

ffmpeg -i test.avs -vcodec copy -f nut - | mpv -vo=sdl -

Or if you've built said mpv against the patched FFmpeg, just opening the script directly like you would have with ffplay:

mpv --vo=sdl test.avs

If you're passing it to mpv's gpu vo, use --vf=format=fmt:yuv420p so that mpv converts the bgra to a viewable format (it doesn't have to be yuv420p). Since that's completely external to the script itself, the issue is definitely in the way the playback application is talking to the display driver, or a straight-up failure of the display driver.

Although weirdly, if I import real video in through ffms2 and use ConvertToRGB32 (or a more explicit ConvertToRGB24().AddAlphaPlane()), ffplay seems to be fine with it. I wonder if it has anything to do with the no-op caching for some filters? Try to revert eb018b8 and if it builds, see if ffplay has any issues with it when using BlankClip.

It seems like VDub2's YUVA support for AviSynth+ (maybe in general?) is still a little finicky. YUVA444 doesn't work - it gives the error you described - but YUVA444P16 seems to be fine.

[magiblot]

Could it be that ffmpeg is one of the first AviSynth+ clients to ever care about the alpha plane?

BlankClip(color=$FF, pixel_type="RGB32")
ConvertToPlanarRGBA()
alpha = "0"
Expr(last, "0", "0", "255", alpha, format="RGBAP8")
Info()
ShowFrameNumber(true)

alpha = 0

alpha = 127

alpha=255

It seems that the alpha plane is zero-initialized by default.

[qyot27]

RGBAdjust(ab=255)

Also seems to do it in a much simpler way than using Expr, backing up the idea that it's something about how alpha is initialized in the RGB formats (seeing as how the YUVA ones don't seem affected by this). ColorYUV doesn't seem to have any alpha-related options even though it says those pix_fmts are supported.

[pinterf]

Zero initialized because color=$FF
color = $FF0000FF will initialize to 255.

[magiblot]

LOL! I feel so dumb.
But the wiki says that color=0 by default. If that is true, BlankClip is creating invisible clips by default. Additionally, the BlankClip article in the wiki doesn't say anything about the alpha channel, which may lead users (me included) to think the color parameter is only about color.

[magiblot]

Obviously, the responsability of maintaining the wiki is not only yours. We should find together a way to make it more intuitive (unfortunately, it seems I cannot edit that article). But certainly, producing invisible clips does not seem like a good default. Maybe that can be improved on AviSynth+'s side.

[pinterf]

Yes, I found other places to fix it
http://avisynth.nl/index.php/Colors
where even the first two hexadecimal digits are explicitely mentioned.
"In hexadecimal notation the number is composed as follows: the first two digits denote the red channel,"
And it is not mentioned how 8 bit values are converted to high bit depth (bit shift when YUV, range stretch when RGB)

[qyot27]

So, this should actually mean we need to fix colors_rgb.avsi and add the alpha values. Even using color=color_blue instead of the hex values still exhibited the problem described here.

And, I'm not understanding how it's parsing the hex values.

color=$FF produces blue. But the full hex for that should be $0000FF (as shown in colors_rgb.avsi). Is it just a quirk of how hex parsing works that omitted zeroes are inserted between the given values and the $ instead of after ($FF0000 produces red)? Is that due to endianness, or something else? Would this act differently if we were running on a big-endian platform (this is important for both ARM and potentially if we ever add PowerPC, as both of those can operate in either little or big endian, with ARM mostly little and PowerPC mostly big).

Then to add alpha in, alpha is weird in that it's seemingly acting in reverse of what users might expect - values for the R-G-B channels run from 0 to 255, with zero being no strength and 255 being full strength. $000000 is thus black and $FFFFFF is white, because those channels are always visible. Alpha, on the other hand, is using it in terms of what alpha is, transparency. 0 is completely opaque (which makes it appear black in ffplay, but as the typical transparent checkerboard placeholder in mpv, arguably far more accurate than black is), 255 is completely transparent.

I think user friendliness might require adding something like an alpha_opacity argument that flips this so that 0=transparent 1.0=opaque. Or implementing a standalone function (Opacity¹) that only controls the visibility of the individual channels and nothing else (as opposed to what Overlay, Layer, or the Show* functions do in manipulating channels).

¹whether this should also include derived functions for each channel - OpacityRed, OpacityAlpha - is another question.

[qyot27]

Obviously, the responsability of maintaining the wiki is not only yours. We should find together a way to make it more intuitive

There are also the main docs. The wiki is currently more up-to-date than the docs are because everyone goes there, but since the [english] docs are now in ReStructuredText format, changes to those are simpler than editing HTML and can be submitted upstream in a normal pull request.

[magiblot]

To my mind, the Alpha channel does not work so differently from RGB: 0 is "I can't see anything" and 255 is "I can see".

[magiblot]

0 is completely opaque (which makes it appear black in ffplay, but as the typical transparent checkerboard placeholder in mpv, arguably far more accurate than black is), 255 is completely transparent.

I would describe it the opposite way. Alpha is a visibility modifier. 0 makes all planes totally transparent (thus the checkerboard pattern) and 255 makes them visble (that is, opaque over the picture being composed underneath). This matches the description of Alpha in Wikipedia. So it seems like we should see Alpha as 'opacity' rather than 'transparency'.

[pinterf]

Then there is another historical fact: Colorbars RGB32 is using 0 for alpha channel.
I wouldn't call it bug, I'm sure there were discussions in the old time about that.
But: ConvertToRGB32 is filling alpha=255 when converted from RGB24.
This is the way it always worked.
Anyway, RGB32 will slowly die, its a compatibility heritage, I guess it was introduced primarily because 32 bit data was quicker to handle than RGB24 pixels with 3byte pixel boundaries.