Does AviSynth+ internally support P010?

[CrendKing]

When I send P010 data to AviSynth+, claiming it as VideoInfo::CS_YUV420P10 and then call some internal functions like Subtitle() or Info(), the text are blurry and green (which should be clear yellow text). However, if I claim it as VideoInfo::CS_YUV420P16, everything works fine.

I tried all CS_YUV420P10, CS_YUV420P12, CS_YUV420P14 and CS_YUV420P16. Only CS_YUV420P16 produces correct output. Other plugins like MVTools also only works when I'm claiming CS_YUV420P16.

Is this a bug or CS_YUV420P10 has completely different meaning?

[pinterf]

P010 is supported through AviSource (see AviSource.cpp)
I made further conversions for P210, P216, P010, P016, v410, v308, v408

But inside Avisynth and the filters solely the planar YUV formats are used. Formats mentioned above are only avaliable through VfW or Avisource interface.

You can check Avihelper.cpp for conversions, e.g. Px10_16_to_yuv42xp10_16

http://avisynth.nl/index.php/AviSource

[pinterf]

Maybe 010 formats are not using 10 bits on LSB but rather on the MSB part. In Avisynth 10 bit data have to be between 0 and 1023. Filters should garantee that range and expect data using this rule.

[CrendKing]

Here is what I did. I run the following commands to get an one-frame P010 video:

ffmpeg -i input.mp4 -c:v rawvideo -pixel_format yuv420p10le -frames:v 1 test.yuv
ffmpeg -pixel_format yuv420p10le -video_size 1280x720 -i test.yuv test.mp4

If I open the "test.yuv" with a hex editor, I can clearly see all WORDs are within the 0x3FF range (little-endian). However, when I play test.mp4 in MPC-HC, the data my filter gets from upstream (e.g. LAV video decoder) are exceeding 0x3FF range. I can confirm the input pin's media type FourCC is P010. Since this is one-frame video, I was expecting the decoded data should exactly match test.yuv. See the screenshots.

What am I doing wrong? Are those bytes some sort of compressed data? Sorry if this does not seem to be very related to AviSynth at this point. I tried to search but can't find much information about what P010 decoded data should look like.

[pinterf]

P010 stores values left shifted 6 bits, as if the values were originally 16 bits.
https://docs.microsoft.com/en-us/windows/win32/medfound/10-bit-and-16-bit-yuv-video-formats

"When I send P010 data to AviSynth+, claiming it as VideoInfo::CS_YUV420P10"
Try shifting back your data by 6 bits before you send it a Avisynth.

I don't know if it works:
clip_10bit_but_really_16bit = your input
real_10bit = clip_10bit_but_really_16bit.ConvertBits(10, truerange=false)

Or or Expr
real_10bit = clip_10bit_but_really_16bit.Expr("x 64 /")

When I have put all these formats into AviSource, there is a technique when we are able to request a specific format during the negotiation phase. When I asked for P010 from the input provider I got it. If I asked for YV12 I got it (clearly the driver did the format conversion for me).

I was playing with MagicYUV, and I could ask for a bunch of different formats even when the file itself had a propriatery MagicYUV FourCC. The driver was able to convert it to P010 whatever special MagicYUV FourCC I was using for the file generation originally.

[pinterf]

Does ffmpeg have pix format p010le?

[CrendKing]

Thanks. I got it now. I misunderstood the picture from Microsoft Docs. Like you said, I just need to right shift every WORD by 6 bits. Also, the reason why faking as P016 works is because, like that Microsoft document says, P016 is just a more scaled P010 (by 2^6=64 times). Without right shift, P010 is P016. Also, no precision loss as it says. I'll just stick to faking P016. Feel free to close the issue.

In short, P010 has MSB zero-padded, while AviSynth expects LSB zero-padded, correct?

Still, it would be nice if AviSynth could take the 6-MSB-padded WORD as P010 input without asking every source plugin to do the convertion.

[CrendKing]

Does ffmpeg have pix format p010le?

My outputs:

# ffmpeg -pix_fmts | grep yuv420
ffmpeg version git-2020-02-20-56df829 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9.2.1 (GCC) 20200122
...

IO... yuv420p                3            12
IO... yuv420p16le            3            24
IO... yuv420p16be            3            24
IO... yuv420p9be             3            13
IO... yuv420p9le             3            13
IO... yuv420p10be            3            15
IO... yuv420p10le            3            15
IO... yuv420p12be            3            18
IO... yuv420p12le            3            18
IO... yuv420p14be            3            21
IO... yuv420p14le            3            21

[pinterf]

Avisynth+ and all plugins (incl. source plugins) must follow and use only the supported formats defined in avisynth.h. Or else, how do you tell Avisynth that your format is special? If you write a source plugin then you can write methods and accept whatever exotic formats from an upper level, like AviSource does which accepts and converts a zillion formats on its interface (see: http://avisynth.nl/index.php/AviSource)

[qyot27]

It should have been:

E:\Documents>ffmpeg -pix_fmts list | grep p010
ffmpeg version r96207+6 master-f1353ce222 HEAD-24f87c7974
 contains: datetime new_pkgconfig silent_invoke vapoursynth_alt versioninfo
 Copyright (c) 2000-2019 the FFmpeg developers
  built on Dec 30 2019 21:19:14 with gcc 9.2.0 (GCC)
  libavutil      56. 38.100 / 56. 38.100
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.102 / 58.  9.102
  libavfilter     7. 70.101 /  7. 70.101
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
IO... p010le                 3            15
IO... p010be                 3            15

P010 and YUV420P10 are not the same, pix_fmt wise. YUV420P10 is fully planar, P010 is not (planar luma, packed chroma). NV12 isn't supported internally for exactly the same reason.

[CrendKing]

Avisynth+ and all plugins (incl. source plugins) must follow and use only the supported formats defined in avisynth.h. Or else, how do you tell Avisynth that your format is special? If you write a source plugin then you can write methods and accept whatever exotic formats from an upper level, like AviSource does which accepts and converts a zillion formats on its interface (see: http://avisynth.nl/index.php/AviSource)

Was the format I'm sending (zero padding at MSB) the standard format or special? I thought that's the one the Microsoft page is describing. That's why when I re-read your comment In Avisynth 10 bit data have to be between 0 and 1023 I figured AviSynth is using different format.

Anyways, I'm not sure if this happens commonly. Since you've already have all the conversion logic done in AviSource, you could add something like VideoInfo::CS_YUV420P10_MSB_ZERO to the avisynth.h list, and move the shifting logic into the core, so that both AviSource and other source filters don't have to repeat the same boilerplate code.

I don't know much of the development philosophy of this project, so don't take me too seriously.

[CrendKing]

P010 and YUV420P10 are not the same, pix_fmt wise. YUV420P10 is fully planar, P010 is not (planar luma, packed chroma). NV12 isn't supported internally for exactly the same reason.

I just tried the p010le format. It produces the exactly same rawvideo file as yuv420p10le. But the re-encoded mp4 is just extremely small and has one green frame. The frame data has the zero padded at LSB though. Interesting.

[qyot27]

I'm not entirely up on the exact details, but my understanding is that P010, et. al, are largely interchange formats between software and the graphics display stack - meant for consumption by the GPU or the APIs dealing with them, like OpenGL. So on a certain level, they are equivalent, but at other points, they definitely aren't. And how you generate such a file may be silently correct, or silently incorrect based on assumptions the software is making about what it's being told to do; I doubt most formats you can pack into MP4 support P010 as-is, and ffmpeg automatically tries to convert back to yuv420p10, with something getting borked in that chain somewhere when the output is wrong.

How does ConvertBits(10, truerange=false) inserted into the filterchain immediately after loading the video act in regard to the output? Because this sounds kind of like a reason why the truerange option exists, if forcing YUV420P16 works.

You can see the actual difference in output if you use FFmpeg to generate unencapsulated rawvideo for yuv420p10le and p010le and then try to play it back in ffplay¹. If you don't get the pix_fmt value exact, it results in clearly wrong output. The CRC32s also don't match, even though the filesizes do.

¹with a script containing Version().ConvertToYUV420().ConvertBits(10) so we have YUV420P10 output from AviSynth+
ffmpeg -i test.avs -f rawvideo -vcodec rawvideo yuv420p10le.yuv and
ffmpeg -i test.avs -f rawvideo -vcodec rawvideo -pix_fmt p010le p010le.yuv.

When playing them back with ffplay, you have to declare the resolution and pix_fmt, but it plays correctly if you use the correct pix_fmt, and incorrectly if you use the other.

ffplay -s 384x104 -pix_fmt yuv420p10le -i yuv420p10le.yuv == correct
ffplay -s 384x104 -pix_fmt p010le -i yuv420p10le.yuv == incorrect, solid green output with very hard-to-see text

ffplay -s 384x104 -pix_fmt p010le -i p010le.yuv == correct
ffplay -s 384x104 -pix_fmt yuv420p10le -i p010le.yuv == incorrect, pink glitchy output

[CrendKing]

Sorry took this long to reply. It looks like p010le.yuv is basically what I'm getting in the filter, and yuv420p10le.yuv is what AviSynth expects. Every 16-bit WORD of yuv420p10le is 1/64 of the corresponding p010le value. So technically avisynth.h is correct that CS_YUV420P10 corresponds to yuv420p10le. I'm just wondering if AviSynth+ could add more formats like p010le to the core. I understand every source plugin developer could do the conversion before passing to AviSynth+, but it is certainly convenient if AviSynth+ could handle more cases.

If this is not at any priority, I'm totally fine. If this request is remotely relevant, I can open a cleaner ticket if you want.

[qyot27]

The only formats we'll accept going forward¹ are fully planar. The few packed formats that do exist in the core are there because they were grandfathered in from classic AviSynth (RGB24, RGBA32, and YUY2), or were adjacent to the ones that were grandfathered in (RGB48, RGBA64).

¹'going forward' meaning since 2016, when the pixel format list was expanded, but effectively even before that, as the goal is to move to all the existing packed formats in the core to getting processed internally as planar.

I suppose one course of action that wouldn't clutter avisynth{_c}.h with pixel formats filter authors will never use is to have a RawConvert() filter that can take arbitrary pixel formats and override them by doing the plane unpacking and bit shifting, via individual arguments or definition files that describe the input format. That way any exotic color format could be processed in/out of the core without any of the filters needing explicit support for them.

input()
RawConvert(input_component_order="y0 y1 y2 u0 v0 u1 v1 u2 v2", output_component_order="y0 y1 y2 u0 u1 u2 v0 v1 v2 a0 a1 a2", [bit shift options per component], bit_depth=10) # taking P010 data and spitting out YUVA420P10
...processing filters that expect YUVA420P10...
RawConvert(input_component_order="y0 y1 y2 u0 u1 u2 v0 v1 v2 a0 a1 a2", output_component_order="y0 y1 y2 u0 v0 u1 v1 u2 v2", [bit shift options per component], bit_depth=10)

or the definition files idea:

input()
RawConvert(input_definition="p010.txt", output_definition="yuva420p10.txt", bit_depth=10) # where the .txt files have to have a certain syntax to be understood as a color format definition file.
...processing filters that expect YUVA420P10...
RawConvert(input_definition="yuva420p10.txt", output_definition="p010.txt", bit_depth=10)

And then give it to a program that expects P010, and basically just tell it to ignore the fact that AviSynth+ is still telling it that the format it's serving out is YUVA420P10. If that's even a necessity.

[CrendKing]

The only formats we'll accept going forward¹ are fully planar

I didn't know this is part of AviSynth's philosophy, partly because like you said you have RGB, and partly because http://avisynth.nl/index.php/Avisynthplus_color_formats didn't mention anything.

The RawConvert() idea is clean on both end (source developer and core developer). The syntax is a bit too detailed for either, but I guess as long as it is not exposed to end users, it is fine. I mean, I can call ConvertBits() today to solve the specific P010 problem, right?

[qyot27]

If it works, yeah.

[CrendKing]

So just FYI, recently I took some time to implement the "shift by 6 bit" change in my project. Basically, as mentioned earlier, I right shift every WORD by 6 bit of received p010le data to AviSynth+ as yuv420p10le, and left shift on the other end. Everything worked as expected. Still, since the process is relatively costly, and faking as p016 is lossless, I will just keep that change as POC.

Still, again, if this kind of conversion is useful not only to my project but also potentially to other use cases of AviSynth+, you guys could add this small function to your conversion library.

[pinterf]

I don't think it is useful in general, there are a zillion of other formats, one would question, why not support P210, P216, P010, P016, v410, v308, v408, etc (I have done some of these conversions in AviSource), then why not big-endian and little-endian, I'd better not add more complexity to this part of Avisynth.

[CrendKing]

Well, P010 and P210 both have exactly the same problem, so that logic solve both. P016 and P216 are exactly what the solution currently are, so there's no need. The others you mentioned might not be as popularly used as, but as a generic engine, I completely understand the position you guys take. So fair enough. Feel free to close the ticket.