support audio visualizations in player #748

[ElderOrb]

No description provided.

[dericed]

This is a great start and thx for breaking it up into the debug section.

I tested this by enabling Normal, Waveform, Audio Bit Scope, and Audio Volume.

the debug output for that is

*** defined video filters ***: 

format=yuv444p,scale
format=yuv422p10le,waveform=intensity=0.1:mode=column:mirror=1:c=1:f=0:graticule=green:flags=numbers+dots:scale=0
*** defined audio filters ***: 

abitscope,drawgrid=w=iw:h=ih/8:t=1:c=gray@0.9
showvolume
*** VIDEO ***: 


*** layout ***: 

0_0|w0_0|0_h0|w0_h0|0_h0+h1|w0_h0+h1
*** video split ***: 

sws_flags=neighbor;split=2[x1][x2];
*** videoFilterString ***: 

[x1]format=yuv444p,scale[y1];[x2]format=yuv422p10le,waveform=intensity=0.1:mode=column:mirror=1:c=1:f=0:graticule=green:flags=numbers+dots:scale=0[z2];[z2][y1]scale2ref[y2][y1];
*** videoXstackInput ***: 

[y1][y2]
*** result ***: 

sws_flags=neighbor;split=2[x1][x2];[x1]format=yuv444p,scale[y1];[x2]format=yuv422p10le,waveform=intensity=0.1:mode=column:mirror=1:c=1:f=0:graticule=green:flags=numbers+dots:scale=0[z2];[z2][y1]scale2ref[y2][y1];[y1][y2]xstack=fill=slategray:inputs=2:layout=0_0|w0_0|0_h0|w0_h0|0_h0+h1|w0_h0+h1
*** AUDIO ***: 


*** audio split ***: 

asplit=2[a1][a2];
*** audioFilterString ***: 

[a1]abitscope,drawgrid=w=iw:h=ih/8:t=1:c=gray@0.9[b1];[a2]showvolume[b2];
*** audioXstackInput ***: 

[b1][b2]
*** result ***: 

asplit=2[a1][a2];[a1]abitscope,drawgrid=w=iw:h=ih/8:t=1:c=gray@0.9[b1];[a2]showvolume[b2];[b1][b2]xstack=fill=slategray:inputs=2:layout=

So to build the result we need:

• adjust the xstack of the audioFilterString:result to use a layout fixed to 0_0|0_h0|0_h0+h1|0_h0+h1+h2|0_h0+h1+h2+h3|0_h0+h1+h2+h3+h4 so the result would be

asplit=2[a1][a2];[a1]abitscope,drawgrid=w=iw:h=ih/8:t=1:c=gray@0.9[b1];[a2]showvolume[b2];[b1][b2]xstack=fill=slategray:inputs=2:layout=0_0|0_h0|0_h0+h1|0_h0+h1+h2|0_h0+h1+h2+h3|0_h0+h1+h2+h3+h4

• then if both videoFilterString:result is non-empty and audioFilterString:result is non-empty then do

videoFilterString[VFS];audioFilterString[AFS];[VFS][AFS]xstack=fill=slategray:inputs=2:layout=0_0|0_h0

else just the non-empty result as is. I suppose if both are empty then use Normal, so for above result the full combined audio and video result should be

sws_flags=neighbor;split=2[x1][x2];[x1]format=yuv444p,scale[y1];[x2]format=yuv422p10le,waveform=intensity=0.1:mode=column:mirror=1:c=1:f=0:graticule=green:flags=numbers+dots:scale=0[z2];[z2][y1]scale2ref[y2][y1];[y1][y2]xstack=fill=slategray:inputs=2:layout=0_0|w0_0|0_h0|w0_h0|0_h0+h1|w0_h0+h1[VFS];asplit=2[a1][a2];[a1]abitscope,drawgrid=w=iw:h=ih/8:t=1:c=gray@0.9[b1];[a2]showvolume[b2];[b1][b2]xstack=fill=slategray:inputs=2:layout=0_0|0_h0|0_h0+h1|0_h0+h1+h2|0_h0+h1+h2+h3|0_h0+h1+h2+h3+h4[AFS];[VFS][AFS]xstack=fill=slategray:inputs=2:layout=0_0|0_h0

When I try locally with ffmpeg/ffplay, this works:

ffmpeg -i /Betacam_SP_Test_ffv1.mkv -filter_complex "sws_flags=neighbor;split=2[x1][x2];[x1]format=yuv444p,scale[y1];[x2]format=yuv422p10le,waveform=intensity=0.1:mode=column:mirror=1:c=1:f=0:graticule=green:flags=numbers+dots:scale=0[z2];[z2][y1]scale2ref[y2][y1];[y1][y2]xstack=fill=slategray:inputs=2:layout=0_0|w0_0|0_h0|w0_h0|0_h0+h1|w0_h0+h1[VFS];asplit=2[a1][a2];[a1]abitscope,drawgrid=w=iw:h=ih/8:t=1:c=gray@0.9[b1];[a2]showvolume[b2];[b1][b2]xstack=fill=slategray:inputs=2:layout=0_0|0_h0|0_h0+h1|0_h0+h1+h2|0_h0+h1+h2+h3|0_h0+h1+h2+h3+h4[AFS];[VFS][AFS]xstack=fill=slategray:inputs=2:layout=0_0|0_h0" -f nut - | ffplay -