topology: sof-adl-max98357a-rt5682-waves-2way: solution without changing waves wrapper code

[johnylin76]

Add support for four max98357a speaker amplifiers running in TDM mode which format is 8 slots with 32 bit slot/sample width on ADL boards.

To implement the 2-way woofer/tweeter speaker function in SOF, there is a demux to create 2 streams for the branched pipelines. A Waves-codec widget is applied before the demux for audio quality enhancement, and the other Waves-codec
widgets are located one per branch for EQ/DRC including band-split filters of Woofer and Tweeter.

There is a mux before DAI to merge the branched Woofer and Tweeter streams into one 4-channel stream which is fed into DAI then.

The formation of this split-and-merge pipeline is a bit tricky. The topology graph is as following:

  #host PCM0 -> WAVES -> DEMUX -> WAVES ---+                         <PIPE#9>
                             |             |
                             +--> WAVES -> MUX -> DAI (Speaker SSP)  <PIPE#1>

There are two pipelines with ID #1 and #9, where PIPE#9 is set DAI#1 as the scheduled component to make them being scheduled together. They are connected via DEMUX#9 and MUX#1.

The priority of PIPE#9 should be higher than PIPE#1 to assure PIPE#9 is scheduled first (PCM should be the first component for the copy process).

Moreover, the commit of mux_params() modification is required to support the split-and-merge pipeline design. In the commit mux_params() will update the stream channels for each sink buffer with regard to lookup tables, which are sourced from the matrices we provided in the topology file.

We know that the mux (or demux) is the channel-adjustable widget which handles the channel remap accordingly. In the existing cases so far, there is at most one mux or demux lied on the pipeline graph for each single endpoint device. The stream channels of all pipeline buffers will be distinguished by the both ends of the mux (or demux). Buffers located in the path toward PCM end will adopt channels as PCM params, while the ones located in the path toward DAI end will adopt channels as HW params.

However, in our case one demux and one mux are placed. That is, after the cuts on the spot of mux and demux there are 3 pieces: the path toward PCM end, the path toward DAI end, and the intermediate paths between mux and demux.

In the current pipeline_params() propagation the channels update will occur at the first mux/demux it traversed starting from PCM (downstream propagation for playback pipelines). So the stream params setup along pipeline buffers will be:

  #host PCM0[2] -[2]-> WAVES -[2]-> DEMUX -[4]-> WAVES -[4]---+
                                        |                     |
                                        +--[4]-> WAVES -[4]-> MUX -[4]-> DAI (Speaker SSP[4])

(The numbers in brackets[ ] are the setup channels after for buffers, where PCM params specifies 2-channel and HW params obtained from TDM SSP is 4.)

This setup is not as our expectation. The demux here performs stream duplication so the stream channels should be 2 for both sink and source buffers (2 sinks and 1 source). The mux plays the role of stream merger so the source buffers are 2-channel and the sink is 4-channel (1 sink and 2 sources).

  #host PCM0[2] -[2]-> WAVES -[2]-> DEMUX -[2]-> WAVES -[2]---+
                                        |                     |
                                        +--[2]-> WAVES -[2]-> MUX -[4]-> DAI (Speaker SSP[4])

In practice, this information is specified in the mux/demux matrices provided in the topology, which are interpreted to lookup tables stored in comp_data of mux/demux. Therefore, "effective channels" could be counted by the number of valid copy elements in lookup tables for each sink buffer. We should apply "effective channels" to the downstream buffers during pipeline_params() propagation.

There are assumptions on calculating "effective channels", such like the output stream doesn't have the mixing channel (mapped to multiple input streams or channels), or the empty channel either (mapped to nothing).

[plbossart]

It really feels like this could be handled with a DEMUX/MUX, and just having some means of creating an 8ch stream and process it as needed. It'd be a lot more efficient than with those split-joined pipelines, no?

[johnylin76]

I forgot to make the problem statement and the design purpose first. Let me make the illustration here.

For project Taniks, the internal speaker has 4 amplifiers so 4-channel stream is requested by HW params. The first two channels are the left/right input of Woofer (low-freq bass speakers) and the last two channels are the left/right input of Tweeter (mid & high-freq speakers). The speaker PCM is the 2-channel (stereo) source. That is, the band-split process is required through the pipeline to transform 2-channel stream [L,R] into 4-channel [WL,WR,TL,TR].

The Waves codec widget is provided by Waves Audio partner, which is implemented by their proprietary library with a codec_adapter wrapper. The limitation of their library is that it only supports the processing of 2-channel audio stream with interleaved format. That's the reason I split into 2 pipelines(2-channel) by demux instead of 1 pipeline(4-channel). A Waves widget is connected to each pipeline for EQ+DRC tuning. EQ inside the Waves widget should perform the band-splitting to partition the low-freq audio and mid & high-freq for Woofer and Tweeter respectively. The Waves widget before demux is in charge of audio enhancement.

        cfg:audio enhancement   cfg:low-pass + drc (woofer)
                          :                         :
   #host PCM0[2] -[2]-> WAVES -[2]-> DEMUX -[2]-> WAVES -[2]---+
                                         |                     |
                                         +--[2]-> WAVES -[2]-> MUX -[4]-> DAI (Speaker SSP[4])
                                                    :
                               cfg:high-pass + drc (tweeter)

I had proposed another solution which doesn't split the pipeline but needs major modification on the wrapper code #5157
In short the wrapper solution supports the adoption of Waves library on 4-channel stream input/output. However due to the limitation it requires extra costs of both time and memory to extract the interleaved samples from 4-channel to 2-channel stream inside the wrapper.

I prefer the solution provided in this PR more because this is more modularized and more extensible. We don't need to modify the wrapper code which should be owned and maintained by the partner accordingly.

[lgirdwood]

@singalsu any inputs ? and do we need to make the frag API updates to mux/demux ?
@johnylin76 how is performance and config, have you seen any issues ? Just asking as the topology has a loop (and we detect this in the walk), but I think you are the first person to test loops in real life.

[plbossart]

@johnylin76 Now I get what you're trying to do, and IMHO you've chosen the complex path with separate pipelines. This may work out of the box if you're lucky or require fundamental changes to the pipeline logic since this sort of topology has never been used. changing the library to use a 'stride' parameter would have been more self-contained.

@lgirdwood there's no loop here, more of a unidirectional split-join, but to your point I don't see how the logic of propagating params/commands would work in this case: there is a risk that the downstream part gets the same information twice (two hw_params/trigger. etc).

[johnylin76]

Yeah this still requires more testings and will be also verified on Waves partner side and ODM side. I didn't see any issue with this topology. I also have traced the walk of all stages: pipeline init, set_params, set_hw_params, trigger, task scheduling, and copy. They are all as expected with the frag update on mux/demux.

By the way, theoretically that is not a "loop" since the flow is directional in tplg graph. The "loop" should be more complicated to deal with by the walk algorithm.

To make the further illustration of the scheduling of the proposed speaker pipeline implementation, first to know that both PIPE#9 and PIPE#1 will be put into the schedule list during the trigger walking started from PCM0. And PIPE#9 is prior to PIPE#1.

Upon per scheduled pipeline task, the processing order (comp_copy()) of each component on the speaker pipeline is as follows:

order:          1               2            3            4
       #host PCM0[2] -[2]-> WAVES -[2]-> DEMUX -[2]-> WAVES -[2]---+                                PIPE#9
                                             |                     |
                                             +--[2]-> WAVES -[2]-> MUX -[4]-> DAI (Speaker SSP[4])  PIPE#1
order:                                                    5          6          7

This order assures that for the data in source buffer(s) is always ready on comp_copy() call for all components.

[johnylin76]

To keep the existing cases unaffected, maybe adding a flag in config would be better to apply buffer channels by lookup tables and keep it off as default.
Or a config flag to indicate whether to propagate channels for stream parameter: 0 -> NOT propagate (the default behavior); 1 -> propagate. This is even simpler. Thoughts?

[lgirdwood]

@johnylin76 we were looking at something like this to propagate and align configuration within pipelines. #1280
It's partially upstream atm, would this fit in with your case ?

[johnylin76]

Thanks for the information.
As I see it #1280 had derived the solution that performs hw_params walk from DAI and then PCM params walk from host to overwrite, with comp_params() for supporting parameter tuning. It's like a turning point between PCM and HW params, for example:

host (16kHz) -----> SRC -----> DAI (48kHz)

SRC is the turning point, and sample rate is changed from PCM to HW preference. However it only works for cases with single turning point. If there are two turning points:

host (16kHz) -----> SRC1 -----(32kHz)----> SRC2 -----> DAI (48kHz)

How do we handle the parameter of the intermediate slice (from SRC1 to SRC2) if our expectation is neither PCM nor HW preference?

My case is kind of alike as two-turning-points case, where both DEMUX and MUX make channels update.

host (2-ch) ----> DEMUX ---(?-ch)---> MUX ----> DAI (4-ch)

IMHO it may be not enough to provide two choices (PCM or HW preference) only for the propagation of pipeline params.

[sys-pt1s]

Can one of the admins verify this patch?

[johnylin76]

PR is obsolete