Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mutichannel (routing)? #1126

Open
capoei opened this issue Sep 2, 2021 · 87 comments
Open

mutichannel (routing)? #1126

capoei opened this issue Sep 2, 2021 · 87 comments

Comments

@capoei
Copy link

capoei commented Sep 2, 2021

Is there any way to add multi-channel mangement/routing?
just as an example my use case (which would also require the adition of the possibility of a other instance of the convolver):
stereo signal > room correction convolution > crossovers via convolution resulting in 4 channels > routing of channels to my audio interface outputs
@wwmm
Copy link
Owner

wwmm commented Sep 2, 2021 •
edited

Is there any way to add multi-channel mangement/routing?

Unfortunately not. There are other open issues asking for similar multichannel features too. But integrating them in an application that evolved around stereo audio processing is really hard. From the user interface to our presets system and effects pipeline the amount of changes required to handle multichannel features properly is huge. And to make things harder I do not have a surround system. My speakers are a stereo 2.0 from Edifier. They are good but not suitable to actually develop multichannel features.

As what you ask seems to require that only the output node actually need to have more than 2 outputs it may be a little more doable than what other people are asking. Let's keep this issue open for now.

which would also require the adition of the possibility of a other instance of the convolver

Like most of the plugins we use our convolver only has 2 input/output ports. A new plugin capable of outputting more than 2 channels while having 2 input channels would be required. And it would probably have to be the last plugin in the pipeline.

@capoei
Copy link
Author

capoei commented Sep 4, 2021

so there is no wa to include a "split" plugin that splits the plugin chain into 2 or more (stereo ones)?
also when I talked about another instance, it is atm not possible even in the stereo chain. I can't put one convolver after another.
and yes, the crossover would be the last in the chain anyways. so mayby it is posssible to make a "output plugin" where you could put other plugins in paralel?

@capoei
Copy link
Author

capoei commented Sep 4, 2021

the "output plugin" could have a split feature, where on every split you get 2 new outputs with the stereo inputs. that way the 5.1 guys could make 4 chains and control front/back/center/LFE independently on the output

@wwmm
Copy link
Owner

wwmm commented Sep 4, 2021 •
edited

so there is no wa to include a "split" plugin that splits the plugin chain into 2 or more (stereo ones)?

Not inside EasyEffects. But I think it can be done if you create a PipeWire loopback device and set it as your default output. Or if you select it as EasyEffects output in our device selection menu. Loopback devices should be able to do something close to what you want. Take a look at https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1447#note_1006488

@capoei
Copy link
Author

capoei commented Sep 5, 2021

is it maybe possible to extend the output section in the pipewire tab so that we can select a running aplication as output device?

@wwmm
Copy link
Owner

wwmm commented Sep 5, 2021

is it maybe possible to extend the output section in the pipewire tab so that we can select a running aplication as output device?

It depends on what kind of application you are thinking about. Usually the target application is the one that should be exporting a standard output device that would be naturally visible in our device selection menu.

@capoei
Copy link
Author

capoei commented Sep 5, 2021 via email

@capoei
Copy link
Author

capoei commented Sep 5, 2021 via email

@capoei
Copy link
Author

capoei commented Sep 5, 2021 via email

@capoei
Copy link
Author

capoei commented Sep 5, 2021

ok. I use pactl to create a selectable device in EE and manage the conections with pw-link.

@wwmm
Copy link
Owner

wwmm commented Sep 5, 2021

ok. I use pactl to create a selectable device in EE and manage the conections with pw-link.

In these cases where a very specific solution is needed this is probably the better option. It will give you more flexibility.

@Digitalone1
Copy link
Contributor

Digitalone1 commented Sep 5, 2021 •
edited

@capoei which distribution are you using? Could you install easyeffects-git from AUR or compile from source to test the upmixing feature that I'd like to integrate in the application (only for the stream coming from the output pipeline)?

@capoei
Copy link
Author

capoei commented Sep 6, 2021 via email

@capoei
Copy link
Author

capoei commented Sep 6, 2021

@capoei which distribution are you using? Could you install easyeffects-git from AUR or compile from source to test the upmixing feature that I'd like to integrate in the application (only for the stream coming from the output pipeline)?

just installed easyeffects-git-6.1.0.r30 from AUR.
couldn't find said feature. is it suposed to appear in the plugin list?

@Digitalone1
Copy link
Contributor

Digitalone1 commented Sep 6, 2021 •
edited

just installed easyeffects-git-6.1.0.r30 from AUR.
couldn't find said feature. is it suposed to appear in the plugin list?

No, I will add it in the next days.

I will notice you when it's ready.

@capoei
Copy link
Author

capoei commented Sep 6, 2021

ok

@Digitalone1
Copy link
Contributor

@capoei unfortunately the solution I implemented a month ago it's not working anymore.

Until Pipewire does not introduce an easy and transparent way to do channel remapping, or @wwmm to find some better solution, this issue won't be resolved.

@capoei I recommend you to return to stable easyeffects from Arch upstream repository.

@capoei
Copy link
Author

capoei commented Sep 8, 2021

gotcha

@capoei
Copy link
Author

capoei commented Sep 9, 2021

this filter chain stuff is not usefull? https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Filter-Chain#virtual-surround

@wwmm
Copy link
Owner

wwmm commented Sep 9, 2021 •
edited

this filter chain stuff is not usefull?

The filter chain is designed for people that want to set a few filters editing a few configuration files. It is not convenient for use in third party tools. Setting parameters for it on the fly or getting input and output levels will probably be a pain. Assuming it is even possible.

@Digitalone1
Copy link
Contributor

Indeed this should be done by Pipewire in a transparent way, as it is done with resampling.

@Digitalone1 Digitalone1 mentioned this issue Sep 27, 2021
Closed
@Digitalone1
Copy link
Contributor

@capoei if you can test, make a PKGBUILD text file with the following content: https://pastebin.com/ANi6e62u

And install with

makepkg -sri

Open dconf, locate

/com/github/wwmm/easyeffects/loopback-mode

and set it TRUE. Close easyeffects from terminal:

easyeffects -q

Restart it and play something. Let me know if the channel remapping is working.

@capoei
Copy link
Author

capoei commented Sep 27, 2021

Screenshot_20210927_145736

@Digitalone1 Digitalone1 mentioned this issue Oct 17, 2021
Closed
@JohnyPeaN
Copy link

Hi @Digitalone1, could the loopback test fork be brought up to date with master or even merged? I'm using that fork for a while and it is usable. Its at least better solution than no solution.

@Digitalone1
Copy link
Contributor

@JohnyPeaN Can you link it here? I made it months ago, don't even remember it.

@Digitalone1
Copy link
Contributor

@JohnyPeaN are you using #1171? As far as I remember, it was working to me, but then Pipewire upgraded and I was not able to get it working anymore. Is it working to you?

@JohnyPeaN
Copy link

This one.
https://github.com/Digitalone1/easyeffects/tree/loopback
Its version 6.1.2 so those commits are quite off and don't apply.

@JohnyPeaN
Copy link

@JohnyPeaN are you using #1171? As far as I remember, it was working to me, but then Pipewire upgraded and I was not able to get it working anymore. Is it working to you?

Yes, seems to be the same. Working here on newest pipewire (0.3.51).

@Digitalone1
Copy link
Contributor

Digitalone1 commented Aug 22, 2022 •
edited

@ClumsyDerp the tool is helvum. But you don't really need it. Just install pavucontrol and redirect the loopback output to the multichannel device.

@ClumsyDerp
Copy link

Tinkered around with the tools but it appears I got a different setup from JohnyPean.

Helvum Screenshot (attempted to recreate JohnyPean's setup)

Bottom line: The ADI 2 DAC is my actual physical device and only has two channels as shown by helvum and is entirely bypassed in my example, which strikes me as wrong - or rather, I'd have to somehow throw eight channels into two. From my limited understanding I may just be sending my sound in circles right now, if even that 🤡

The "Virtual Surround Sink" is a "fake" Pipewire 7.1 sink, as per their example.

Desired outcome: Apply EE equalizer features but retain 7.1 virtual surround.

@Digitalone1
Copy link
Contributor

You should enable Firefox in EasyEffects. EasyEffects should be set to output to myloop (as it seems to already be configured).

Then you go in Pavucontrol and redirect myloop_playback to the DAC.

@Digitalone1
Copy link
Contributor

For what I said, which is the usual configuration, you won't get the up/downmix because the DAC is only 2 channels.

For what you want, it seems the virtual surround you got is not enough because the 7.1 virtual surround sink is not an output stream. So you can't redirect it in pavucontrol.

You could try to send EasyEffects to 2.0 Virtual Surround Sink and to to redirect the 7.1 Virtual Surround Sink to the DAC. But I'm afraid you can't do it from pavucontrol and the only option is doing it manually, which is not a true downmix like you you wanted.

@Digitalone1
Copy link
Contributor

Oh, now I see from the sink-virtual-surround-7.1-hesuvi.conf that the capture is the 7.1 and the playback is the 2.0, so I think want you want can't be done because we don't have a way to upmix from 2 to 8 channels.

The upmix we suggested here is happening because fortunately we can link 2 channel filters to 2 channel virtual sinks. Then Pipewire does the upmix on its own.

For what you want, you need first an upmix and then a downmix. That configuration is not good and EasyEffects can't do both.

@wwmm
Copy link
Owner

wwmm commented Aug 22, 2022

The ADI 2 DAC is my actual physical device and only has two channels as shown by helvum
Desired outcome: Apply EE equalizer features but retain 7.1 virtual surround.

Even without EasyEffects in the way it does not make much sense to me to create this kind of virtual surround if your sound card has only 2 channels. If I am not mistaken the virtual surround created by PipeWire and Pulseaudio is intended for hardware that actually have all these channels. So even without EasyEffects you won't be retaining the 7.1 surround. It will be downmixed to stereo. Am I missing something here?

@ClumsyDerp
Copy link

ClumsyDerp commented Aug 22, 2022 •
edited

The ADI 2 DAC is my actual physical device and only has two channels as shown by helvum
Desired outcome: Apply EE equalizer features but retain 7.1 virtual surround.

Even without EasyEffects in the way it does not make much sense to me to create this kind of virtual surround if your sound card has only 2 channels. If I am not mistaken the virtual surround created by PipeWire and Pulseaudio is intended for hardware that actually have all these channels. So even without EasyEffects you won't be retaining the 7.1 surround. It will be downmixed to stereo. Am I missing something here?

I can't argue for pipewire's way of handling things, as well as potential magic that is going on under the surface but I can vouch for a spacial upgrade compared to stereo and all directions seem to be fully in tact. It's comparable, if not exceeding Window's spacial sound API - in this instance compared to DTS:X, which I also use/own.

If you happen to own a pair of headphones feel free to try it out for yourself, it only takes a few minutes to set up.

Simply copy the file content from { name = libpipewire-module-filter-chain [...] } into your pipewire.cfg's context.modules = [...] section and replace the hrir.wav's references to an actual reference file on your system.

@wwmm
Copy link
Owner

wwmm commented Aug 22, 2022 •
edited

It's comparable, if not exceeding Window's spacial sound API - in this instance compared to DTS:X, which I also use/own.

Interesting. I had the impression that the downmixing to stereo would kill any spatial feeling that had been added by the virtual surround.

But in this case I do not see why a custom loopback device between your virtual surround sink and EasyEFfects virtual sink should not work. I admit I did not read all the conversation in detail because I am at my work now. But if downmixing is not a problem a loopback from 7.1 to stereo should work.

@Digitalone1
Copy link
Contributor

Digitalone1 commented Aug 22, 2022 •
edited

  • Firefox 2.0 -> EasyEffetcts Sink 2.0 -> MyLoop 2.0 (in EasyEffects enable Firefox, add MyLoop in excluded apps and set MyLoop as EasyEffects preferred output)
  • MyLoop_Playback 7.1 -> Virtual Surround Sink 7.1 (redirect in pavucontrol, add Virtual Surround Sink in EasyEffects excluded apps list)
  • Virtual Surround Sink 2.0 -> ADI-2 DAC 2.0 (redirect in pavucontrol)

Try it. Maybe you're lucky.

@ClumsyDerp
Copy link

ClumsyDerp commented Aug 22, 2022 •
edited

Unfortunately did not work, but I might as well been doing something wrong.

Picture.

I pretty much build what you've suggested, haven't done anything with pavucontrol yet, though.

Additional info:

  • Firefox was playing back a video, so there was audio playing
  • I've cycled all playback devices with both 7.1 sink in EE enabled and disabled an no constellation caused EE to pick up on any "output"
  • MyLoop is set as output
  • Virtual Sink 2.0 -> EE 2.0 connection also makes no difference whether it's in place or not

I'll dig a bit into pavucontrol.

Cheers.

@Digitalone1
Copy link
Contributor

Digitalone1 commented Aug 22, 2022 •
edited

Close Plasma System Settings, it won't help you.

You have to redirect the streams with Pavucontrol (install it if you does not have it).

Uncheck the enable checkbutton for Virtual Surround in EasyEffects (leave it excluded).

In Pavucontrol, first tab (Players), MyLoop_Playback to Virtual Surround and Virtual Surround Sink to ADI DAC.

@ClumsyDerp
Copy link

ClumsyDerp commented Aug 22, 2022 •
edited

Hope this is what it's supposed to be; opening Pavucontrol makes a mess of helvum 🤡

No idea why MyLoop_Playback receives two PulseAudio Volume Controls.

Beyond that the only way to get sound out of FF is to route it against the ADI DAC directly and then neither EE works, nor the surround sound. MyLoop_Playback and Virtual Surround remain dead quiet.

At this point I already feel as though I'm in way over my head 🙈

@Digitalone1
Copy link
Contributor

Digitalone1 commented Aug 22, 2022 •
edited

In pavucontrol is everything good. The issues is virtual surround 2.0 going into EasyEffects sink. Remove that link from helvum.

@ClumsyDerp
Copy link

ClumsyDerp commented Aug 23, 2022 •
edited

Hey, sorry for the late response. Sleep, work, you know the drill.

That said, I got good news. Some of the mess cleaned itself up (MyLoop_Playback no longer has two PulseAudio Volume Controls) upon boot and it now works as expected and further the way you laid out above. Confirmed EQ effects to be applied and the surround sink still plays back on all channels.

Link

Sorry for hijacking this particular issue, should it have been misplaced.

Thanks for the support, much appreciated and if nothing else I hope it'll serve others down the line.

Cheers!

@rijnhard
Copy link

this is probably a naive comment, but if I have to think about my dream scenario for multi-channel audio handling,

multi channel source
          --> (filter all channels) e.g. some funny bunch of filters I slap together for my own Dynamic Range Compression
          --> (start individual channel filters) 
                   | -> FR/FL  --> (2 chan filter) 
                   | -> Centre --> (1 chan filter)
                   | -> RL/RR  --> (2 chan filter)
          --> (end individual channel filters) 
          --> (filter all channels)
          --> FFmpeg encoding for output format

I remember back when I used windows, with K-Lite I used to construct filter & processing graphs in all sorts of weird and wonderful ways. and then re-encode the output so that I could feed it into a sound card or a receiver over HDMI.

@rijnhard
Copy link

rijnhard commented Aug 29, 2022 •
edited

(barring the FFmpeg part which I think is really out of scope unless somehow the output can marked as passthrough for pipewire)

a structure like that could be really flexible for anyone wants to handle any kind of processing,
the choice to combine 2 channels together for a 2channel filter for example is also pretty useful.

It doesn't directly solve upmixing or downmixing but can be a good structural building block for how to cater for that later on.
Maybe easier basic mixing options would simply be:

  • clone a stream e.g. Clone FR [from 2ch source] into RR[multi-chan output]
  • merge stream e.g. merge FR & FL into Centre

Probably the biggest consequence is that filters need to be aware of how many channels they can accept and how many the pipe is trying to provide it, which it shouldn't allow if there's a mismatch.

@wwmm
Copy link
Owner

wwmm commented Aug 29, 2022 •
edited

this is probably a naive comment, but if I have to think about my dream scenario for multi-channel audio handling,

I would not call it naive. In a multichannel setup it is a reasonable expectation to be able to apply different filters for each channel. It just isn't what EasyEffects evolved for. An approach similar the Windows EQualizer APO would make more sense in this case.

Probably the biggest consequence is that filters need to be aware of how many channels they can accept and how many the pipe is trying to provide it, which it shouldn't allow if there's a mismatch.

There are 2 ways to create filters in PipeWire but if I remember well in both of them you have to hardcode the number of filter channels. So the filter would probably have to be made by default with a large number of channels.

But the worse is dealing with third party filters that are designed for stereo. I think that the Windows Equalizer APO uses its own filters for everything. What makes things a lot easier. But most of the ones used in EasyEffects are LV2 filters designed for stereo.

@rijnhard
Copy link

rijnhard commented Aug 29, 2022 •
edited

I would not call it naive. In a multichannel setup it is a reasonable expectation to be able to apply different filters for each channel. It just isn't what EasyEffects evolved for. An approach similar the Windows EQualizer APO would make more sense in this case.

So that would mean a rearchitecture, ouch. Painful. If that were to happen then you would want to basically make it a graph engine.

There are 2 ways to create filters in PipeWire but if I remember well in both of them you have to hardcode the number of filter channels. So the filter would probably have to be made by default with a large number of channels.

I don't think this is a problem really. I'll elaborate below.

But the worse is dealing with third party filters that are designed for stereo. I think that the Windows Equalizer APO uses its own filters for everything. What makes things a lot easier. But most of the ones used in EasyEffects are LV2 filters designed for stereo.

You don't actually need full multi channel support on every filter, most will work fine applied to a single or 2 channels, and with a bit of elbow work and some ui magic (linking a couple instances of the same filter together) you could use 2 channel filters pretty effectively.

With a good foundation you can always work on a specific set of filters being multi channel, or the community can. To be a bit cheeky I don't think that's really the goal here. Of course that's the dream, but not the starting point, by comparison easy effects is still a young project, so I don't think anyone expects it to solve all those rather complex problems in a short timeline.

Of course this is all subject to your vision for it and what you want it to be at the end of the day.

@wwmm
Copy link
Owner

wwmm commented Aug 29, 2022

So that would mean a rearchitecture, ouch. Painful.

Very painful. When I started this project I had things like Viper4Android in mind. And as I have a stereo sound system stereo audio inevitably became a central point in EasyEffects design.

You don't actually need full multi channel support on every filter, most will work fine applied to a single or 2 channels, and with a bit of elbow work and some ui magic (linking a couple instances of the same filter together) you could use 2 channel filters pretty effectively.

As multichannel support is never going to happen to third party stereo plugins ignoring one of the 2 channels would be the only option when trying to apply them in a surround setup. Technically it will probably work but there are some annoyances. Our graphical interface will still try to show 2 level meters for example. So besides the special handling to links in the pipeline the plugin window would have to adapt to each scenario and showing only what makes sense. Nothing particularly complicated to ensure. But I would expect other small inconsistencies to show themselves along the way.

There are also plugins that do not make any sense at all outside of a stereo setup and that we would have to make sure are never added to multichannel processing. The crossfeed and the stereo tools plugins are one example.

But the hard part is that we would be essentially talking about one pipeline for each surround channel. At the end of the day this is what would happen with plugins that have only one or two channels. Handling them and allowing the user to configure them through our window would be quite challenging considering that PipeWire may change the sound card profile at any moment.

Of course this is all subject to your vision for it and what you want it to be at the end of the day.

I think it would be super cool to support multichannel. But I have to be realist. With the main developers (me and @Digitalone1) being stereo users and the amount of work necessary for proper multichannel support being really big I do not see this happening even in the far future. Maybe once we finally start to support multiple filter instances a good solution comes to my mind. But at this moment it feels like a distant dream.

@wwmm wwmm unpinned this issue Nov 26, 2022
@ilippert
Copy link

with respect to #1126 (comment)

This is persistent, no need to create and connect loopback by hand everytime and seems to be working great. Will use it this way to see if it has some caveats.

I agree that it works across boots, great.

However, when I send my computer to sleep/suspension, the loop device stops subsequently working. I reboot, and then it works again.

@wwmm
Copy link
Owner

wwmm commented Nov 28, 2022

However, when I send my computer to sleep/suspension, the loop device stops subsequently working. I reboot, and then it works again.

It may be worth to tell this to PipeWire developers. If only the loopback is not coming back after the PC sleeps it may be a bug on their side.

@ilippert
Copy link

ilippert commented Nov 28, 2022 •
edited

It may be worth to tell this to PipeWire developers.

done – https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2862

@wwmm wwmm mentioned this issue Jan 28, 2023
Open
@wwmm wwmm mentioned this issue Feb 24, 2023
Closed
@wwmm wwmm mentioned this issue Apr 16, 2023
Open
@fiftydinar fiftydinar mentioned this issue Aug 26, 2023
Closed
@kode54
Copy link

kode54 commented Sep 5, 2023

What about upmixing mono playback sources to stereo? Is there a filter I can add that will only upmix as necessary?

@wwmm
Copy link
Owner

wwmm commented Sep 5, 2023 •
edited

What about upmixing mono playback sources to stereo? Is there a filter I can add that will only upmix as necessary?

There is the Stereo Tools plugin but it is not going to do it ondemand only to what is necessary. It will change the stereo matrix of everything that is passing through the effects pipeline.

@b1r63r
Copy link

b1r63r commented Feb 9, 2024 •
edited

I think my proposed roadmap would be

  1. offer the same number of inputs as the selected output
    coding would then be limited to handling the difficulties in switching number of inputs when output switches etc.
    run effects only on the front stereo speakers and pass everything else through unchanged.
    this would of course lead to some timing issues between channels, but ok for now.

  2. add basic timing adjustments for individual channels
    i guess this will be needed to delay the pass-through channels so they sync up with the filtered ones, as well as to cater for speaker placement in 5.1 or 7.1 systems
    these must be tied to the selected output since 5.1 speaker system needs different timings than a 5.1 headphone.

  3. add handling of the subwoofer channel.
    offer the ability to remove the sub channel from what is presented to the clients so that filtering on the front channels includes low frequency signal. then apply a 2.0 -> 2.1 filter that directs lf output to the sub (of course given that the selected output is one with a subwoofer channel). this way the subwoofer gets integrated into the filter chain without having to deal with true multi channel yet.

  4. think hard about how to do proper multi channel filtering.
    maybe by implementing a multi-channel equalizer as first step, keep lv2 effects as 2 channel only, and define a way to make new multi channel filters. Maybe there is a standard already for multi channel plugins?

i think this would give the fastest path to enabling a useable multi-channel easyeffects. then let it evolve from there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@kode54 @b1r63r @rijnhard @wwmm @JohnyPeaN @ilippert @capoei @Digitalone1 @Winnus @ClumsyDerp