Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow running multiple profiles/pipelines simultaneously and per-app profile/pipelines selection #1586

Open
v-fox opened this issue Jun 20, 2022 · 7 comments

Comments

@v-fox
Copy link

v-fox commented Jun 20, 2022

EasyEffects Version

6.2.5+124~git20220617.753a863d

What package are you using?

openSUSE

Distribution

Tumbleweed (rolling)

Describe the bug

Ideally, all audio sources should target 24-bit/144dB mixing with baseline of somewhere around -24 (actually, -48 but with automatic DRC). However, that would require always maxing out user volume which is a major health hazard for ears. Especially, considering that only movies are mixed properly most of the time and games are in majority of cases.

In real world there are at least 3 distinct casual cases:

  1. Games that properly target headphones, professional music albums and movies when user wants full dynamic range.
  2. Improperly mixed movies and games or when user wants to boost & limit volume with DRC.
  3. Bad sources which are neither normalized or limited and have no business attempting to use full DR. This is majority of online web-media.

To accommodate them I have to constantly switch default output in a PA mixer and also switch profiles in EE which is a major annoyance. Especially with EE not having knobs that could be promptly turned up/down with a mouse.

Expected Behavior

For such cases I made up separate profiles with different amount of aggressiveness in the limiter (and exciter, which is good for haters of "the cold digital sound"). I would like to use #1 for an audio player and by default in headphone DAC/output, #2 for a video player and by default in speaker DAC/output and #3 for web browsers on both outputs (with option to switch to it for a video player when dealing with badly mixed local files).

Debug Log

No response

Additional Information

~/.config/easyeffects/output/default.json
{
    "output": {
        "blocklist": [],
        "exciter": {
            "amount": 1.0,
            "blend": 0.0,
            "ceil": 16000.0,
            "ceil-active": false,
            "harmonics": 6.0,
            "input-gain": 0.0,
            "output-gain": 0.0,
            "scope": 7902.0
        },
        "limiter": {
            "alr": true,
            "alr-attack": 4.0,
            "alr-knee": 3.0,
            "alr-release": 768.0,
            "attack": 8.0,
            "dithering": "24bit",
            "external-sidechain": false,
            "gain-boost": true,
            "input-gain": 0.0,
            "lookahead": 8.0,
            "mode": "Herm Thin",
            "output-gain": 0.0,
            "oversampling": "Full x3(3L)",
            "release": 8.0,
            "sidechain-preamp": 0.0,
            "stereo-link": 100.0,
            "threshold": 0.0
        },
        "plugins_order": [
            "stereo_tools",
            "exciter",
            "limiter"
        ],
        "stereo_tools": {
            "balance-in": 0.0,
            "balance-out": 0.0,
            "delay": 0.0,
            "input-gain": -3.0,
            "middle-level": 0.0,
            "middle-panorama": 0.0,
            "mode": "LR > LR (Stereo Default)",
            "mutel": false,
            "muter": false,
            "output-gain": 0.0,
            "phasel": false,
            "phaser": false,
            "sc-level": 1.0,
            "side-balance": 0.0,
            "side-level": 0.0,
            "softclip": false,
            "stereo-base": 0.0,
            "stereo-phase": 0.0
        }
    }
}
~/.config/easyeffects/output/default-boosted.json
{
    "output": {
        "blocklist": [],
        "exciter": {
            "amount": 3.0,
            "blend": 0.0,
            "ceil": 16000.0,
            "ceil-active": false,
            "harmonics": 6.0,
            "input-gain": 0.0,
            "output-gain": 0.0,
            "scope": 7902.0
        },
        "limiter": {
            "alr": true,
            "alr-attack": 4.0,
            "alr-knee": 3.0,
            "alr-release": 768.0,
            "attack": 8.0,
            "dithering": "24bit",
            "external-sidechain": false,
            "gain-boost": true,
            "input-gain": 0.0,
            "lookahead": 8.0,
            "mode": "Herm Thin",
            "output-gain": 0.0,
            "oversampling": "Full x3(3L)",
            "release": 8.0,
            "sidechain-preamp": 0.0,
            "stereo-link": 100.0,
            "threshold": -12.0
        },
        "plugins_order": [
            "stereo_tools",
            "exciter",
            "limiter"
        ],
        "stereo_tools": {
            "balance-in": 0.0,
            "balance-out": 0.0,
            "delay": 0.0,
            "input-gain": -3.0,
            "middle-level": 0.0,
            "middle-panorama": 0.0,
            "mode": "LR > LR (Stereo Default)",
            "mutel": false,
            "muter": false,
            "output-gain": 0.0,
            "phasel": false,
            "phaser": false,
            "sc-level": 1.0,
            "side-balance": 0.0,
            "side-level": 0.0,
            "softclip": false,
            "stereo-base": 0.0,
            "stereo-phase": 0.0
        }
    }
}
~/.config/easyeffects/output/default-overloaded.json
{
    "output": {
        "blocklist": [],
        "exciter": {
            "amount": 3.0,
            "blend": 0.0,
            "ceil": 16000.0,
            "ceil-active": false,
            "harmonics": 9.0,
            "input-gain": 0.0,
            "output-gain": 0.0,
            "scope": 7902.0
        },
        "limiter": {
            "alr": true,
            "alr-attack": 4.0,
            "alr-knee": 6.0,
            "alr-release": 768.0,
            "attack": 8.0,
            "dithering": "24bit",
            "external-sidechain": false,
            "gain-boost": true,
            "input-gain": 0.0,
            "lookahead": 8.0,
            "mode": "Herm Thin",
            "output-gain": 0.0,
            "oversampling": "Full x3(3L)",
            "release": 8.0,
            "sidechain-preamp": 0.0,
            "stereo-link": 100.0,
            "threshold": -24.0
        },
        "plugins_order": [
            "stereo_tools",
            "exciter",
            "limiter"
        ],
        "stereo_tools": {
            "balance-in": 0.0,
            "balance-out": 0.0,
            "delay": 0.0,
            "input-gain": -3.0,
            "middle-level": 0.0,
            "middle-panorama": 0.0,
            "mode": "LR > LR (Stereo Default)",
            "mutel": false,
            "muter": false,
            "output-gain": 0.0,
            "phasel": false,
            "phaser": false,
            "sc-level": 1.0,
            "side-balance": 0.0,
            "side-level": 0.0,
            "softclip": false,
            "stereo-base": 0.0,
            "stereo-phase": 0.0
        }
    }
}
@wwmm
Copy link
Owner

wwmm commented Jun 20, 2022

Especially with EE not having knobs that could be promptly turned up/down with a mouse.

I did not understand. The controls accept mouse input. Or are you talking about the way audio plugins usually handle controls with the mouse? I mean that way where you click and drag the mouse to change values. If yes this isn't how gtk and QT widgets usually interact with the mouse. So probably can not be done.

per-app profile/pipelines selection

Some people asked this in the past. I do not remember the issues. But the problem is that it is almost on the same level of creating another application. Our window would have to go through some serious redesigning for per app pipelines to be controlled. And there is also the question about how to handle multiple pipelines and the application redirection.

As almost all apps still use the Pulseaudio API a null sink still is desirable when redirecting them because they can not see PipeWire filters. I can force the link from the PipeWire side but the app device selection won't be able to to that. But if null sinks are going to be used this means having one null sink for each pipeline. It will be messy...

Another headache is handling the presets... But the worse is that the people that do not need this will probably ask for the current behavior and window workflow to be still available. Doing all of this requires a huge amount of work and a very clever redesigning. That is why I say it is on the same level as creating another application.

Implementing automatic preset loading based on the selected output/input device already gave me way more headaches than I was expecting. Multiple pipelines on top of this is something I am not sure it is going to be doable without another huge round of work.

I have to be honest... It would be a nice feature to have but I doubt it is going to happen. Even multiple filters instances in the same pipeline is already requiring some serious redesigning...

@wwmm
Copy link
Owner

wwmm commented Jun 20, 2022

I would like to use #1 for an audio player and by default in headphone DAC/output, #2 for a video player and by default in speaker DAC/output and #3 for web browsers on both outputs (with option to switch to it for a video player when dealing with badly mixed local files).

By the way with the available preset autoloading you almost have the first 2 points because the hardware being used is different. But the third one about the browsers will require multiple pipelines indeed.

@Digitalone1
Copy link
Contributor

Per-app profiles are too complex. The only way to do this in the current state is configuring Pipewire manually to do it. I don't think EasyEffects will ever have this feature.

@v-fox
Copy link
Author

v-fox commented Jun 20, 2022

Especially with EE not having knobs that could be promptly turned up/down with a mouse.

I did not understand. The controls accept mouse input. Or are you talking about the way audio plugins usually handle controls with the mouse? I mean that way where you click and drag the mouse to change values. If yes this isn't how gtk and QT widgets usually interact with the mouse. So probably can not be done.

Yes, it allows "grabbing" a round knob and yanking it fast, like one would do to a physical one. A "graded" (with "fast" steps of 1 or 3 dB and ability to fine-tune slowly with wheel or keyboard) slider could be suitable alternative.

By the way with the available preset autoloading you almost have the first 2 points because the hardware being used is different. But the third one about the browsers will require multiple pipelines indeed.

Yeah but even that requires constant switching of devices in "obsolete" PA mixer and then profiles in EE. It would even be easier to always route to both devices and just mute "unwanted" one.

However, that uncovers glitches in PW attempting to run both devices with different period and quantum sizes from same pipeline node. In particular 2 issues:

  1. It likes to consider USB DAC "batch" device, so it halves its period. Which may prompt PW to use different quantum for them and weirdly "mute" (likely, passing silence or unrouting device) one of them while showing them working and unmuted or produce broken screeching sound.
  2. USB device may have low limit on hardware buffer size and PW likes to cap max quantum to 1/4 of that size. But if both are used it may try quantum above 1/4 of hardware buffer, unless manually restricted in config, and lead to same glitches.

Previously PW would always cap quantum, so with "standard" buffer of 8192 you would get 2048 of usable bytes. Which gives mere 10ms of max latency for 192khz mode. Although, it proved workable at 3072 somehow but no more. So if you put 8ms as global min quantum in PW then going above 16ms on device level is dangerous. Luckily, putting such cap in pipewire.conf does not limit app nodes. Unluckily, nothing seems to limit app nodes, hardcoded ones simply stop any output if protocol limits (such as ones in PA or JACK settings) are imposed. But the latter case seems safe, unlike device node limitations.

Some people asked this in the past. I do not remember the issues. But the problem is that it is almost on the same level of creating another application. Our window would have to go through some serious redesigning for per app pipelines to be controlled. And there is also the question about how to handle multiple pipelines and the application redirection.

As almost all apps still use the Pulseaudio API a null sink still is desirable when redirecting them because they can not see PipeWire filters. I can force the link from the PipeWire side but the app device selection won't be able to to that. But if null sinks are going to be used this means having one null sink for each pipeline. It will be messy...

PA apps don't need to see real devices, only first pipeline node. You can just name those pipelines with their profile names instead of one generic "EE Sink/Source". But the problematic part is to route PA apps yet hide it as selectable default device to avoid auto-switching into itself. Though, it's another issue.

However, with such system selecting "default" device should be irrelevant because EE would completely take over routing of entry/pipeline and exit/hardware nodes. Could it be possible to keep a single visible virtual "stub" EE sink/source for apps that are "opted" for processing and then just fool them by routing into appropriate pipeline's entry node (keep visible stub always unrouted but create another link from the app to an actual, "hidden" pipeline entry ?) that is not advertised in PA protocol ? Then pipelines' exit nodes could be separately routed into whatever devices, even along with "free" (untouched) apps.

This way, having EE as default device in PA protocol (and PW in general) would even be preferable because EE then could "protect" apps from being routed wrongly by enforcing its pipeline exit node rules and there be no auto-switching of "default" device because EE's virtual sink/source will never go away.

Another headache is handling the presets... But the worse is that the people that do not need this will probably ask for the current behavior and window workflow to be still available. Doing all of this requires a huge amount of work and a very clever redesigning. That is why I say it is on the same level as creating another application.

But with default null configuration it should not look any more different. I would just put all apps into a default "profile" (pipeline) which would exit at default PW device, unless customized. Of course, there should be no limit at adding "exit" devices, just as there be no limit to adding apps to "entry".

Additionally, "tags" could be used to route apps and devices as groups, such as advertised "role". By default there could be no tags other than PW/PA's roles and user would need to add them but some more generic ones could be guessed and assigned, like "device type" (pci/usb/bluetooth/wifi-direct, wired/wireless) and "app type" (music-manager, media-player, web-browser).

Implementing automatic preset loading based on the selected output/input device already gave me way more headaches than I was expecting. Multiple pipelines on top of this is something I am not sure it is going to be doable without another huge round of work.

Yeah, that would be one of the reasons while such app does not exist. But you, probably, already dealt with the worst of it and the biggest reason for the lack of it, likely, was deficiency of PA and JACK protocols. With PW there should be enough metadata and leeway to control routing. This will make EE mandatory for all desktop Linux installations and with good reason: non-processed audio and bad audio device handling conventions are extremely problematic for all user experience.

Per-app profiles are too complex. The only way to do this in the current state is configuring Pipewire manually to do it. I don't think EasyEffects will ever have this feature.

How so ? And how PW is even involved ? All EE needs to do is create a whole pipeline per an active profile, just like it does now with a single one, then route each apps in a starting node of its selected pipeline and route last node of a pipeline in selected devices.

PW does not seem to be actively interested in it, WP is more problematic than obsoleted PW-MS. But EE is already a pipeline builder with advanced routing. Might as well extend its entry and exit handling. At least it's the one that goes in the right direction so far.

@Digitalone1
Copy link
Contributor

You make it too easy, we don't even have the ability to spawn multiple filters of the same plugin in the current state.

I think you can do what you want with a specific Pipewire configuration (I don't know if it can be done with wireplumber).

@wwmm
Copy link
Owner

wwmm commented Jun 20, 2022

However, with such system selecting "default" device should be irrelevant because EE would completely take over routing of entry/pipeline and exit/hardware nodes. Could it be possible to keep a single visible virtual "stub" EE sink/source for apps that are "opted" for processing and then just fool them by routing into appropriate pipeline's entry node (keep visible stub always unrouted but create another link from the app to an actual, "hidden" pipeline entry ?) that is not advertised in PA protocol ? Then pipelines' exit nodes could be separately routed into whatever devices, even along with "free" (untouched) apps.

The app can be forced to an ordinary node. Its device selector won't be able to do that and apps like Pavucontrol will show that the destination device is unknown. But as far as I remember it works. It is doable. The problem is that the current code will be turned upside down while implementing this...

How so ? And how PW is even involved ? All EE needs to do is create a whole pipeline per an active profile, just like it does now with a single one, then route each apps in a starting node of its selected pipeline and route last node of a pipeline in selected devices.

Just creating a new pipeline is going to be useless. For example the window has to manage these independent pipelines. Otherwise you are not going to be able to set different parameters in each of them. So our graphical interface will have to go through deep changes to allow this. And that is a lot of gtk code to change.

There is no point in having multiple pipelines if our presets code expects only 2. So all the code that handles presets files and the binding of each key in the preset file to the GSettings data base will have to de adapted to this. Again it is a lot of code to change.

The preset menu will have to be redesigned. After all once you click there it is going to apply settings to what if there are multiple pipelines?

The spectrum will probably need some changes too. Is it going to show the combination of all pipelines or something else?

The preset autoloading will also need some changes. With multiple pipelines the profile loaded based on the used device will be applied to what? We can't just leave this part of the code as it is...

Is all of this also going to be done in the microphone pipeline? That is another thing to consider... And if the answer is yes the amount of work doubles...

The blocklist code will probably have to be adapted to this new workflow...

Once work is started on this other things are probably going to appear...

In the end there will be tons of small things scattered through our 30K lines of code that will have to be changed in one way or another to ensure consistency. Nothing in our current code was designed considering an arbitrary number of pipelines. Only one for the mic and another for the output device. What you are proposing is doable. I agree with this. But as it is a workflow that is radically different from how things are managed right now it is in no way simple to do. Specially without forcing this new behavior on people that are totally fine with one pipeline handling everything.

@retnag
Copy link

retnag commented Jan 30, 2024

I came across this issue looking for a solution with a similar use-case. Now I can see this is a major piece of work, and probably won't be addressed, I just wanted to let you know the Author isn't alone...

My ideal setup:
Pipeline1: use a deep-noise-canceller and speech processor on the meeting I am in (all day for work), to filter out keyboard or other background sounds of people that forget to mute themselves or just have a bad mic setup.
Pipleine2: an equalizer and bass enhancer for the music I am listening to during work (fine-tuned for my specific set of headphones).

-> Obviously, using the speech enhancers from pipeline1 with the music apps makes it unlistenable, and the bass-enhancer/equalizer is not really necessary for the meeting audio (but thats something I could live with tbh...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants