Nouveau support can be dropped (Prime offloading is the better thing)

[karolherbst]

If you want to use your nvidia card you should use Prime offloading as described there: https://nouveau.freedesktop.org/wiki/Optimus/

And if a user has a too old Kernel for that, then he will run into different issues and also can't get a faster PCIe link than 2.5, which really has a big performance hit with bumblebee so the performance is bad either way (not so bad with PRIME offloading).

There is really no need to still maintain the nouveau bumblebee code afaik.

[ArchangeGabriel]

How do you switch fastly from nouveau to nvidia and back for testing purposes? I can do that with Bumblebee, but not Bumblebee + Prime AFAIK. Because if PRIME is on, Bumblebee can’t work, right?

[Lekensteyn]

Ack, nouveau is a good replacement for Bumblebee + bbswitch since it provides both PRIME support and runtime PM (if it works). I see on #nouveau that you get quite some support requests mentioning nouveau and Bumblebee, either we keep support (but print a warning once) or remove it.

Using PRIME requires a restart of X (I don't think you can "hotplug" a driver in Xorg, can you?), so either you use Bumblebee and get the blob for acceleration + bbswitch for power saving or do not use Bumblebee and use nouveau for acceleration + bbswitch + external monitor support.

Is this assessment correct?

[karolherbst]

@ArchangeGabriel no problem. I have bumblebeed running all the time and just load nouveau when I need to. Actually you will need DRI3, so you won't need the nouveau DDX, otherwise yeah, it is messy.

But I can unload/load nouveau at any time without restarting X and also run something with bumblbee (after I unloaded nouveau and turned it off via bbswitch)

[karolherbst]

@Lekensteyn not with DRI3, but you need a hacky xorg.conf:

Section "ServerFlags"
        Option      "AutoAddGPU" "false"
EndSection

Section "Device"
    Identifier "IntelCard"
    Driver "intel"
    Option "DRI" "3"
EndSection

Section "Device"
        Identifier "nvidia"
        Driver "dummy"
        BusID "PCI:01:00:0"
EndSection

[karolherbst]

well with Dri3 you also don't get vdpau offloading, but well...

[Lekensteyn]

@karolherbst Hmm, maybe I should have said hotplugging a device. As far as I know, if the graphics card is off during the start of X (normal case with Bumblebee), then X won't pick it up and interfere. Assume that I use Bumblebee and then decide to try nouveau + PRIME, then somehow Xorg must somehow pick up this device after loading.

[karolherbst]

@Lekensteyn with DRI3 it doesn't need to be picked up by X. X simply doesn't matter.

[ArchangeGabriel]

@karolherbst Not sure to understand the whole thing. Without the Xorg conf file you gave above, when modprobing nouveau (after echoing ON to bbswitch), I still get no other providers than the Intel one. I suppose that in order for this to work, I need the above Xorg conf snippet. Can test it right now, but does it work with the card being off at startup? Are there any drawbacks?

[karolherbst]

@ArchangeGabriel with DRI3 prime offloading you don't need to setup the offloader thing anymore (and it won't get listed in --listproviders anyway). It just works out of the box, because a kernel drm API is being used in this case.

The xorg.conf file is just there (especially the dummy part), so that Xorg won't grab the nvidia gpu and makes unloading nouveau impossible (because the nouveau DDX would hold a reference to the driver)

and the DRI 3 part is there to actually enable DRI3 for the intel DDX.

[karolherbst]

well there are still some DRI2 only things like reverse prime (using display port on nvidia GPU) and VDPAU offloading, but that's something bumblebee doesn't support anyway, so you are better of Prime here.

[ArchangeGabriel]

Awesome! So OK, my previous session must have been bogus for some reason because DRI_PRIME=1 glxinfo was still returning Intel even after echoing ON to bbswitch and modprobing nouveau, but now it works fine and I can even run UT4 without crashing on startup. Obviously, there is still the great perf issue from reclocking, which amounts to better perfs on my Intel HD4600 than Nvidia Geforce GT750M, but with all your great work in that area, it should be working in a near future. ;)

So, here is my proposal: add a new backend in Bumbleblee, named prime (optirun -b prime), that only works with nouveau under DRI3 Intel (must add checks for this), and basically just turn ON the card, modprobe nouveau, set DRI_PRIME=1 and fire. On exit, unload nouveau, turn OFF the card. So, that’s quite like none, shouldn’t be hard to implement (biggest part is checks I think). What do you think about it?

And then, advertises people wanting to use nouveau to use that.

The main difference versus raw PRIME is PM: bbswitch vs nouveau. Also, since the card is OFF at startup, you don’t need any special xorg conf file (except DRI3, which could become the default one day).

[ArchangeGabriel]

In fact, closer to --no-xorg than -b none.

[karolherbst]

if you want to add a prime backend, then also pass runpm=0 to nouveau. Usually nouveau should take care of turning off/on the gpu itself

[ArchangeGabriel]

Sure, in fact we should ever do that when using bbswitch.

[karolherbst]

well do be honest, I don't think it is a good idea to add support for that in bumblebee, it would be better to just concentrate on nvidia

[ArchangeGabriel]

My idea was to allow people to try PRIME very easily if they are currently on Bumblebee, or to be able to switch from one to the other rapidly. Without that, it would require to:

1. sudo tee /proc/acpi/bbswitch <<< ON
2. sudo modprobe nouveau (with the eventual runpm=0 arg, not sure how that works on cmdline)
3. DRI_PRIME=1 someapp
4. sudo modprobe -r nouveau
5. sudo tee /proc/acpi/bbswitch <<< OFF

While all this could be a simple optirun -b prime someapp. Even dropping nouveau support in primus mode right now seems bad, because it could help people understand the perfs limitation.

[karolherbst]

yeah well, but the above can be done with a simple script as well

[ArchangeGabriel]

Sure. The advantage here is you have less risks that someone try to use both at the same time since Bumblebee would complain about it (or should at least). Else, what about packaging a prime script that:

1. Check whether bbswitch is in use and then power the card ON and modprobe nouveau if that one isn’t loaded yet.
2. Run the app.
3. rm nouveau if it wasn’t loaded initially and echo OFF to bbswitch if that was the initial state.

That seems a bit far from a “simple” script, so would prefer Bumblebee integration personally.

[Lekensteyn]

@ArchangeGabriel nouveau can already power off using runtime PM, it does not need bbswitch for that purpose. Runtime PM is used to determine whether there are any users, and otherwise removes power automatically.

Bumblebee was created because it was not possible to render and display at once, so as a hack a secondary X server was started and something (VirtualGL, primus) is used to copy stuff over. Now that nouveau can support this scenario using PRIME, I think that support for this in Bumblebee is not too important. Most of the time, if detection of the blob fails, nouveau gets loaded and people start digging (debugging) in the wrong direction.

There are parallels between runtime PM and Bumblebee:

• Bumblebee tracks the number of users (optirun X, optirun -b none X, etc.) while for runtime PM, the kernel tracks the usage count (lspci trying to read the PCI config space, Xorg opening /dev/dri/..., etc.)
• Bumblebee allows you to disable PM by setting PMMethod=none. You can also disable runtime PM by writing on to /sys/bus/pci/devices/0000:01:00.0/power/control (where 01:00.0 is the PCI ID of the Nvidia device).
• If there were previously no users and a new one arrives, bumblebeed will power on the GPU via bbswitch. Runtime PM will call the appropriate runtime_resume callbacks of a driver.
• If the last user is gone (optirun exits), bumblebeed will power off the GPU via bbswitch. Likewise, runtime PM invokes the appropriate runtime_suspend callbacks to do the same.

The advantage of runtime PM over managing the state in bumblebeed is that enables other software to use the GPU without having to communicate with bumblebeed. For this reason it is maybe better to explicitly document that Bumblebee should be disabled/removed when using PRIME (if PRIME works properly, then Bumblebee should not be needed). The env var DRI_PRIME does not have to be set if the drirc file is configured (see https://bugs.freedesktop.org/show_bug.cgi?id=75913). So a new optirun -b prime mode is not really needed.

I also have to admit that the current version of bbswitch is also a bit of a hack. It does not really take ownership of a device, but changes the power state anyway on certain events (/proc/acpi/bbswitch commands, suspend/resume notifications). The latter notifications were necessary because the kernel expected the device to be in D0 state during suspend (and then saves/restores the PCI config space on suspend/resume). bbswitch therefore puts the device back in D0 before suspending. With newer devices this wastes power and apparently increases heat.

We also told users not to load a driver (nouveau, nvidia) while the card is OFF because bad things happen, that can be avoided by having a PCI driver that actually takes ownership of the PCI device. There is an acpi-pr3 branch of bbswitch that does this (and also solves the above s/r notifier problem), but that basically becomes a driver like nouveau except that it does not have other features of a graphics driver.

The future is nouveau + runtime PM. I don't know the current state of the blob, but as long as that does not support runtime PM, then we have Bumblebee + bbswitch + blob for achieving the same.

[gsgatlin]

Hello. So when I tested a while back nouveau did not work on my test system. My DIS: card was always on unless I added kernel arguments such as "video=VGA-2:d" or unless I installed bumblebee. But now that bug seems to have been fixed in fedora 24 so I guess that is one less argument for keeping nouveau support in bumblebee.

So like the fedora distribution is super strict about what kinds of software are available in the distro.

If you guys went ahead with plans to add a "-prime" option then perhaps bumbleee would be suitable for inclusion into fedora. Ben Rosser has actually agreed to review the package last night so that I can make further changes to it so it meets the quality standards of the distro. So if that was the future then I can keep trying.

But if bumblebee becomes a "closed source driver only" thing than I think it is probably not proper nor possible to have bumblebee in fedora (https://fedoraproject.org/wiki/Forbidden_items https://fedoraproject.org/wiki/Forbidden_items?rd=ForbiddenItems#NVIDIA http://fedoraproject.org/wiki/Packaging/Guidelines#No_External_Kernel_Modules ) and I can keep hosting the packages at my third party repository (for as long as I work at NCSU) and clean up the bumblebee package so that it will not try to work with nouveau. And that way Ben will not waste time reviewing a package that is not suitable for inclusion into fedora.

So it seems like maybe bumblebee 4.0 will drop the idea of nouveau support? Maybe I should edit the docs on the fedora wiki to remove mention of nouveau + bumblebee? In my tests, the only distro that bumblebee+nouveau worked faster then PRIME on was RHEL 7 / CentOS 7. On all the fedoras, PRIME was faster. maybe RHEL 7 was slower because it has an older kernel. (3.10)

Anyways, I'm sorry to interject with fedora's rather strict policies and rules and such. Just thought it may help with the discussion. I think maybe some users are installing bumblebee on fedora to use with nouveau driver when really they will have better luck with DRI_PRIME instead.

[gsgatlin]

Oh and to follow up, I think either adding a "-prime" option or dropping nouveau support completely would help. I guess I don't have a personal opinion on it though.

[karolherbst]

@gsgatlin but adding bumblebee just for nouveau without upstream support for nvidia is also kind of silly.

Anyway, PRIME ist faster, because it has smaller bandwith requiernments and doesn't utilize the PCIe bus as much as bumblebee does. Also with reverse PRIME you can even use your displays on the nvidia GPU and have seamingless integration in your desktop. Something bumblebee won't be able to do at all (even a -prime flag won't help here)

[gsgatlin]

@karolherbst well, I originally envisioned that the parts that were "legal" (VirtualGL, bumblebee) could go in the distro and forbidden items could go in some other external third party repo. But if bumblebee should go in a third party repo due to only supporting nvidia I can just close the red hat bugzilla ticket as "won't fix" or something like that.

[Lekensteyn]

@gsgatlin I expect that most users will use the blob with Bumblebee. Nouveau works better without Bumblebee so I would recommend not to persue inclusion of Bumblebee into Fedora and instead focus on getting nouveau to work better.

Todos for this issue in Bumblebee:

• docs: note that nouveau is not a supported driver anymore.
• docs: explain how to use nouveau in various scenarios (acceleration (PRIME(/DRI3?)), external monitor (reverse PRIME)) and note that runtime PM replaces bbswitch.
• code: remove auto-detection of nouveau, remove xorg config snippet.

If really desired, people can still set the driver to nouveau, but this should not be supported.

@ArchangeGabriel @Thulinma What is your opinion on this? Any user scenarios that are overlooked here?

[Thulinma]

Absolutely, nouveau + bumblebee just makes no sense anymore. The official support through PRIME is pretty much complete (I couldn't find any issues, anyway). Drop it like it's hot.

[bluca]

Maybe the only use case where it makes sense is on LTS or legacy distribution releases, where the kernel/xorg/mesa support for PRIME is not available yet. I'm actually not sure when all the necessary blocks were introduced. Is Precise (Ubuntu 12.04) too old? But for that we can simply keep the old version in the PPA IMHO.

[gsgatlin]

@Lekensteyn Ok. Thanks for the clarification. This should allow me to simplify some stuff inside the rpm package and have one yum repo instead of two. I'll just use the bumblebee-nonfree one for all the various packages.

I will update the wiki to remove all the "nouveau + bumblebee" documentation. And I'll try to point people in that fedora wiki at https://nouveau.freedesktop.org/wiki/Optimus/ to try to help them learn2nouveau on optimus.

[karolherbst]

@bluca DRI3 offloading was added in 3.17 through the rendered nodes. I think the normal offloading was some releases before that, don't know which one though

[ArchangeGabriel]

OK, so I say we drop then. We just need to tell people how to switch easily from one to another. I’ll try to do some cleaning in the code in a coming PR tomorrow.

[Lekensteyn]

@bluca Only auto-detection and a xorg.conf snippet will be removed code-wise, so if LTS users somehow feel the need for using nouveau without PRIME, they can still do so by setting Driver=nouveau in bumblebee.conf and installing the old xorg.conf.nouveau file.

12.04 is a bit old, the latest blob update happened on 13 Nov 2015
(see changelog http://packages.ubuntu.com/source/precise/nvidia-graphics-drivers-340-updates), but the release is still supported until April 2017. I guess that we can keep the older 3.0 version and update later on request of users.

[bluca]

That's ok, I don't mind maintaining 3.2 in the ppa for one more year. 12.04 has to stay behind anyway as there's no libkmod. I have to do it for Debian Jessie anyway, and besides I already backported the bug fixes.

[gsgatlin]

Also, nouveau is no longer supported. I'm sorry. nouveau already has power saving. You should just use PRIME with that. I have not had a chance to update the docs. see:

#773

So sorry about that.

[gsgatlin]

sorry. ignore the noise.

[BernardoGO]

Is it normal to have less performance using prime+nouveau than using the Intel 5600?
I did exactly as @ArchangeGabriel proposed using DRI3 and it worked well but it has lower FPS than intel.

I have a GTX970m on P650SE.

[karolherbst]

@BernardoGO currently nouveau doesn't change the GPU clocks. Also things like glxgears are PCIe bottlenecked, so those won't be faster than intel ever even with full reclock support.

Even worse, your 970m is a GM204 GPU, which means we can't change the clocks without risking damaging the GPU, because without the signed PMU firmware from nvidia we can't

1. change the fan (not important for you)
2. know if the hardware will shutdown on high temperatures

So for a rather long time you will be better of with the Intel GPU, because there is nothing we can do about it and this isn't like the normal reasons "we have no time", but "nvidia screws us over"

[pmatos]

@gsgatlin you say nouveau has power saving but not supported for NV130 (Pascal) (i.e. NVIDIA geforce 1050), therefore bumblebee is still relevant. Or have I misunderstood?

[pmatos]

@gsgatlin I am deeply confused reading this thread. You say nouveau has power saving but the nouveau feature matrix says differently:
https://nouveau.freedesktop.org/wiki/FeatureMatrix/
There's no DONE stamp for any arch in nouveau.

[gsgatlin]

@pmatos bumblebee is still relevant if you wish to use closed source driver. Its just that the developers would prefer that bumblebee not be used with nouveau any longer since they have PRIME now. If your card is not supported yet by nouveau then maybe you would need to use nvidia if you don't want your battery to die quickly.

[pmatos]

@gsgatlin My card is a Geforce 1050 Mobile, supported by Nouveau. I suppose my path is therefore nouveau + prime for gpu offloading. According to nouveau devs there's support in nouveau for automatically suspending the nvidia card when not in use so that should be enough for me.

[unoexperto]

Guys, has anyone made offloading work with egpu ? I just got AKiTiO Node and while I was able to use it with Nouveau as with Nvidia drivers using nvidia-prime I couldn't make offloading work neither with Bumblebee nor with Nouveau.

With Nouveau I simply don't see Nvidia card in output of xrandr --listproviders but I see it in output of lspci | grep VGA.

Any thoughts on what to do?

[Lekensteyn]

@unoexperto consider asking in #nouveau on Freenode IRC about it, your problem is unrelated to this issue.

[Zeioth]

PRIME supports nouveu, but not nvidia privative drivers. If you need nvidia privative, bumblebee is still the best thing. Nvidia is working on it.

EDIT: I just discovered bumblebee doesn't support vukan yet. So while this feature is implemented, nouveau+prime is probably the best option for Nvidia users.

[PierfrancescoArdino]

@Zeioth actually you can use Bumblebee + primus_vk to be able to use vulkan with bumblebee