GL backend emits "No protocol specified" if XOpenDisplay() fails

[jsquyres]

What version of hwloc are you using?

This was originally reported in open-mpi/ompi#7701: the user complained that mpiruning an app produced an errant No protocol specified message.

Through a very long series of comments, we determined by looking at debugging output from Open MPI that the No protocol... message is emitted right when Open MPI will be invoking functionality in libhwloc.

From there, we had the user run lstopo, and we noticed that it also emits the No protocol specified message.

From that, we concluded that something in libhwloc is likely the source of the No protocol... message.

Since the most recent comments on the Open MPI github issue are about Ubuntu 20.04, I assume we're talking about https://packages.ubuntu.com/focal/hwloc, which is Ubuntu-packaged hwloc 2.1.0.

Which operating system and hardware are you running on?

Similar to above, I assume the users are running on Ubuntu 20.04. This is from my Ubuntu 20.04 VM (which does not show the errant 'No protocol...` message):

Linux ubuntu-2004 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Details of the problem

open-mpi/ompi#7701 (comment) says:

lstopo does indeed emit No protocol specified

CC: @ke7kto @YannChemin @chhu @rhc54

[bgoglin]

@ke7kto @YannChemin @chhu Can you export HWLOC_COMPONENTS_VERBOSE=1 in the environment and post the debug messages from lstopo? I want to try to locate which hwloc component causes the issue.

[philippkraft]

I am the OP of open-mpi/ompi#7701, installed hwloc with apt install hwloc on the machine where the message occured the first time (an AMD EPYC 7452 32-Core Processor (Rome)). I get the following output (including the no protocol problem).

$ export HWLOC_COMPONENTS_VERBOSE=1
$ lstopo

Registered discovery component `no_os' phases 0x2 with priority 40 (statically build)
Registered discovery component `xml' phases 0x1 with priority 30 (statically build)
Registered discovery component `synthetic' phases 0x1 with priority 30 (statically build)
Registered discovery component `linux' phases 0x7a with priority 50 (statically build)
Registered discovery component `x86' phases 0x2 with priority 45 (statically build)
Registered discovery component `gl' phases 0x10 with priority 10 (from plugin /usr/lib/x86_64-linux-gnu/hwloc/hwloc_gl)
Registered discovery component `opencl' phases 0x10 with priority 10 (from plugin /usr/lib/x86_64-linux-gnu/hwloc/hwloc_opencl)
Registered discovery component `pci' phases 0x48 with priority 20 (from plugin /usr/lib/x86_64-linux-gnu/hwloc/hwloc_pci)
Enabling discovery component `linux' with phases 0x7a (among 0x7a)
Enabling discovery component `x86' with phases 0x2 (among 0x2)
Enabling discovery component `no_os' with phases 0x2 (among 0x2)
Excluding discovery component `xml' phases 0x1, conflicts with excludes 0x1
Excluding discovery component `synthetic' phases 0x1, conflicts with excludes 0x1
Enabling discovery component `pci' with phases 0x48 (among 0x48)
Enabling discovery component `gl' with phases 0x10 (among 0x10)
Enabling discovery component `opencl' with phases 0x10 (among 0x10)
Final list of enabled discovery components: linux(0x7a),x86(0x2),no_os(0x2),pci(0x48),gl(0x10),opencl(0x10)
No protocol specified
Disabling discovery component `linux'
Disabling discovery component `x86'
Disabling discovery component `no_os'
Disabling discovery component `pci'
Disabling discovery component `gl'
Disabling discovery component `opencl'

Machine (126GB total)
...

Additional machines without the warning

I have installed hwloc now on additional machines I am not using with OpenMPI, and I do not get the no protocol message:

1. Ubuntu 20.04 in Windows subsystem for Linux 2 on a Intel(R) Core(TM) i9-9900 CPU @ 3.10GHz (8 cores) - that is the system I use as the ssh client for the other systems
2. Ubuntu 20.04 on a Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz (6 cores)
3. Ubuntu 18.04 on a Intel(R) Xeon(R) CPU X5680 @ 3.33GHz (6 cores)

[bgoglin]

@philippkraft Thanks. Assuming it's caused by one of the graphics-related backend, can you set HWLOC_COMPONENTS=-opencl,-gl in the environment to see if the warning goes away when OpenCL and GL backends are disallowed? Do you have GPUs (NVIDIAs?) on all these machines?

[philippkraft]

Computer 0, NVIDIA GPU GeForce GTX 1050 Ti
lstopo --version = 2.1.0

$ export HWLOC_COMPONENTS_VERBOSE=-opencl,-gl
$ lstopo
No protocol specified
...

Computer 1: lspci does not report anything in WSL2, but there is a AMD Radeon RX550
Computer 2: lspci reports a Matrox Electronics Systems Ltd. G200 graphics card (it is a DELL server, couple of years old, inherited)
Computer 3: NVIDIA Geforce GT 1030, but on Ubuntu 18.04, lstopo --version = 1.11.9

[bgoglin]

Oops sorry, I meant HWLOC_COMPONENTS=-opencl,-gl (no _VERBOSE).

[philippkraft]

Now it is gone!

Sorry I don't no why i wrote "verbose", you asked for the correct flag...

I can install hwloc from git on Computer 3, to see if the new version reports "No protocol" on another computer with an NVIDIA card. But only if that might help.

[bgoglin]

I fixed my comment, the _VERBOSE was there initially.
Can you try check that -gl (without -opencl) is enough? I assume it comes from the GL backend using X11 libs to query the locality of NVIDIA GPU displays. In the meantime, I am trying to find a machine with a NVIDIA GPU used for graphics and not only for HPC :)

[philippkraft]

-gl is enough. The GPU is not used for graphics, normally.

[bgoglin]

Thanks. I managed to reproduce this issue after chmod 0 ~/.Xauthority. The warning comes from libX11 when our call to XOpenDisplay() fails because of authentication issues. I don't see any way to avoid it yet. And libX11 is barely maintained so I don't hope any fix anytime soon :/

[ke7kto]

Are there any performance issues to be aware of when using that flag?

[bgoglin]

The only downside is that the hwloc topology won't contain the locality of NVIDIA X11 screens (this locality is only useful for very specific applications).
Disable the backend may slightly reduce the hwloc discovery time but I think it's negligible.