Mac: Cannot connect to X server running my singularity container

[zoeye859]

Version of Singularity:

What version of Singularity are you using? Run:

$ singularity version
3.3.0-rc.1.658.g7427b73f1.dirty

(Freshly installed singularity Mac desktop version)

Expected behavior

I build my own singularity container, the usage and command history can be found at (https://github.com/zoeye859/presto_v3_singularity). It worked fine on multiple ubuntu machines, therefore I assume it worked OK.

However, when I try to run this container on my Mac (and my colleague's Mac), the X server cannot be connected, this means that I cannot interact with pop-up images using X11.

> ./pgdemo1
Graphics device/type (? to see list, default /NULL): /xw
PGPLOT /xw: cannot connect to X server [1xx.1xx.xx.1:0.0]

If I change the $DISPLAY path:

> export DISPLAY=localhost:0.0
> ./pgdemo1
Graphics device/type (? to see list, default /NULL): /xw
PGPLOT /xw: cannot connect to X server [localhost:0.0]

If choose /PS, the .ps file can be produced.
When running on ubuntu, the X server can plot images without problem.

After intensive research and many experiments, I believe it is something to do with the X11 forwarding on Mac, I think I'd better to submit an issue here to try some luck.

You may be interested in the following error too:

> singularity shell PSRSOFT_yhy.simg
Singularity PSRSOFT_yhy.simg:~> xhost local:root
xhost:  unable to open display "localhost:0.0"

There was once I got the error saying

X11 connection rejected because of wrong authentication.
xhost:  unable to open display "localhost:10.0"

Steps to reproduce this behavior

> singularity shell PSRSOFT_yhy.simg
Singularity PSRSOFT_yhy.simg:~> xhost local:root
xhost:  unable to open display "localhost:0.0"

Use a Mac machine

You can download PSRSOFT_yhy.simg following the README at (https://github.com/zoeye859/presto_v3_singularity)

What OS/distro are you running

$ cat /etc/os-release

Mac OS CATALINA 10.15.6 (19G73)

How did you install Singularity

My install history can be found at (https://github.com/zoeye859/presto_v3_singularity/blob/master/INSTALLATION%20RECORDS.md)

[zoeye859]

Your help will be greatly appreciated!

Thank you very much!

Best,
Zoe

[zoeye859]

Btw, I have already tried the following on my Mac:

sudo vi /etc/ssh/sshd_config

And change the following two lines:

X11Forwarding yes
X11DisplayOffset 10

Also,

> xhost + 127.0.0.1
127.0.0.1 being added to access control list

[jmstover]

It's not a 127.0.0.1 address. IIRC Mac defaults to 192.168.64.0/24. The networking... isn't fun. Must be done with DHCP because the framework needs to assign a dhcp address to allow communication through.

The "easiest" (though not best) is to do: xhost + on the Mac side. That should open it completely to make sure the X communication is happening right. Once you know it is working right, you can limit it to the specific address space.

[zoeye859]

Thank you very much for your quick response! Super efficient!

It's not a 127.0.0.1 address. IIRC Mac defaults to 192.168.64.0/24. The networking... isn't fun. Must be done with DHCP because the framework needs to assign a dhcp address to allow communication through.

I don't know much about DHCP. So I decide to go forward with your next suggestion:

The "easiest" (though not best) is to do: xhost + on the Mac side. That should open it completely to make sure the X communication is happening right. Once you know it is working right, you can limit it to the specific address space.

This didn't solve my problem though. Here is what I did on my Mac:

> export DISPLAY=:0
> /opt/X11/bin/xhost +
access control disabled, clients can connect from any host
> xhost + 
access control disabled, clients can connect from any host

Then I went to run my singularity container:

> singularity shell PSRSOFT_yhy.simg
Singularity PSRSOFT_yhy.simg:~/FRB_work> xhost local:root 
xhost:  unable to open display "192.168.64.1:0.0"

Any further comments?

Thank you so much!

Best,
Zoe

[dtrudg]

Can you try ssh-ing to some kind of straight Linux machine and running and X term application so that it will display on your Mac, to verify XQuartz is working okay.... or has that already been done?

Please also note the current status of Singularity Desktop for Mac: https://sylabs.io/singularity-desktop-macos/

Please Note - Singularity Desktop for macOS is a Beta release, and has not been updated for compatibility with the new SIF signature format in Singularity 3.6.0.

Sylabs™ is currently engaging with the community to plot a course for the future of the Mac version of Singularity. In the meantime, please be aware that the release has not received recent security and bug fixes, and should be used at your own risk.

[jmstover]

to verify XQuartz is working okay

To expand on this... In the XQuartz preferences, you need to goto Security and then check the box that says: Allow connections from network clients

That's off by default.

[zoeye859]

Thank you very much Jason and David!

The error still applies.

Can you try ssh-ing to some kind of straight Linux machine and running and X term application so that it will display on your Mac, to verify XQuartz is working okay.... or has that already been done?

I ssh to a ubuntu machine and then type

> xclock

I saw a tiny clock window pop up, this means that my XQuartz in my Mac works fine.

To expand on this... In the XQuartz preferences, you need to goto Security and then check the box that says: Allow connections from network clients

That's off by default.

I also check the box that says:Allow connections from network clients

Then I restart XQuartz and tried:

yehaoyang@Yes-MacBook-Pro FRB_work % open -a xquartz                   
yehaoyang@Yes-MacBook-Pro FRB_work % singularity shell PSRSOFT_yhy.simg
Singularity PSRSOFT_yhy.simg:~/FRB_work> xhost local:root
No protocol specified
xhost:  unable to open display "192.168.64.1:0.0"

There is one new error message No protocol specified, which I haven't seen before.

I googled this error message but didn't find a way to fix this problem. Is there anything else I can do?

Please Note - Singularity Desktop for macOS is a Beta release, and has not been updated for compatibility with the new SIF signature format in Singularity 3.6.0.

Sylabs™ is currently engaging with the community to plot a course for the future of the Mac version of Singularity. In the meantime, please be aware that the release has not received recent security and bug fixes, and should be used at your own risk.

I understand that this Mac version of Singularity is still under development. I would love to know whether this X11 forwarding thing is related to the current version, or it is just because my mistakes at some settings.

Thank you very much!

Best,
Zoe

[jmstover]

Run xhost + on your Mac not in the image.

[zoeye859]

Run xhost + on your Mac not in the image.

Ahhh!

It works!

Tried it on another Mac machine, also works! Hooray!

Thank you so much Jason and David!!!!

In summary:

• Download the newest singularity Desktop MacOS (it would not work on another Mac until installing the newest singularity)
• "In the XQuartz preferences, you need to goto Security and then check the box that says: Allow connections from network clients", thanks to Jason
• restart XQuartz

You may need to type the following (or not)

export DISPLAY=:0
/opt/X11/bin/xhost +

Then try to run your singularity container, it should work now.

I will try to ask another colleague to reproduce the results, as I have tried so many different things on these two Macs following Google search results. Just want to check for sure.

Thank you so much!! A huge problem for me solved!

Have a great night! (11:04pm here) Sending my virtual hugssssssss!

[jmstover]

Sounds good. Holler if you have any other issues.

@dctrud ... Right, I can't close these... This can probably be closed now. :)

[harunkurtdev]

Hello everbody,

if you getting same error for docker,

first of all, you must give on the this screen,

open screen bellow like this,

so find you security, give permission like this.

you can try over with Xquartz terminal.

on the xquartz terminal

/opt/X11/bin/xhost + 127.0.0.1 

I used code for

 docker run -it --rm -e DISPLAY=docker.for.mac.host.internal:0 -v /tmp/.X11-unix:/tmp/.X11-unix ubuntu /bin/bash

and for x11-apps code on the terminal

apt-get update
apt-get install x11-apps