New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Black png returned when using a vnc session #117

Open
durack1 opened this Issue Jan 19, 2017 · 77 comments

Comments

Projects
None yet
6 participants
@durack1
Member

durack1 commented Jan 19, 2017

@danlipsa @doutriaux1 I just ran the demo script BlueMarble_huss_satellite_view.html and received the appropriate graphic on the active VNC screen. What was saved to the png is below, and is certainly not what I was expecting:
bluemarble_huss_satellite_view_daytimeview
This is certainly not the expected behavior. What additional information do you require to better replicate the issue:

In [1]: import cdat_info
In [2]: cdat_info.version()
Out[2]: ['', 'v2.8']

@durack1 durack1 added the bug label Jan 19, 2017

@durack1

This comment has been minimized.

Member

durack1 commented Jan 19, 2017

@danlipsa @doutriaux1 this is even better, when I change the:

canvas = vcs.init(size=2)

to

canvas = vcs.init(size=2,bg=True)

I then get:
bluemarble_huss_satellite_view_daytimeview

@danlipsa

This comment has been minimized.

Contributor

danlipsa commented Jan 20, 2017

@durack1 Do you have the huss_Amon_HadGEM2-AO_historical_r1i1p1_186001-200512.nc file?
@aashish24 @doutriaux1 We should add the examples to the test suite, maybe as a separate section to be able to run them automatically.
I think the black PNG is what we have seen before - VNC does not support OpenGL. Our suggestion was to run VNC through Virtual GL.

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 20, 2017

Dan with recent updates my ubunutu seems to seg fault instantly via x forwarding. RH6 has been sending black png for a while now via x forwarding. @durack1 use the nox version, I know it sucks but that one works.

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@danlipsa this file is 187MB, it's got all the timesteps 186101 to 200512. I've pulled out a single timestep in the linked file (102KB) let me know if this isn't suitable:
huss_Amon_HadGEM2-AO_historical_r1i1p1_186001.nc.zip

How do I run VNC through Virtual GL?

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 just confirming to install the *-nox I need the following conda command right?

conda create -n uvcdat280-nox -c conda-forge -c uvcdat uvcdat vcs-nox

Is that right? How about the vtk-cdat-nox? Or is that magically also grabbed as a dependency of vcs-nox?

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 is it right that both the vcs AND the vcs-nox versions (along with their dependencies) are being installed? Seems like this is going to cause me a lot of problems..?

[duro@ocean 090605_FLR2_sptp]$ conda create -n uvcdat280nox -c conda-forge -c uvcdat uvcdat vcs-nox
Fetching package metadata .............                                                                   
Solving package specifications: .                                                                         

Package plan for installation in environment /export/durack1/anaconda2/envs/uvcdat280nox:

The following NEW packages will be INSTALLED:

... 
    dv3d:                               2.8-py27_2                        uvcdat     
    dv3d-nox:                           2.8-py27_2                        uvcdat     
...
    vcs:                                2.8-py27_2                        uvcdat
    vcs-nox:                            2.8-py27_2                        uvcdat
    vcsaddons:                          2.8-py27_2                        uvcdat
    vcsaddons-nox:                      2.8-py27_2                        uvcdat
    vistrails:                          2.8-2                             uvcdat
    vtk-cdat:                           7.1.0.2.8-py27_2                  uvcdat
    vtk-cdat-nox:                       7.1.0.2.8-py27_2                  uvcdat
...
@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 20, 2017

@durack1 you want: conda create -n uvcdat280nox -c conda-forge -c uvcdat uvcdat-nox

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 perfect, I'll give that a spin

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 unfortunately no luck, I successfully installed uvcdat-nox which only had the appropriate *-nox versions and this is still the output:
bluemarble_huss_satellite_view_daytimeview

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 20, 2017

@danlipsa this might be bad then! @durack1 try with a slightly more recent vcs
conda create -n uvcdat280nox -c conda-forge -c uvcdat/label/nightly -c uvcdat uvcdat-nox

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 ok so installed the nightly uvcdat-nox (after fixing spaces in the command) and now I get:

(uvcdatNightlyNox) duro@ocean:[170118_PaperPlots_input4MIPs]:[2307]> python make_paperplots.py
Traceback (most recent call last):
  File "make_paperplots.py", line 9, in <module>
    import vcs
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/vcs/__init__.py", line 57, in <module>
    from utils import *  # noqa
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/vcs/utils.py", line 6, in <module>
    import boxfill
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/vcs/boxfill.py", line 25, in <module>
    import VCS_validation_functions
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/vcs/VCS_validation_functions.py", line 5, in <module>
    import genutil
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/genutil/__init__.py", line 5, in <module>
    from grower import grower
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/genutil/grower.py", line 4, in <module>
    import cdms2 as cdms
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/cdms2/__init__.py", line 33, in <module>
    from dataset import createDataset, openDataset, useNetcdf3, \
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/cdms2/dataset.py", line 6, in <module>
    import Cdunif
ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/cdms2/Cdunif.so)
@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 20, 2017

ok... so let's try this one on the bleeding edge:
conda create -n uvcdat280nox -c conda-forge -c uvcdat/label/patterns -c uvcdat uvcdat-nox

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 the only patterns sourced package is vtk-cdat-nox should I proceed?

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 20, 2017

yes, that's all you need.

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 no good, basically same output as above

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 20, 2017

@danlipsa that's a problem. @durack1 ok then you need to set your LD_LIBRARY_PATH unfortunately

export LD_LIBRARY_PATH=/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/

let me know.

@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 doesn't help, same output:

(uvcdatPatternsNox) duro@ocean:[170118_PaperPlots_input4MIPs]:[2308]> python make_paperplots.py
(uvcdatPatternsNox) duro@ocean:[170118_PaperPlots_input4MIPs]:[2308]> export LD_LIBRARY_PATH=/export/duro/anaconda2/envs/uvcdatPatternsNox/lib/
(uvcdatPatternsNox) duro@ocean:[170118_PaperPlots_input4MIPs]:[2309]> python make_paperplots.py
@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 20, 2017

hum... can you post:

ls /export/duro/anaconda2/envs/uvcdatPatternsNox/lib
ls /export/duro/anaconda2/lib
@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 20, 2017

works for me on oceanonly

(PATT)doutriaux1@oceanonly:[~]:[1709]> python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Dec  6 2015, 18:08:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> x.plot(s)
<vcs.displayplot.Dp object at 0x7f99c80b8050>
>>> 
(PATT)doutriaux1@oceanonly:[~]:[1709]> which python
/usr/local/anaconda2/envs/PATT/bin/python
(PATT)doutriaux1@oceanonly:[~]:[1707]> conda create -n PATT -c conda-forge -c uvcdat/label/patterns -c uvcdat uvcdat-nox
@durack1

This comment has been minimized.

Member

durack1 commented Jan 20, 2017

@doutriaux1 it doesn't work, you don't get
bluemarble_huss_satellite_view_daytimeview_demo
Rather you get
bluemarble_huss_satellite_view_daytimeview
If you're lucky

@durack1

This comment has been minimized.

Member

durack1 commented Jan 21, 2017

@doutriaux1 ok, so if I run uvcdatPatternsNox with bg=False then I get:
bluemarble_huss_satellite_view_daytimeview

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 21, 2017

ok that makes sense (sort of) @danlipsa we should look into why bg=True freaks out on mesa builds. But we could also always turn bg to True on mesa builds

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 21, 2017

thanks @durack1

@durack1

This comment has been minimized.

Member

durack1 commented Jan 21, 2017

@doutriaux1 I can also confirm a similar (but not identically sized output file) png is generated with uvcdat280nox and bg=False

And why not, here 'tis:
bluemarble_huss_satellite_view_daytimeview

@durack1

This comment has been minimized.

Member

durack1 commented Jan 21, 2017

@danlipsa how do I go about running VNC through Virtual GL? While I can get graphics to write out to a file with these *-nox builds, I can't see anything interactively and that's a pretty horrible user experience

@danlipsa

This comment has been minimized.

Contributor

danlipsa commented Jan 23, 2017

@durack1 Here are the instructions I followed to setup vnc/virtual GL on my ubuntu and connect from my mac. I used the latest version for vnc/Virtual GL rather the version specified in the instructions. I think you'll need root access on the linux machine.

I mostly followed instructions from
http://www.paraview.org/ParaView3/Doc/Nightly/www/js-doc/index.html#!/guide/virtualgl_turbovnc_howto

with help from:
https://cdn.rawgit.com/VirtualGL/virtualgl/2.5.1/doc/index.html

Make sure you install the 64 bit versions for virtualgl and turbovnc. Otherwise they won't work! I know because I did this first! :-(

When I start the vnc server I run
/opt/TurboVNC/bin/vncserver -geometry 1440x900 -xstartup ~/.vnc/xstartup

You can do this from ssh and pass the geometry for your own client.

Where xtartup contains:
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
startxfce4 &
#gnome-session&

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources

On the mac client I run:
open vnc://galati:5901

When running an OpenGL application instead of:

# ./app

type

# vglrun app
@danlipsa

This comment has been minimized.

Contributor

danlipsa commented Jan 23, 2017

@doutriaux1 Do you install the gcc 4.8.4 package when installing cdat? This should take care of the missing symbol and will remove the need for LD_LIBRARY_PATH.

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jan 23, 2017

@danlipsa I think we're good now, @durack1 was able to run the nox version so I think we do not need the LD_LIBRARY_PATH thing.

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 14, 2017

@sankhesh I think that would be awesome.

@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 16, 2017

I'm going to try this with mesa tomorrow to see if that works too.

I was able to setup VirtualGL and TurboVNC on a vagrant Ubuntu 16.04 box with mesa following the instructions on the wiki page.

@doutriaux1 Were you able to set it up?

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

@sankhesh trying on crunchy now

the follwing does not exists on RH6:

Run the virtualGL server config script from text console (or remote via

ssh)

sudo /opt/VirtualGL/bin/vglserver_config
@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

I get this:

doutriaux1@crunchy:[~]:[15975]> sudo /opt/VirtualGL/bin/vglserver_config
sudo: /opt/VirtualGL/bin/vglserver_config: command not found
doutriaux1@crunchy:[~]:[15976]> locate vglserver_config
/usr/bin/vglserver_config
doutriaux1@crunchy:[~]:[15977]> sudo /usr/bin/vglserver_config
sudo: /usr/bin/vglserver_config: command not found
@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 19, 2017

@doutriaux1 It could be a permissions issue when trying to install VirtualGL. Do you have full permissions in /opt/ ? Ref: https://sourceforge.net/p/virtualgl/discussion/401860/thread/6f2e2505/

I am not sure why you have /usr/bin/vglserver_config. Could it be from a previous installation? The default install location for VirtualGL is in /opt. See https://cdn.rawgit.com/VirtualGL/virtualgl/2.5.2/doc/index.html#hd005001

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

@sankhesh I sudoed it. Anyhow I'm trying now now with vgl 2.4 which my yum was able to find and install. More soon

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

@sankhesh I get the server up and running
theres is no glxgears in the bin directory
but the turboserver starts but runnnig vglrun leads to this:

doutriaux1@crunchy:[~]:[15502]> export DISPLAY=crunchy.llnl.gov:3
doutriaux1@crunchy:[~]:[15503]> vglrun python
[VGL] NOTICE: Automatically setting VGL_CLIENT environment variable to
[VGL]    128.15.245.181, the IP address of your SSH client.
Python 2.7.13 | packaged by conda-forge | (default, May  2 2017, 12:48:11) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
CDMS system error: No such file or directory
CDMS I/O error: Opening file /usr/local/anaconda2/envs/2.10/share/uvcdat/sample_data/clt.nc
Traceback (most recent call last):
  File "/export/doutriaux1/.pythonrc", line 32, in <module>
    f=cdms2.open(fnm)
  File "/usr/local/anaconda2/envs/2.10/lib/python2.7/site-packages/cdms2/dataset.py", line 357, in openDataset
    return CdmsFile(path, mode, mpiBarrier=CdMpi)
  File "/usr/local/anaconda2/envs/2.10/lib/python2.7/site-packages/cdms2/dataset.py", line 1090, in __init__
    raise CDMSError('Cannot open file %s (%s)' % (path, err))
cdms2.error.CDMSError: Cannot open file /usr/local/anaconda2/envs/2.10/share/uvcdat/sample_data/clt.nc (No error)
>>> import vcs
>>> x=vcs.init()
>>> x.plot([1,2,3,4,3,2,1,2,3,4])
No protocol specified
[VGL] ERROR: Could not open display :0.
@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

Also the follwing hangs and never renders anything

doutriaux1@crunchy:[~]:[15503]> vglrun xeyes
[VGL] NOTICE: Automatically setting VGL_CLIENT environment variable to
[VGL]    128.15.245.181, the IP address of your SSH client.

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

@sankhesh do I need to connect to crunchy via ssh or via another vnc client the instructions are not clear about this.

@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 19, 2017

@sankhesh do I need to connect to crunchy via ssh or via another vnc client the instructions are not clear about this.

I don't think it matters how you're connected to the server. As long as you have a VNC server session running, you should have a valid X environment.

@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 19, 2017

Also the follwing hangs and never renders anything

doutriaux1@crunchy:[~]:[15503]> vglrun xeyes
[VGL] NOTICE: Automatically setting VGL_CLIENT environment variable to
[VGL] 128.15.245.181, the IP address of your SSH client.

I think it might be rendering something in the VNC display. Try connecting via VNC to see if you see the xeyes window.

@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 19, 2017

I'm not sure why the DISPLAY environment variable is not supplied to python. Perhaps try setting it within python using os.environ

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

@sankhesh it's a total failure on crunchy. Please try.
Instructions might need update. Here's what I did.

  • started vglserver on crunchy
  • download turnbvnc client on mac
  • connect to crunchy from mac via trubovnc
  • if I run vglrun python I get the bad display :0 error
  • If I run the regular python it seg fault on opening of display...

Please try from crunchy

@danlipsa can you please try on crunchy as well?

Thanks.

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

@sankhesh whilerunning vglrun python the os.environ["DISPLAY"] return correctly crunchy:3.
But I do get bad display error...

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 19, 2017

Also tried via regular ssh to crunchy

@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 19, 2017

@doutriaux1 I think your setup should work. Try connecting via a VNC client and run xeyes to see if a window shows up.

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 20, 2017

@sankhesh both
vglrun xeyes
and
xeyes

work fine in my turbovnc client

but

doutriaux1@crunchy:[~]:[15936]> vglrun glxgears
[VGL] NOTICE: Automatically setting VGL_CLIENT environment variable to
[VGL]    128.15.245.181, the IP address of your SSH client.
No protocol specified
[VGL] ERROR: Could not open display :0.
doutriaux1@crunchy:[~]:[15937]> glxgears 
Error: couldn't get an RGB, Double-buffered visual
@danlipsa

This comment has been minimized.

Contributor

danlipsa commented Jun 20, 2017

@doutriaux1 I get the same message if I run vglrun from an ssh connected to the server. You have to run vglrun only from turbovnc connected to your server.

@danlipsa

This comment has been minimized.

Contributor

danlipsa commented Jun 20, 2017

@doutriaux1 This is my message:

~]$ vglrun glxgears
[VGL] NOTICE: Automatically setting the DISPLAY environment variable to
[VGL]    10.9.0.8:0.0, based on the IP address of your SSH client.
Error: couldn't open display 10.9.0.8:0.0
[~]$ 
@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 20, 2017

@sankhesh both
vglrun xeyes
and
xeyes

work fine in my turbovnc client

Awesome 👍 👏
It means you have the setup working.

but

[VGL] ERROR: Could not open display :0.

Could you please share the output of the following two commands (when connected via TurboVNC client):

/opt/VirtualGL/bin/glxinfo -display :3.0

and

vglrun -sp /opt/VirtualGL/bin/glxspheres
@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 20, 2017

@danlipsa I am getting this while connected via turbovnc

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 21, 2017

@sankhesh do you have an account on crunchy? I would like for you to try it. Because It's definitely not working for me from my mac.

@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 21, 2017

@doutriaux1 I don't. How do I get one?

@durack1

This comment has been minimized.

Member

durack1 commented Jun 21, 2017

@hoang1 can you help out here?

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 21, 2017

@durack1 @sankhesh is taken care of. No worries.

@sankhesh

This comment has been minimized.

Contributor

sankhesh commented Jun 22, 2017

@doutriaux1 Could you please try the setup on a local desktop (Ubuntu or Red Hat)?

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 22, 2017

@sankhesh both my linux boxes are dead right now... I'll see if I can try the setup from a vm/docker

@doutriaux1

This comment has been minimized.

Member

doutriaux1 commented Jun 22, 2017

@sankhesh I'm going to look into this as well: https://github.com/yantis/docker-virtualgl

@doutriaux1 doutriaux1 modified the milestones: 3.0, post 3.0 Mar 29, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment