New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dashboard doesnt work after 3.8-12.02 upgrade #1215

Closed
barnhill opened this Issue Mar 16, 2016 · 7 comments

Comments

Projects
None yet
3 participants
@barnhill

barnhill commented Mar 16, 2016

after the 3.8-12.02 all updates on the dashboard do not work

Screen shot of my dashboard after update. Before the update all the content worked just fine.
capture

@phillxnet

This comment has been minimized.

Member

phillxnet commented Mar 16, 2016

@barnhill Thanks for opening an issue on this, I did see this myself late last night but didn't manage to root out why. There is some info given in the browser console window via developer tools which points towards sysinfo changes so it looks like the kernel upgrade in our first testing channel update has changed how this information is made available. As far as I can tell it's purely aesthetic and involves:

// Connect globally to sysinfo as the information is in the breadcrumb
RockStorSocket.sysinfo = io.connect('/sysinfo', {'secure': true, 'force new connection': true});

RockStorSocket.diskWidget = io.connect('/disk-widget', {'secure': true, 'force new connection': true});

RockStorSocket.networkWidget = io.connect('/network-widget', {'secure': true, 'force new connection': true});

RockStorSocket.cpuWidget = io.connect('/cpu-widget', {'secure': true, 'force new connection': true});

RockStorSocket.memoryWidget = io.connect('/memory-widget', {'secure': true, 'force new connection': true});

So Disk Activity, Network, CPU and Memory widgets affectd, presumably all rooted in sysinfo requests.

I was at the time working on a js element and assumed I'd broken things but looks like it was the kernel upgrade or another along with it but all the others look harmless enough, ie yum.log shows only:

Installed: kernel-ml-4.4.5-1.el7.elrepo.x86_64
Updated: rockstor-3.8-12.02.x86_64
Updated: kernel-ml-headers-4.4.5-1.el7.elrepo.x86_64
Updated: nss-util-3.19.1-9.el7_2.x86_64
Updated: libssh2-1.4.3-10.el7_2.1.x86_64

Don't see anything in Rockstor updates but there may have been a js library change I didn't catch. And don't see any changes in rockstor-jslibs.

dead-widgets

Also just seen more info / exposition on this in the dev log forum thread:-
https://forum.rockstor.com/t/dev-log-for-upcoming-3-8-13/1231/4?u=phillxnet
Linking here to help tie info together.

@phillxnet

This comment has been minimized.

Member

phillxnet commented Mar 16, 2016

By way of possible clue I also had the following in my upgrade email notification.
Noting here quickly as have to dash:

systemctl daemon-reload done
  Updating   : kernel-ml-headers-4.4.5-1.el7.elrepo.x86_64                 3/10
  Updating   : nss-util-3.19.1-9.el7_2.x86_64                              4/10
  Updating   : libssh2-1.4.3-10.el7_2.1.x86_64                             5/10
  Cleanup    : rockstor-3.8-11.20.x86_64                                   6/10
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/wsgi.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/win32util.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/util.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/timeout.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/threadpool.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/threading.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/thread.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/subprocess.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/ssl.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/socket.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/server.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/select.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/resolver_thread.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/resolver_ares.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/queue.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/pywsgi.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/pool.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/os.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/monkey.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/lock.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/local.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/hub.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/greenlet.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/fileobject.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/event.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/coros.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/core.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/baseserver.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/backdoor.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/ares.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/_util.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/_threading.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/_sslgte279.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/_ssl2.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/_semaphore.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/gevent-1.0.2-py2.7-linux-x86_64.egg/gevent/__init__.pyc: remove failed: No such file or directory
  Cleanup    : kernel-ml-4.1.0-1.el7.elrepo.x86_64                         7/10
warning: file /lib/modules/4.1.0-1.el7.elrepo.x86_64/modules.softdep: remove failed: No such file or directory
warning: file /lib/modules/4.1.0-1.el7.elrepo.x86_64/modules.devname: remove failed: No such file or directory
  Cleanup    : kernel-ml-headers-4.4.3-1.el7.elrepo.x86_64                 8/10
  Cleanup    : nss-util-3.19.1-4.el7_1.x86_64                              9/10
  Cleanup    : libssh2-1.4.3-10.el7.x86_64                                10/10

This could be specific to this machine though as I have done dev there.

@phillxnet

This comment has been minimized.

Member

phillxnet commented Mar 16, 2016

For me a reboot into 4.3.3 doesn't help things so I guess we are back to js library issues maybe.

@phillxnet

This comment has been minimized.

Member

phillxnet commented Mar 16, 2016

OK looks like we have a gevent library issue here.
3.8-12 had:-

gevent-1.0.2-py2.7-linux-x86_64.egg
3.8-12.02 has:-
gevent-1.1.0-py2.7-linux-x86_64.egg

gevent socketio and gevent_websocket are unchanged at:
gevent_socketio-0.3.6-py2.7.egg
gevent_websocket-0.9.5-py2.7.egg

@phillxnet

This comment has been minimized.

Member

phillxnet commented Mar 16, 2016

Maybe recently closed issue 771 over at https://github.com/gevent/gevent is relevant as I suspect the dashboard does make many fast calls.

I have partially confirmed gevent version as related by replacing the 1.1.0 version with a tar zipped version from 3.8-12 (version 1.0.2). The dashboard then returns to prior function after a reboot.
There is however then an Error: Invalid value for attribute transform="null" which I'm not sure existed or not before (in dev tools console.)

The indicated issue 771 upstream at gevent has been sorted but I'm unsure how we get these fixes right now as a little unfamiliar with eggs.

@schakrava

This comment has been minimized.

Member

schakrava commented Mar 19, 2016

Thanks @phillxnet , you saved me time with your root cause analysis which is spot on. I've tested to confirm that you are indeed correct.

Here's the long story. gevent-socketio has become maintainer-less for some time now and the tech debt is increasing steadily. I think the right solution here is to use Tornado with it's builtin websocket support. That will remove our dependency on gevent* entirely, though using vanilla gevent without gevent-socketio should be manageable. We do have the CentOS python .34 frankenversion problem that doesn't sit well will either with our working gevent version(1.0.2) or as I found out yesterday, the latest gevent version(1.1.0). So, we don't have an easy path out of this dependency hell atm.

The temporary solution is to freeze the versions that we know worked well until this update, as your investigation suggests.

@schakrava

This comment has been minimized.

Member

schakrava commented Mar 19, 2016

Fixed by #1220

@schakrava schakrava closed this Mar 19, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment