update script for CometVisu installation

[peuter]

Python helper script that let the user decide which version to install (latest release or latest nightly build).
Downloads the CometVisu, extracts the archive, copies old config folder, and custom.css files (from designs) to the new CometVisu and replaces the old installation (which is moved to a backup folder).

Are there any other user customization files that should be copied?

[coveralls]

Coverage decreased (-0.01%) to 60.11% when pulling 2cbc003 on peuter:install-script into 915f908 on CometVisu:develop.

[peuter]

I dont know which python versions are available on wiregate, please test if the script runa on thta device.

[ChristianMayer]

Python versions on WireGate are 2.5 and 2.7 with

ll `which python`
lrwxrwxrwx 1 root root 18 27. Dez 2015  /usr/bin/python -> /usr/bin/python2.7

[ChristianMayer]

• Documentation (manual) is missing
• just running it also doesn't help much:

  # python update.py 
  you have to specify a path to your current CometVisu installation
  

• Running python update.py --help shows in the first line usage: update.py - CometVisu update script what I think is not intuitve, better is e.g. this random example

  $ cat --help
  Aufruf: cat [OPTION]... [DATEI]...
  Aneinanderfügung aller DATEI(en) sortiert auf die Standardausgabe schreiben.
  

• running it with a empty dir gives a traceback
• finally, copying a valid version file there gives:

# python update.py /tmp/tst/in
Traceback (most recent call last):
  File "update.py", line 169, in <module>
    nightly = get_latest_nightly()
  File "update.py", line 29, in get_latest_nightly
    response = request.urlopen(NIGHTLY_PATH)
  File "/usr/lib/python2.7/urllib.py", line 86, in urlopen
    return opener.open(url)
  File "/usr/lib/python2.7/urllib.py", line 207, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.7/urllib.py", line 436, in open_https
    h.endheaders(data)
  File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 776, in send
    self.connect()
  File "/usr/lib/python2.7/httplib.py", line 1161, in connect
    self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
  File "/usr/lib/python2.7/ssl.py", line 381, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 143, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
    self._sslobj.do_handshake()
IOError: [Errno socket error] [Errno 104] Die Verbindung wurde vom Kommunikationspartner zur�ckgesetzt

[peuter]

I did improve the documentation and the handling of empty/not existing target directories.

The last one seems to be a connection problem on your system, I'm not sure how to handle that. As in this case the script cannot do anything, I added a try..catch block around the network calls, but I am not sure if that is the right solution.

[peuter]

Might be related to this one: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=678035

Could you try the workaround mentioned in the last post?

[ChristianMayer]

Just following that page without knowing what I was doing:

# python
Python 2.7.3 (default, Dec  3 2014, 10:50:19) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl, inspect, socket
>>> exec inspect.getsource(ssl.wrap_socket).replace("PROTOCOL_SSLv23","PROTOCOL_SSLv3") in dict(inspect.getmembers(ssl.wrap_socket))["func_globals"]
>>> ssl.wrap_socket(socket.create_connection(('dl.bintray.com', 443)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 12, in wrap_socket
  File "/usr/lib/python2.7/ssl.py", line 143, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
    self._sslobj.do_handshake()
socket.error: [Errno 104] Connection reset by peer

[peuter]

Sorry there isn't really much I can do here without a wiregate. I just tried the script in a docker container (debian wheezy with python 2.7.3) and it works fine in that setup. So its not just about the python version.

Unless someone can provide me a "wiregate-like" environment in a VM or a docker image, I can't help here.

b.t.w. executing the lines mentioned in #719 (comment) in my docker container runs into the same error, but:

import socket, ssl
sg = ssl.wrap_socket(socket.create_connection(('dl.bintray.com', 443)), ssl_version=ssl.PROTOCOL_SSLv23)

did work without error.

[peuter]

Is there a way to savely detect, that the script is running on a wiregate. In case the script could just use a wget call to download the files. But even wget might fail to download, should be tested before modifiying the script.

@ChristianMayer Can you test if the files can be downloaded with wget on the wiregate?

[ChristianMayer]

You are right wget also doesn't work on the wiregate any more:

# wget https://dl.bintray.com/cometvisu/CometVisu/                                                                                                                    
--2018-10-07 11:52:54--  https://dl.bintray.com/cometvisu/CometVisu/                                                                                                                        
Aufl�sen des Hostnamen �dl.bintray.com�.... 159.122.18.156                                                                                                                                  
Verbindungsaufbau zu dl.bintray.com|159.122.18.156|:443... verbunden.                                                                                                                       
Es ist nicht m�glich, eine SSL-Verbindung herzustellen.
                                                                                                                                     
# curl https://dl.bintray.com/cometvisu/CometVisu/                                                                                                                    
curl: (35) Unknown SSL protocol error in connection to dl.bintray.com:443                                                                                                                   

I'll open a ticket about the broken SSL but I wouldn't be surprised when they don't want to put much effort into the wiregate anymore as they are concentrating on the Timberwolf.

For our side: Could we also get this data with ordinary HTTP and no SSL?

To figure out whether we are running on a wiregate is easy:

# uname -n    
wiregate1234

[peuter]

I dont think that those pages are available via http. I just get forwarded to https if I try to. So the only thing left is disabling this script if its running on a wiregate until they fix the problem, which might never happen.

So detecting a wiregate is possible by checking if the hostname starts with wiregate? Is this really reliable? The user might have changed the hostname, or is that impossible on a wiregate?

[ChristianMayer]

It is unlikely that it was changed.
We could also check the existance of the directory /etc/wiregate/.
Or, probably the most reliable:

# uname -r
3.5.1-wiregate-1.46

I can't think of a situation where a Non-Wiregate would run this kernel and I can also not think about a situation where the wiregate would run a stock kernel.
(Crazy people who would run their own kernel on a wiregate could also fix the OpenSSL stuff easily...)