Merge branch '1.x' into 'master'

VIDA-NYU · Jul 7, 2021 · 8e87347 · 8e87347
2 parents 7e9a72c + 9388783
commit 8e87347
Show file tree

Hide file tree

Showing 77 changed files with 162 additions and 697 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,7 +1,7 @@
 Changelog
 =========
 
-1.1.0 (???)
+2.0.0 (???)
 -----------
 
 Behavior change:
@@ -11,6 +11,27 @@ Features:
 * Configuration file contains the walltime taken by each run
 * It is now possible to upload or download any file via its full path
 
+1.1 (2021-07-06)
+----------------
+
+(reprounzip-vistrails didn't change)
+
+Bugfixes:
+* Fix possible crash reading `docker inspect` output
+* Fix reprozip-jupyter on more recent Tornado
+* Fix failure in reprounzip-docker upload if `/bin/sh` is missing
+* Have `reprounzip directory` execute with the correct interpreter (e.g. packed `ld-linux.so`)
+* Fix "invalid cross-device link" errors in vagrant download if the destination is not on the same device as the unpacked directory
+
+Enhancements:
+* Add PyQt5 compatibility to reprounzip-qt
+* reprounzip-docker: Keep ownership of uploaded files
+* Add a new certificate to download parameters, valid through 2121 (previous expires 2024)
+* Always show full path of executed files in `reprozip testrun`
+* Improved merging of pack on top of base image in reprounzip-docker
+* Made sure the root is a mountpoint in reprounzip-vagrant, as some applications expect it (e.g. Elasticsearch)
+* Disable unpacker options that won't work in reprounzip-qt (e.g. 'directory' and 'chroot' on non-Linux)
+
 1.0.16 (2019-02-06)
 -------------------
 

diff --git a/LICENSE.txt b/LICENSE.txt
@@ -1,4 +1,4 @@
-Copyright (C) 2014-2017, New York University
+Copyright (C) 2014, New York University
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

diff --git a/README.md b/README.md
@@ -1,10 +1,10 @@
 [![Build Status](https://github.com/VIDA-NYU/reprozip/workflows/Test/badge.svg)](https://github.com/VIDA-NYU/reprozip/actions)
-[![Coverage Status](https://codecov.io/github/VIDA-NYU/reprozip/coverage.svg?branch=1.0.x)](https://codecov.io/github/VIDA-NYU/reprozip?branch=1.0.x)
-[![Documentation Status](https://readthedocs.org/projects/reprozip/badge/?version=1.0.x)](https://docs.reprozip.org/en/1.0.x/)
+[![Coverage Status](https://codecov.io/github/VIDA-NYU/reprozip/coverage.svg?branch=1.x)](https://codecov.io/github/VIDA-NYU/reprozip?branch=1.x)
+[![Documentation Status](https://readthedocs.org/projects/reprozip/badge/?version=1.x)](https://docs.reprozip.org/en/1.x/)
 [![Matrix](https://img.shields.io/badge/chat-matrix.org-blue.svg)](https://riot.im/app/#/room/#reprozip:matrix.org)
 [![Say Thanks!](https://img.shields.io/badge/Say%20Thanks-!-1EAEDB.svg)](https://saythanks.io/to/remram44)
 [![status](https://img.shields.io/badge/JOSS-10.21105%2Fjoss.00107-green.svg)](http://joss.theoj.org/papers/b578b171263c73f64dfb9d040ca80fe0)
-[![DOI](https://img.shields.io/badge/DOI/10.5281%2Fzenodo.2558333-green.svg)](https://doi.org/10.5281/zenodo.2558333)
+[![DOI](https://img.shields.io/badge/DOI/10.5281%2Fzenodo.5081097-green.svg)](https://doi.org/10.5281/zenodo.5081097)
 
 ReproZip
 ========
@@ -119,7 +119,7 @@ For more information, please visit [the informational page](https://sustainable-
 License
 -------
 
-* Copyright (C) 2014-2017, New York University
+* Copyright (C) 2014, New York University
 
 Licensed under a **BSD 3-Clause license**. See the file [LICENSE.txt](LICENSE.txt) for details.
 

diff --git a/docs/conf.py b/docs/conf.py
@@ -51,7 +51,7 @@
 
 # General information about the project.
 project = u'ReproZip'
-copyright = u'2014-2017, New York University'
+copyright = u'2014, New York University'
 
 authors = [u'Fernando Chirigati', u'Remi Rampin',
            u'Juliana Freire', u'Dennis Shasha']

diff --git a/docs/man/reprounzip-docker.1 b/docs/man/reprounzip-docker.1
@@ -80,6 +80,6 @@ Dennis Shasha
 .fi
 .RE
 .SH COPYRIGHT
-Copyright 2014-2017 New York University.
+Copyright 2014 New York University.
 
 .RB "Licensed under a" " BSD 3-Clause license." " See the LICENSE file included with the software for details."
diff --git a/docs/man/reprounzip-vagrant.1 b/docs/man/reprounzip-vagrant.1
@@ -71,6 +71,6 @@ Dennis Shasha
 .fi
 .RE
 .SH COPYRIGHT
-Copyright 2014-2017 New York University.
+Copyright 2014 New York University.
 
 .RB "Licensed under a" " BSD 3-Clause license." " See the LICENSE file included with the software for details."
diff --git a/docs/man/reprounzip.1 b/docs/man/reprounzip.1
@@ -142,6 +142,6 @@ Dennis Shasha
 .fi
 .RE
 .SH COPYRIGHT
-Copyright 2014-2017 New York University.
+Copyright 2014 New York University.
 
 .RB "Licensed under a" " BSD 3-Clause license." " See the LICENSE file included with the software for details."
diff --git a/docs/man/reprozip.1 b/docs/man/reprozip.1
@@ -106,6 +106,6 @@ Dennis Shasha
 .fi
 .RE
 .SH COPYRIGHT
-Copyright 2014-2017 New York University.
+Copyright 2014 New York University.
 
 .RB "Licensed under a" " BSD 3-Clause license." " See the LICENSE file included with the software for details."
diff --git a/reprounzip-docker/LICENSE.txt b/reprounzip-docker/LICENSE.txt
@@ -1,4 +1,4 @@
-Copyright (C) 2014-2017, New York University
+Copyright (C) 2014, New York University
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

diff --git a/reprounzip-docker/reprounzip_docker.py b/reprounzip-docker/reprounzip_docker.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 
@@ -700,7 +700,7 @@ def docker_run(args):
 
 class ContainerUploader(FileUploader):
     def __init__(self, target, input_files, files, unpacked_info,
-                 docker_cmd='docker'):
+                 docker_cmd=['docker']):
         self.unpacked_info = unpacked_info
         self.docker_cmd = docker_cmd
         config = load_config(target / 'config.yml', True)
@@ -821,10 +821,8 @@ def docker_upload(args):
 
 
 class ContainerDownloader(FileDownloader):
-    def __init__(
-        self, target, files, image, all_=False,
-        docker_cmd=['docker'],
-    ):
+    def __init__(self, target, files, image, all_=False,
+                 docker_cmd=['docker']):
         self.image = image
         self.docker_cmd = docker_cmd
         FileDownloader.__init__(self, target, files, all_=all_)

diff --git a/reprounzip-qt/LICENSE.txt b/reprounzip-qt/LICENSE.txt
@@ -1,4 +1,4 @@
-Copyright (C) 2014-2017, New York University
+Copyright (C) 2014, New York University
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

diff --git a/reprounzip-qt/reprounzip_qt/__init__.py b/reprounzip-qt/reprounzip_qt/__init__.py
@@ -1,5 +1 @@
-# Copyright (C) 2014-2017 New York University
-# This file is part of ReproZip which is released under the Revised BSD License
-# See file LICENSE for full license details.
-
-__version__ = '1.0.16'
+__version__ = '2.0.0'
diff --git a/reprounzip-qt/reprounzip_qt/gui/__init__.py b/reprounzip-qt/reprounzip_qt/gui/__init__.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip-qt/reprounzip_qt/gui/common.py b/reprounzip-qt/reprounzip_qt/gui/common.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip-qt/reprounzip_qt/gui/run.py b/reprounzip-qt/reprounzip_qt/gui/run.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip-qt/reprounzip_qt/gui/unpack.py b/reprounzip-qt/reprounzip_qt/gui/unpack.py
@@ -1,10 +1,11 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 
 import os
 from qtpy import QtCore, QtWidgets
 import subprocess
+import sys
 
 import reprounzip_qt.reprounzip_interface as reprounzip
 from reprounzip_qt.gui.common import ROOT, ResizableStack, \
@@ -26,6 +27,10 @@ def add_row(self, label, widget):
     def options(self):
         return {'args': []}
 
+    @staticmethod
+    def available():
+        return True
+
 
 class DirectoryOptions(UnpackerOptions):
     def __init__(self):
@@ -34,6 +39,10 @@ def __init__(self):
             QtWidgets.QLabel("(directory unpacker has no option)"),
             0, 0, 1, 2)
 
+    @staticmethod
+    def available():
+        return sys.platform.startswith('linux')
+
 
 class ChrootOptions(UnpackerOptions):
     def __init__(self):
@@ -73,6 +82,10 @@ def options(self):
 
         return options
 
+    @staticmethod
+    def available():
+        return sys.platform.startswith('linux')
+
 
 class DockerOptions(UnpackerOptions):
     def __init__(self):
@@ -245,11 +258,13 @@ def __init__(self, package='', **kwargs):
                          QtCore.Qt.AlignTop)
         ulayout = QtWidgets.QVBoxLayout()
         self.unpackers = QtWidgets.QButtonGroup()
-        for i, name in enumerate(n for n, c in self.UNPACKERS):
+        for i, (name, opts) in enumerate(self.UNPACKERS):
             radio = QtWidgets.QRadioButton(name)
             radio.unpacker = name
             self.unpackers.addButton(radio, i)
             ulayout.addWidget(radio)
+            if not opts.available():
+                radio.setDisabled(True)
         layout.addLayout(ulayout, 1, 1, 1, 2)
 
         group = QtWidgets.QGroupBox(title="Unpacker options")

diff --git a/reprounzip-qt/reprounzip_qt/main.py b/reprounzip-qt/reprounzip_qt/main.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip-qt/reprounzip_qt/qt_terminal.py b/reprounzip-qt/reprounzip_qt/qt_terminal.py
@@ -1,14 +1,19 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 
-import cgi
 import locale
 import logging
 from qtpy import QtCore, QtWidgets
 import sys
 
 
+if sys.version_info >= (3,):
+    from html import escape
+else:
+    from cgi import escape
+
+
 logger = logging.getLogger('reprounzip_qt')
 
 
@@ -80,7 +85,7 @@ def __init__(self, cmdline, env, input_enabled=False,
 </style>
 ''')
         self.text.append('<span style="color: blue;">%s</span>' %
-                         cgi.escape(' '.join(cmdline)))
+                         escape(' '.join(cmdline)))
 
     def _enter(self):
         cmd = self.input.text()
@@ -91,13 +96,13 @@ def _read_stdout(self):
         out = self.process.readAllStandardOutput()
         out = bytes(out).decode(locale.getpreferredencoding() or 'UTF-8',
                                 'replace')
-        self.text.append('<span>%s</span>' % cgi.escape(out))
+        self.text.append('<span>%s</span>' % escape(out))
 
     def _read_stderr(self):
         out = self.process.readAllStandardError()
         out = bytes(out).decode(locale.getpreferredencoding() or 'UTF-8',
                                 'replace')
-        self.text.append('<span class="err">%s</span>' % cgi.escape(out))
+        self.text.append('<span class="err">%s</span>' % escape(out))
 
     def _finished(self, code, status):
         good = False

diff --git a/reprounzip-qt/reprounzip_qt/reprounzip_interface.py b/reprounzip-qt/reprounzip_qt/reprounzip_interface.py
@@ -1,16 +1,18 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 
 import itertools
 import logging
 import os
+from pathlib import Path
 import pickle
 import subprocess
 import sys
 import time
 import yaml
 
+from reprounzip.common import load_config
 from reprounzip_qt.qt_terminal import run_in_builtin_terminal
 
 
@@ -94,17 +96,19 @@ def __init__(self, name, path, is_input, is_output):
         self.is_input = is_input
         self.is_output = is_output
 
+    def __repr__(self):
+        return '<FileStatus name=%r>' % self.name
+
 
 class FilesStatus(object):
     def __init__(self, directory):
         self.directory = directory
-        with open(os.path.join(directory, 'config.yml')) as fp:
-            config = yaml.safe_load(fp)
+        config = load_config(Path(directory) / 'config.yml', True)
 
-        self.files = [FileStatus(f['name'], f['path'],
-                                 f.get('read_by_runs'),
-                                 f.get('written_by_runs'))
-                      for f in config.get('inputs_outputs') or []]
+        self.files = [
+            FileStatus(name, f.path, f.read_runs, f.write_runs)
+            for name, f in config.inputs_outputs.items()
+        ]
         logger.info("Loaded %d files from the configuration", len(self.files))
         self._refresh()
 

diff --git a/reprounzip-qt/reprounzip_qt/usage.py b/reprounzip-qt/reprounzip_qt/usage.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip-vagrant/LICENSE.txt b/reprounzip-vagrant/LICENSE.txt
@@ -1,4 +1,4 @@
-Copyright (C) 2014-2017, New York University
+Copyright (C) 2014, New York University
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

diff --git a/reprounzip-vagrant/reprounzip_vagrant/__init__.py b/reprounzip-vagrant/reprounzip_vagrant/__init__.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip-vagrant/reprounzip_vagrant/run_command.py b/reprounzip-vagrant/reprounzip_vagrant/run_command.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip-vistrails/LICENSE.txt b/reprounzip-vistrails/LICENSE.txt
@@ -1,4 +1,4 @@
-Copyright (C) 2014-2017, New York University
+Copyright (C) 2014, New York University
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

diff --git a/reprounzip-vistrails/reprounzip_vistrails.py b/reprounzip-vistrails/reprounzip_vistrails.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip/LICENSE.txt b/reprounzip/LICENSE.txt
@@ -1,4 +1,4 @@
-Copyright (C) 2014-2017, New York University
+Copyright (C) 2014, New York University
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

diff --git a/reprounzip/reprounzip/main.py b/reprounzip/reprounzip/main.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip/reprounzip/pack_info.py b/reprounzip/reprounzip/pack_info.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip/reprounzip/parameters.py b/reprounzip/reprounzip/parameters.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprounzip/reprounzip/signals.py b/reprounzip/reprounzip/signals.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.