Merge branch '1.x' into 'master'

VIDA-NYU · Aug 11, 2021 · ff71cde · ff71cde
2 parents 040c4a8 + aa28386
commit ff71cde
Show file tree

Hide file tree

Showing 6 changed files with 191 additions and 24 deletions.
diff --git a/CITATION.cff b/CITATION.cff
@@ -0,0 +1,141 @@
+cff-version: 1.2.0
+message: If you use this software, please cite it as below.
+authors:
+  - family-names: Rampin
+    given-names: Remi
+    affiliation: New York University
+    orcid: https://orcid.org/0000-0002-0524-2282
+    website: https://remi.rampin.org/
+  - family-names: Freire
+    given-names: Juliana
+    affiliation: New York University
+    orcid: https://orcid.org/0000-0003-3915-7075
+    website: https://vgc.poly.edu/~juliana/
+  - family-names: Chirigati
+    given-names: Fernando
+    affiliation: New York University
+    orcid: https://orcid.org/0000-0002-9566-5835
+    website: http://fchirigati.com/
+  - family-names: Shasha
+    given-names: Dennis
+    affiliation: New York University
+    orcid: https://orcid.org/0000-0002-7036-3312
+    website: http://cs.nyu.edu/shasha/
+  - family-names: Rampin
+    given-names: Vicky
+    affiliation: New York University
+    orcid: https://orcid.org/0000-0003-4298-168X
+    website: https://vicky.rampin.org/
+license: BSD-3-Clause
+url: https://www.reprozip.org/
+repository-code: https://github.com/VIDA-NYU/reprozip
+title: ReproZip
+abstract: |
+  ReproZip is a tool aimed at simplifying the process of creating reproducible experiments from command-line executions, a frequently-used common denominator in computational science. It tracks operating system calls and creates a package that contains all the binaries, files and dependencies required to run a given command on the author's computational environment (packing step).
+keywords: [python, linux, docker, reproducibility, provenance, reproducible-research, reproducible-science]
+references:
+  - type: proceedings
+    doi: 10.1145/2882903.2899401
+    conference:
+      name: "SIGMOD '16"
+      website: https://www.sigmod2016.org/
+      city: San Francisco
+      country: US
+    authors:
+      - family-names: Rampin
+        given-names: Remi
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0002-0524-2282
+        website: https://remi.rampin.org/
+      - family-names: Freire
+        given-names: Juliana
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0003-3915-7075
+        website: https://vgc.poly.edu/~juliana/
+      - family-names: Chirigati
+        given-names: Fernando
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0002-9566-5835
+        website: http://fchirigati.com/
+      - family-names: Shasha
+        given-names: Dennis
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0002-7036-3312
+        website: http://cs.nyu.edu/shasha/
+    date-published: 2016-06-26
+    year: 2016
+    month: 6
+    title: "ReproZip: Computational Reproducibility With Ease"
+    abstract: |
+      We present ReproZip, the recommended packaging tool for the SIGMOD Reproducibility Review. ReproZip was designed to simplify the process of making an existing computational experiment reproducible across platforms, even when the experiment was put together without reproducibility in mind. The tool creates a self-contained package for an experiment by automatically tracking and identifying all its required dependencies. The researcher can share the package with others, who can then use ReproZip to unpack the experiment, reproduce the findings on their favorite operating system, as well as modify the original experiment for reuse in new research, all with little effort. The demo will consist of examples of non-trivial experiments, showing how these can be packed in a Linux machine and reproduced on different machines and operating systems. Demo visitors will also be able to pack and reproduce their own experiments.
+  - type: proceedings
+    doi: 10.21105/joss.00107
+    journal: Journal of Open Source Software
+    authors:
+      - family-names: Rampin
+        given-names: Remi
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0002-0524-2282
+        website: https://remi.rampin.org/
+      - family-names: Freire
+        given-names: Juliana
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0003-3915-7075
+        website: https://vgc.poly.edu/~juliana/
+      - family-names: Chirigati
+        given-names: Fernando
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0002-9566-5835
+        website: http://fchirigati.com/
+      - family-names: Shasha
+        given-names: Dennis
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0002-7036-3312
+        website: http://cs.nyu.edu/shasha/
+      - family-names: Rampin
+        given-names: Vicky
+        affiliation: New York University
+        orcid: https://orcid.org/0000-0003-4298-168X
+        website: https://vicky.rampin.org/
+    date-published: 2016-12-01
+    year: 2016
+    month: 12
+    title: "ReproZip: The Reproducibility Packer"
+preferred-citation:
+  type: proceedings
+  doi: 10.1145/2882903.2899401
+  conference:
+    name: "SIGMOD '16"
+    website: https://www.sigmod2016.org/
+    city: San Francisco
+    country: US
+  authors:
+    - family-names: Rampin
+      given-names: Remi
+      affiliation: New York University
+      orcid: https://orcid.org/0000-0002-0524-2282
+      website: https://remi.rampin.org/
+    - family-names: Freire
+      given-names: Juliana
+      affiliation: New York University
+      orcid: https://orcid.org/0000-0003-3915-7075
+      website: https://vgc.poly.edu/~juliana/
+    - family-names: Chirigati
+      given-names: Fernando
+      affiliation: New York University
+      orcid: https://orcid.org/0000-0002-9566-5835
+      website: http://fchirigati.com/
+    - family-names: Shasha
+      given-names: Dennis
+      affiliation: New York University
+      orcid: https://orcid.org/0000-0002-7036-3312
+      website: http://cs.nyu.edu/shasha/
+  date-published: 2016-06-26
+  year: 2016
+  month: 6
+  title: "ReproZip: Computational Reproducibility With Ease"
+  abstract: |
+    We present ReproZip, the recommended packaging tool for the SIGMOD Reproducibility Review. ReproZip was designed to simplify the process of making an existing computational experiment reproducible across platforms, even when the experiment was put together without reproducibility in mind. The tool creates a self-contained package for an experiment by automatically tracking and identifying all its required dependencies. The researcher can share the package with others, who can then use ReproZip to unpack the experiment, reproduce the findings on their favorite operating system, as well as modify the original experiment for reuse in new research, all with little effort. The demo will consist of examples of non-trivial experiments, showing how these can be packed in a Linux machine and reproduced on different machines and operating systems. Demo visitors will also be able to pack and reproduce their own experiments.
+version: "1.1"
+date-released: 2021-07-06
+doi: 10.5281/zenodo.5081097
diff --git a/reprounzip/reprounzip/__main__.py b/reprounzip/reprounzip/__main__.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprozip-jupyter/reprozip_jupyter/__init__.py b/reprozip-jupyter/reprozip_jupyter/__init__.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/reprozip/reprozip/__main__.py b/reprozip/reprozip/__main__.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 

diff --git a/tests/check_images.py b/tests/check_images.py
@@ -1,10 +1,12 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.
 
 import logging
+import re
 import requests
 import time
+from urllib.parse import urlencode
 
 from reprounzip.parameters import _bundled_parameters
 
@@ -109,6 +111,41 @@ def check_vagrant():
         raise AssertionError("Missing Vagrant boxes")
 
 
+def list_docker_tags(repository, token=None):
+    headers = {}
+    if token is not None:
+        headers['Authorization'] = 'Bearer %s' % token
+    res = requests.get(
+        'https://%s/v2/%s/%s/tags/list' % (
+            repository[0], repository[1], repository[2],
+        ),
+        headers=headers,
+    )
+    if token is None and res.status_code == 401:
+        # Authenticate
+        m = re.match(
+            r'Bearer realm="([^"]+)",service="([^"]+)"',
+            res.headers['www-authenticate'],
+        )
+        if m is None:
+            res.raise_for_status()
+        scope = 'repository:%s/%s:pull' % (repository[1], repository[2])
+        res = requests.get(
+            m.group(1) + '?' + urlencode({
+                'service': m.group(2),
+                'scope': scope,
+            }),
+        )
+        res.raise_for_status()
+        token = res.json()['token']
+        # Try again with token
+        return list_docker_tags(repository, token)
+
+    res.raise_for_status()
+
+    return res.json()['tags']
+
+
 def check_docker():
     error = False
 
@@ -129,35 +166,24 @@ def check_docker():
         else:
             tag = 'latest'
         if len(image) == 1:
-            image = ['docker.io', 'library'] + image
+            image = ['index.docker.io', 'library'] + image
         elif len(image) == 2:
-            image = ['docker.io'] + image
+            image = ['index.docker.io'] + image
         repositories.setdefault(tuple(image[:3]), set()).add(tag)
 
     # Check that each repository has the required tags
     for repository, tags in repositories.items():
-        registry = repository[0]
-        if registry == 'docker.io':
-            registry = 'hub.docker.com'
-        else:
-            raise AssertionError("Registry unsupported: %s" % registry)
-        url = (
-            'https://' + registry
-            + '/v1/repositories/' + repository[1]
-            + '/' + repository[2]
-            + '/tags'
-        )
-        res = requests.get(url)
-        if res.status_code != 200:
+        try:
+            actual_tags = list_docker_tags(repository)
+        except requests.HTTPError as e:
             logger.error(
                 "Docker repository not found: %d %s",
-                res.status_code,
-                url,
+                e.response.status_code,
+                e.request.url,
             )
             error = True
             continue
-        actual_tags = res.json()
-        actual_tags = set(entry['name'] for entry in actual_tags)
+        actual_tags = set(actual_tags)
         for tag in tags:
             if tag not in actual_tags:
                 logger.error(

diff --git a/tests/test_rails_filter.py b/tests/test_rails_filter.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2017 New York University
+# Copyright (C) 2014 New York University
 # This file is part of ReproZip which is released under the Revised BSD License
 # See file LICENSE for full license details.