Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide reproducible software environment deployment with GNU Guix #1

Open
wants to merge 3 commits into
base: full-replication
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: build
on: [ push ]
jobs:
build:
name: Build environment
runs-on: ubuntu-22.04
steps:
- name: Guix cache
uses: actions/cache@v2
with:
path: ~/.cache/guix
# use a key that (almost) never matches
key: guix-cache-${{ github.sha }}
restore-keys: |
guix-cache-
- name: Install Guix
uses: PromyLOPh/guix-install-action@v1
- name: Checkout
uses: actions/checkout@v2
- name: Build environment
run: guix time-machine -C channels.scm -- shell -m manifest.scm -- python --version
20 changes: 18 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,26 @@ Recanatesi *et al.* present a model of memory retrieval based on a Hopfield mode
Dependencies
------------

Python 3 with packages in `requirements.txt`.
Using [GNU Guix](https://guix.gnu.org), set up the complete software
environment with:

```
$ pip install -r requirements
guix time-machine -C channels.scm -- \
shell -m manifest.scm --container
```

The `channels.scm` file instructs how to [replicate the exact Guix
revision used for
testing](https://guix.gnu.org/manual/en/html_node/Replicating-Guix.html),
while `manifest.scm` defines [the software
environment](https://guix.gnu.org/manual/en/html_node/Writing-Manifests.html)
of this computational experiment.

Alternatively, if you are not using Guix, install Python 3 and then
install all the packages in `requirements.txt` with:

```
pip install -r requirements
```

Usage
Expand Down
18 changes: 18 additions & 0 deletions channels.scm
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
;; Set of GNU Guix channels to use to reproduce this computational
;; experiment. Run:
;;
;; guix time-machine -C channels.scm -- shell -m manifest.scm
;;
;; to enter the environment built from this very Guix revision.

(list (channel

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity, shouldn't channel inferiors (that provide packages versions which are as close as possible to the minimal version constraints in requirements.txt) be defined?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is more of a philosophical than a technical question. It depends on the criteria that the authors applied to define those minimal version constraints, and that is hardly ever documented. In practice, judging from my own experience, version constraints are rarely useful for reproducibility. The fundamental assumption that "x.x or higher" yields the same results is simply wrong. Only strict version equality (same commit ID) has practical value.

(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git")
(branch "master")
(commit
"dc90c0807d0a46cdd4b0a2c2b3f9becca9f97285")
(introduction
(make-channel-introduction
"9edb3f66fd807b096b48283debdcddccfea34bad"
(openpgp-fingerprint
"BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA")))))
20 changes: 20 additions & 0 deletions manifest.scm
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
;; GNU Guix manifest to set up the execution environment of this
;; computational experiment. Run:
;;
;; guix shell -m manifest.scm
;;
;; to enter the environment, possibly adding '--container' to avoid
;; interference with the host system.

(specifications->manifest
'("coreutils"
"findutils"
"sed"
"grep"

"python-wrapper"
"python-matplotlib"
"python-numpy"
"python-pandas"
"python-scipy"
"python-tqdm"))
9 changes: 4 additions & 5 deletions plot.job
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/bin/sh
############## This section states the requirements the job requires:
#SBATCH --job-name=re2015p
#SBATCH --time 15:00
Expand All @@ -8,10 +8,9 @@
#SBATCH -o logs/log.out
############## Here starts the actual UNIX commands and payload:

module purge
module load anaconda

srun python plot.py
exec guix time-machine -C channels.scm -- \
shell -m manifest.scm --pure -- \
srun python plot.py

# Make sure to edit according to your needs, especially:
# - Working directory (under a "SBATCH" line)
Expand Down
11 changes: 5 additions & 6 deletions simulation.job
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@
#!/bin/bash
#!/bin/sh
############## This section states the requirements the job requires:
#SBATCH --job-name=rec2015
#SBATCH --time 3:00:00
#SBATCH -c 1
#SBATCH --mem=70000MB
#SBATCH --ntasks=1
#SBATCH --array=0-9999
###SBATCH --array=0-9999 # FIXME: not working with slurm@22.05
#SBATCH -o logs/log.out
############## Here starts the actual UNIX commands and payload:

module purge
module load anaconda

srun python simulation.py $SLURM_ARRAY_TASK_ID
exec guix time-machine -C channels.scm -- \
shell -m manifest.scm --pure -- \
srun python simulation.py $SLURM_ARRAY_TASK_ID

# Make sure to edit according to your needs, especially:
# - Working directory (under a "SBATCH" line)
Expand Down