Skip to content
No description, website, or topics provided.
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
grid
.gitignore
README.md
example.py
setup.py

README.md

GRID

Install

pip install git+https://github.com/mariogeiger/grid

or in local

python setup.py install

Usage

The file example.py is a minimal script that can be used with grid.

python -m grid results "python example.py" --a 1 2 3 --b 5 6 7

All the combination of (1, 2, 3) and (5, 6, 7) are executed in simultaneously

[a=1 b=5] python example.py --pickle results/07405.pkl --a 1 --b 5
[a=1 b=6] python example.py --pickle results/04524.pkl --a 1 --b 6
[a=1 b=7] python example.py --pickle results/08265.pkl --a 1 --b 7
[a=2 b=5] python example.py --pickle results/00672.pkl --a 2 --b 5
[a=2 b=6] python example.py --pickle results/06681.pkl --a 2 --b 6
[a=2 b=7] python example.py --pickle results/06732.pkl --a 2 --b 7
[a=3 b=5] python example.py --pickle results/02937.pkl --a 3 --b 5
[a=3 b=6] python example.py --pickle results/08754.pkl --a 3 --b 6
[a=3 b=7] python example.py --pickle results/08521.pkl --a 3 --b 7
[a=2 b=5] computation 1 / 3
[a=2 b=6] computation 1 / 3
[a=1 b=7] computation 1 / 3
[a=1 b=6] computation 1 / 3
[a=1 b=5] computation 1 / 3
[a=2 b=7] computation 1 / 3
[a=3 b=6] computation 1 / 3
[a=3 b=7] computation 1 / 3
[a=3 b=5] computation 1 / 3
[a=2 b=5] computation 2 / 3
[a=2 b=6] computation 2 / 3
[a=1 b=7] computation 2 / 3
[a=1 b=6] computation 2 / 3
[a=1 b=5] computation 2 / 3
[a=2 b=7] computation 2 / 3
[a=3 b=6] computation 2 / 3
[a=3 b=7] computation 2 / 3
[a=3 b=5] computation 2 / 3
[a=2 b=5] computation 3 / 3
[a=2 b=6] computation 3 / 3
[a=1 b=7] computation 3 / 3
[a=1 b=6] computation 3 / 3
[a=1 b=5] computation 3 / 3
[a=2 b=7] computation 3 / 3
[a=3 b=5] computation 3 / 3
[a=3 b=6] computation 3 / 3
[a=3 b=7] computation 3 / 3
python -m grid results "python example.py" --a 1 2 3 --b 5 6 7 8

The results already done are not executed again

[a=1 b=5] already done
[a=1 b=6] already done
[a=1 b=7] already done
[a=1 b=8] python example.py --pickle results/07530.pkl --a 1 --b 8
[a=2 b=5] already done
[a=2 b=6] already done
[a=2 b=7] already done
[a=2 b=8] python example.py --pickle results/03495.pkl --a 2 --b 8
[a=3 b=5] already done
[a=3 b=6] already done
[a=3 b=7] already done
[a=3 b=8] python example.py --pickle results/06283.pkl --a 3 --b 8
[a=2 b=8] computation 1 / 3
[a=3 b=8] computation 1 / 3
[a=1 b=8] computation 1 / 3
[a=2 b=8] computation 2 / 3
[a=3 b=8] computation 2 / 3
[a=1 b=8] computation 2 / 3
[a=3 b=8] computation 3 / 3
[a=2 b=8] computation 3 / 3
[a=1 b=8] computation 3 / 3

Example: using slurm and only varying parameter b

python -m grid results "srun --partition gpu --qos gpu --gres gpu:1 --time 3-00:00:00 --mem 12G --pty python example.py --a 4" --b:int 5 6 7

Optional arguments

--n N maximum number of job running in the same time

--sleep S time to sleep in seconds between two job launch

Subcommands

  • pyhton -m grid.clear log_dir remove files with no content (typically resultant of interrupted runs)
  • pyhton -m grid.merge log_dir_src log_dir_dst
  • pyhton -m grid.info log_dir
You can’t perform that action at this time.