Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to python 3 #50

Merged
merged 52 commits into from
Feb 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
0194c80
Add new conda environment file
HealthyPear Nov 22, 2021
119aae1
Make split_dataset usable
HealthyPear Nov 22, 2021
bcd8787
Merge branch 'master' into upgrade_python_3
HealthyPear Nov 24, 2021
7697b49
Update conda environment file
HealthyPear Nov 25, 2021
22cde04
Remove requirements file in favor of new conda environment
HealthyPear Nov 25, 2021
e9768d8
Update setup script
HealthyPear Nov 25, 2021
f496941
Update submit jobs
HealthyPear Nov 25, 2021
a6cb6fd
Update .gitignore
HealthyPear Nov 25, 2021
e790610
Update CI settings
HealthyPear Nov 25, 2021
0b10ea0
Fix stale-code bug in CI
HealthyPear Nov 25, 2021
ac07b4d
Fix environment file extension during install
HealthyPear Nov 25, 2021
43e2bbe
Add license file
HealthyPear Jan 15, 2022
d180ba3
Define the default analysis workflow
HealthyPear Jan 15, 2022
c0c37d9
Make .gitignore similar to protopipe
HealthyPear Jan 15, 2022
cd219d0
Add a prospector style file
HealthyPear Jan 15, 2022
1341fce
Delete old analysis workflow file
HealthyPear Jan 15, 2022
9e45e22
Containerization
HealthyPear Jan 15, 2022
4b02bd0
Remove old singularity recipe file
HealthyPear Jan 15, 2022
9b538fe
Move pilot script and grid config file to aux
HealthyPear Jan 15, 2022
d300b15
Add a logging configuration file
HealthyPear Jan 15, 2022
09074b3
Packaging, logging and testing
HealthyPear Jan 15, 2022
0a6dfc5
Update CI, testing and dependencies
HealthyPear Jan 15, 2022
9b679f1
Improve merge script
HealthyPear Jan 16, 2022
1833ceb
Address many codacy complaints and improve testing
HealthyPear Jan 16, 2022
9072fa8
Merge oprionally without fancy logger
HealthyPear Jan 19, 2022
3ff477b
Simplify script for filtering replicas
HealthyPear Jan 19, 2022
73d9dec
Add new script to retrieve LFNs from job
HealthyPear Jan 19, 2022
5850371
Remove old files
HealthyPear Jan 19, 2022
a380411
Add break statement in while/else clause
HealthyPear Jan 19, 2022
62a6886
Fix codestyle in utils.makedir
HealthyPear Jan 19, 2022
8445fcb
Remove CI test script
HealthyPear Jan 20, 2022
d8ee6ed
Update Singularity recipe for python3
HealthyPear Jan 20, 2022
f30b3dc
Set 1 process when syncing directories
HealthyPear Jan 20, 2022
2550b1b
Fix bug introduced with new logging
HealthyPear Jan 20, 2022
ced7380
Now remaining jobs are counted
HealthyPear Jan 20, 2022
d60a9a8
Update and fix Docker recipes
HealthyPear Jan 20, 2022
746a2e4
Improve logger when merging on the grid
HealthyPear Jan 20, 2022
318a61f
Improve logger for get_input_LFNs.py
HealthyPear Jan 20, 2022
29ff35f
Remove call to test.py in CI
HealthyPear Jan 20, 2022
9433ddb
Improve logging in get_input_LFNs.py
HealthyPear Jan 20, 2022
3c33bcb
Improve submitting of jobs
HealthyPear Jan 20, 2022
9d4b543
Remove from README containerization section
HealthyPear Jan 20, 2022
400332e
Fix formatting bug in submit_jobs.py
HealthyPear Jan 22, 2022
a5a5aa0
Improve debug logging in submitting jobs
HealthyPear Jan 28, 2022
76182f4
Add info about energy estimation at submission
HealthyPear Jan 29, 2022
b18230b
Use lazy % formatting
HealthyPear Jan 29, 2022
55fe329
Remove unnecessary "else" after "break"
HealthyPear Jan 29, 2022
5af7c85
More % lazy formatting in logging calls
HealthyPear Jan 29, 2022
047aa77
More lazy % formatting in logging calls
HealthyPear Jan 29, 2022
0b6286e
Fix check of existing file on the grid
HealthyPear Feb 1, 2022
a3a4293
Fix conda env
HealthyPear Feb 1, 2022
9bad6d5
Reformatting with black
HealthyPear Feb 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/bash

if [[ "$INSTALL_METHOD" == "conda" ]]; then
echo "Using conda"
sudo chown -R "$USER" "$CONDA" # Give CONDA permission to its own files
source "$CONDA"/etc/profile.d/conda.sh
conda config --set always_yes yes --set changeps1 no
conda update -q conda # get latest conda version
# Useful for debugging any issues with conda
conda info -a

sed -i -e "s/- python=.*/- python=$PYTHON_VERSION/g" environment_development.yaml
conda install -c conda-forge mamba
mamba env create -n CI --file environment_development.yaml
conda activate CI
else
echo "Using pip"
pip install -U pip setuptools wheel
# This installation is not currently supported
fi
25 changes: 16 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,16 @@ on: [push, pull_request]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
tests:
runs-on: ubuntu-latest
runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: [2.7]
install-method: ["pip"]
os: [ubuntu-latest, macos-latest]
python-version: [3.8]
install-method: ["conda"]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
fetch-depth: 0

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
Expand All @@ -28,12 +29,18 @@ jobs:
INSTALL_METHOD: ${{ matrix.install-method }}
PYTHON_VERSION: ${{ matrix.python-version }}
run: |
source .github/install.sh
python --version
python -m pip install --upgrade pip
python -m pip install pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
pip install codecov pytest-cov
pip install .[all]
pip freeze
- name: Tests
run: pytest --cov=. --cov-report=xml
env:
INSTALL_METHOD: ${{ matrix.install-method }}
run: |
if [[ "$INSTALL_METHOD" == "conda" ]]; then
source $CONDA/etc/profile.d/conda.sh
conda activate CI
conda list
fi
pytest --cov=protopipe --cov-report=xml
- uses: codecov/codecov-action@v1
86 changes: 85 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,86 @@
# Compiled files
.pytest_cache
*.py[co]
*.pyc
**/__pycache__
*.a
*.o
*.so
__pycache__

# Ignore .c files by default to avoid including generated code. If you want to
# add a non-generated .c extension, use `git add -f filename.c`.
*.c

# Other generated files
*/cython_version.py
htmlcov
.coverage
MANIFEST

# ignore version cache file (generated automatically when setup.py is run)
protopipe_grid_interface/_version_cache.py
protopipe_grid_interface/_version.py

# Eclipse editor project files
.project
.pydevproject
.settings

# Pycharm editor project files
.idea

# KDevelop editor project files
*kdev4

# VSCode
.vscode

# Packages/installer info
*.egg
*.eggs
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
distribute-*.tar.gz

# Other
.*.swp
*~
\#*
.\#*

# Mac OSX
.DS_Store

# Rope refactoring data:
.ropeproject

# ipython notebook checkpoints
.ipynb_checkpoints
.cache*
.cache

*.out

#sonarlint command-line output
.sonarlint

# output files (should not be stored in git)
*.log
*.h5
*.hdf5
*.fits
irf.fits.gz
table_best_cutoff.fits
*.pkl.gz

*.pstats
target
.mypy_cache
8 changes: 8 additions & 0 deletions .prospector.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
pep257:
disable:
- D104
- D201
- D202
- D203
- D211
- D213
Loading