Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add mpi4py support, and WISE checkpointing #663

Merged
merged 18 commits into from
Nov 27, 2020
Merged

Add mpi4py support, and WISE checkpointing #663

merged 18 commits into from
Nov 27, 2020

Commits on Nov 25, 2020

  1. add MPI wrapper for runbrick

    @dstndstn
    dstndstn committed Nov 25, 2020
    Configuration menu
    Copy the full SHA
    5cdd878 View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2020

  1. try adding wise_checkpoint. MPI map with timeout doesn't work the way… 

    … I thought it did, so this doesn't work
    @dstndstn
    dstndstn committed Nov 26, 2020
    Configuration menu
    Copy the full SHA
    7934dea View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2020

  1. implement wait-with-timeout for MPI futures

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    dff69ef View commit details
    Browse the repository at this point in the history

  2. wise checkpoints using mpi timed-out

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    6e6e734 View commit details
    Browse the repository at this point in the history

  3. add mpi-runbrick driver script

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    71be984 View commit details
    Browse the repository at this point in the history

  4. mpi script: different way of doing round-robin (cray mpich doesn't su… 

    …pport srun's cyclic distribution)
    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    bd290a4 View commit details
    Browse the repository at this point in the history

  5. report memory use in Time reports

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    03d3be2 View commit details
    Browse the repository at this point in the history

  6. minor tweak to checkpoint-waiting

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    8d1ba3b View commit details
    Browse the repository at this point in the history

  7. comment and pull out mpi4py-to-multiprocessing adapter classes

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    1045c22 View commit details
    Browse the repository at this point in the history

  8. add some comments to runbrick-mpi script

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    256d286 View commit details
    Browse the repository at this point in the history

  9. shifter: switch to mpi4py master version

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    1729d10 View commit details
    Browse the repository at this point in the history

  10. docker: mpi4py master version; mpi4py branch of legacypipe; vanilla qdo

    Dustin Lang committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    4f08af9 View commit details
    Browse the repository at this point in the history

  11. try fixing matplotlib config dir complaint... does not work

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    7045194 View commit details
    Browse the repository at this point in the history

  12. copy config and cache dirs to /tmp

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    ccbd0ed View commit details
    Browse the repository at this point in the history

  13. fix permissions on astropy cache dir

    Dustin Lang committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    60a10cd View commit details
    Browse the repository at this point in the history

  14. Merge branch 'mpi4py' of github.com:legacysurvey/legacypipe into mpi4py

    Dustin Lang committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    a6f1f94 View commit details
    Browse the repository at this point in the history

  15. update runbrick-mpi script for shifter

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    bbc9969 View commit details
    Browse the repository at this point in the history

  16. print how many jobs remaining

    @dstndstn
    dstndstn committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    60ce7b1 View commit details
    Browse the repository at this point in the history