Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

First attempt to use more gpus #339

Merged
merged 86 commits into from
Jan 8, 2020
Merged

First attempt to use more gpus #339

merged 86 commits into from
Jan 8, 2020

Commits on Feb 13, 2019

  1. First attempt to use more gpus

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    be980b2 View commit details
    Browse the repository at this point in the history

  2. Don't mess with arch and gpu inside spawned thread

    @tfarago
    Matthias Vogelgesang authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    3f865d7 View commit details
    Browse the repository at this point in the history

  3. Support more outputs

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    6d005d8 View commit details
    Browse the repository at this point in the history

  4. Enable specifying output's num_dims in constructor

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    4573273 View commit details
    Browse the repository at this point in the history

  5. Simplify multi-GPU output

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    22ef1b0 View commit details
    Browse the repository at this point in the history

  6. InjectProcess: Add scheduler to constructor

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    42aca1b View commit details
    Browse the repository at this point in the history

  7. Add universal reconstructor prototype

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    266b3a8 View commit details
    Browse the repository at this point in the history

  8. Write volume asynchronously

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    f755556 View commit details
    Browse the repository at this point in the history

  9. Create reconstructors when coroutine starts

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    d253b88 View commit details
    Browse the repository at this point in the history

  10. Delete reconstructors once they are finished

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    be30524 View commit details
    Browse the repository at this point in the history

  11. InjectProcess: Don't require new inputs

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    268c1d5 View commit details
    Browse the repository at this point in the history

  12. Convert dark and flat to float32

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    4cb760a View commit details
    Browse the repository at this point in the history

  13. Optimie projection processing 

    don't insert dark and flat all the time and convert to float only the
cropped projection.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    44d97af View commit details
    Browse the repository at this point in the history

  14. Enable starting the graph immediately

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    d6cff85 View commit details
    Browse the repository at this point in the history

  15. Reuse resources 

    in order to speed up start of the processing when new reconstructors are
created.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    1f369aa View commit details
    Browse the repository at this point in the history

  16. Start with cleanup

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    8d4030e View commit details
    Browse the repository at this point in the history

  17. Add a simple online reconstruction addon

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    e181587 View commit details
    Browse the repository at this point in the history

  18. Fix center-position-x optimization

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    e080a93 View commit details
    Browse the repository at this point in the history

  19. Optimize manager coroutine 

    make sure the args are up-to-date and enable waiting for events which will be
useful when average flats and darks will need to be computed before the manager
will start.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    2b3f99b View commit details
    Browse the repository at this point in the history

  20. backproject: Cosmetic changes

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    c514b30 View commit details
    Browse the repository at this point in the history

  21. Make deepcopy and option in the queue coroutine

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    24fa683 View commit details
    Browse the repository at this point in the history

  22. manager: fix number of reconstructors

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    de16daa View commit details
    Browse the repository at this point in the history

  23. reco addon: Enable dark and flat processing

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    53183db View commit details
    Browse the repository at this point in the history

  24. writer addon: Don't make deepcopy

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    88bfa15 View commit details
    Browse the repository at this point in the history

  25. Don't rely on 0-based regions from tofu

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    9042472 View commit details
    Browse the repository at this point in the history

  26. manager: Wait for init thread if block is True

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    3c18440 View commit details
    Browse the repository at this point in the history

  27. Enbable delayed execution

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    cfccbc8 View commit details
    Browse the repository at this point in the history

  28. Enable delayed execution in online reco

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    dcddc0d View commit details
    Browse the repository at this point in the history

  29. InjectProcess: enable data copying

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    42a4d89 View commit details
    Browse the repository at this point in the history

  30. Enable sleep time specification

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    6f44ed6 View commit details
    Browse the repository at this point in the history

  31. Enable regions specification

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    0aeba48 View commit details
    Browse the repository at this point in the history

  32. Manager constructor takes normalization images 

    becase if its coroutine waits for events which mean the setting of dark and flat
has been done than the coroutine would start with dark and flat=None and there
would be no way of changing it.
    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    8c28e64 View commit details
    Browse the repository at this point in the history

  33. Adjust online reco for a new manger coroutine

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    a569f84 View commit details
    Browse the repository at this point in the history

  34. Prepare backprojectors in one thread

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    80b97cf View commit details
    Browse the repository at this point in the history

  35. manager: cleanup the coroutine

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    d672030 View commit details
    Browse the repository at this point in the history

  36. manager: Fix gpu index

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    da2db1b View commit details
    Browse the repository at this point in the history

  37. manager: Enable more threads per GPU

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    2bc29e2 View commit details
    Browse the repository at this point in the history

  38. queue coroutine: Make sure consumer stops

    @MarcusZuber @tfarago
    MarcusZuber authored and tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    3e77ab6 View commit details
    Browse the repository at this point in the history

  39. manager: Use numpy array to store projections 

    Otherwise there is a huge memory leak.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    85be134 View commit details
    Browse the repository at this point in the history

  40. manager: Enable parallel slice writing

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    e325a88 View commit details
    Browse the repository at this point in the history

  41. online: Add walker

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    0cfbaf7 View commit details
    Browse the repository at this point in the history

  42. online: Use numpy to store normalization images

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    0bbef78 View commit details
    Browse the repository at this point in the history

  43. online: clean up averaging coroutine

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    60bfa31 View commit details
    Browse the repository at this point in the history

  44. online: save the flats and darks correctly

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    c6b269e View commit details
    Browse the repository at this point in the history

  45. online: Enable slice writing

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    dc6131b View commit details
    Browse the repository at this point in the history

  46. reco: Wait for event before slice download starts

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    04964af View commit details
    Browse the repository at this point in the history

  47. manager: Rework synchronization

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    8d7aaa5 View commit details
    Browse the repository at this point in the history

  48. manager: Allow execution abort

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    3d94736 View commit details
    Browse the repository at this point in the history

  49. manager: raname _in_index 

    to _num_received_projections which actually represents what the variable means.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    6fc5582 View commit details
    Browse the repository at this point in the history

  50. args: Add region_overlap 

    because phase retrieval requires it, otherwise there are visible artifacts on
the region boundaries.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    7d0a34b View commit details
    Browse the repository at this point in the history

  51. Normalize projections only if supported 

    by the experiment, i.e. darks and flats acquisitions are present.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    58a8d11 View commit details
    Browse the repository at this point in the history

  52. Use tofu's CTGeometry to copy projection region

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    7bc13e6 View commit details
    Browse the repository at this point in the history

  53. Enable more batches

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    2b31c8a View commit details
    Browse the repository at this point in the history

  54. storage: Check for non-None im_writer

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    2dd83f3 View commit details
    Browse the repository at this point in the history

  55. reco: Enable slice directory specification

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    5eb46a4 View commit details
    Browse the repository at this point in the history

  56. reco: Enable manager to receive more projections

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    b536898 View commit details
    Browse the repository at this point in the history

  57. manager: log number of received projections 

    if not enough come.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    2399444 View commit details
    Browse the repository at this point in the history

  58. Add parameter finding

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    3c54f38 View commit details
    Browse the repository at this point in the history

  59. Make z parameter setting safer 

    by checking the choices from tofu.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    f5c8bfb View commit details
    Browse the repository at this point in the history

  60. Rename center_* to center_position_*

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    4ad839a View commit details
    Browse the repository at this point in the history

  61. find_parameter: don't force splittin policy 'one'

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    af40e3a View commit details
    Browse the repository at this point in the history

  62. find_parameter: generalize setting found parameter

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    cc05850 View commit details
    Browse the repository at this point in the history

  63. find_parameter: Enable maximization

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    cd4f314 View commit details
    Browse the repository at this point in the history

  64. Document find_parameter

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    8ebed28 View commit details
    Browse the repository at this point in the history

  65. Add z_parameters and slice_metrics attributes 

    to UniversalBackprojectArgs for user convenience.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    4e29e52 View commit details
    Browse the repository at this point in the history

  66. Rename Universal to General 

    to unify naming with ufo-filters.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    c7d18af View commit details
    Browse the repository at this point in the history

  67. find_parameter: make value storing optional

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    88b54ae View commit details
    Browse the repository at this point in the history

  68. manager: enable gpus specification

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    25e2909 View commit details
    Browse the repository at this point in the history

  69. Enable finding of more parameters at once

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    38d1dd2 View commit details
    Browse the repository at this point in the history

  70. Don't flip the metric by default

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    6cbd4d5 View commit details
    Browse the repository at this point in the history

  71. Use straightforward sag instead of msag

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    640c9c4 View commit details
    Browse the repository at this point in the history

  72. imageprocessing: add filter_low_frequencies

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    a8c705b View commit details
    Browse the repository at this point in the history

  73. Add fwhm to find_parameters

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    f90379d View commit details
    Browse the repository at this point in the history

  74. Enable input specification in find_parameter

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    a181273 View commit details
    Browse the repository at this point in the history

  75. reco addon: Use first normalization image

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    ccf5d3d View commit details
    Browse the repository at this point in the history

  76. Enable OutputTask to return NULL

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    42bc524 View commit details
    Browse the repository at this point in the history

  77. Make manager abortable

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    bcd9686 View commit details
    Browse the repository at this point in the history

  78. Fix flake8

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    3544207 View commit details
    Browse the repository at this point in the history

  79. Add dummy ImagingExperiment

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    e7326bd View commit details
    Browse the repository at this point in the history

  80. Simplify normalization computation in online reco

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    c8b09ba View commit details
    Browse the repository at this point in the history

  81. ufo: Add FlatCorrect class

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    ec90dcd View commit details
    Browse the repository at this point in the history

  82. Online reco: don't broadcast result if unnecessary 

    i.e. there are no slice consumers.
    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    8784ec1 View commit details
    Browse the repository at this point in the history

  83. Don't require width and height by reco args

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    d414b3e View commit details
    Browse the repository at this point in the history

  84. Allow projection crop disabling

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    4c4a048 View commit details
    Browse the repository at this point in the history

  85. Add number of projections info to reco manager

    @tfarago
    tfarago committed Feb 13, 2019
    Configuration menu
    Copy the full SHA
    acad4df View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2019

  1. Simplify source creation by general reco graph

    @tfarago
    tfarago committed Mar 4, 2019
    Configuration menu
    Copy the full SHA
    9ea89fb View commit details
    Browse the repository at this point in the history