Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix max_depth recursion crash in AsynchronousLoader #191

Merged
merged 5 commits into from
Sep 11, 2020
Merged

Fix max_depth recursion crash in AsynchronousLoader #191

merged 5 commits into from
Sep 11, 2020

Commits on Sep 8, 2020

  1. Fix max_depth recursion crash in AsynchronousLoader 

    Some people use dict samples with strings, numpy array etc. They are returned from dataloader and are passed to model. Later those are parsed/replaced/removed inside of step/forward function.

Example of such dict (I use it):
`{'image': tensor([WHATEVER]), 'path': [WHATEVER_STRING], 'target': tensor([WHATEVER_NUMPY]), 'meta': {'PatientAge': tensor([WHATEVER]), 'PatientSex': tensor([WHATEVER]), 'StudyDate': tensor([WHATEVER])}, 'mask': tensor([WHATEVER]), 'bboxes': [[tensor([WHATEVER]), tensor([WHATEVER]), tensor([WHATEVER])]]}`

Sadly, async loader was crashing as max depth of recursion was reached (e.g. when it meets string, it just infinitely goes to else in original AsyncLoader). 

I went to torch.utils.data.default_collate (as it's said that code is based on it), took few lines from there and it works now.

So it will no longer crash at named_tuples, dict-like objects and strings. Those will all be processed.
    @DaGuT
    DaGuT committed Sep 8, 2020
    Configuration menu
    Copy the full SHA
    8e4f9f7 View commit details
    Browse the repository at this point in the history

  2. made "else" as it was before

    Александр Монголин committed Sep 8, 2020
    Configuration menu
    Copy the full SHA
    130d397 View commit details
    Browse the repository at this point in the history

  3. Revert "made "else" as it was before" 

    This reverts commit 130d397.
    Александр Монголин committed Sep 8, 2020
    Configuration menu
    Copy the full SHA
    4f539b2 View commit details
    Browse the repository at this point in the history

  4. autopep8 format

    Александр Монголин committed Sep 8, 2020
    Configuration menu
    Copy the full SHA
    2ba3bd1 View commit details
    Browse the repository at this point in the history

  5. moved np_str_obj_array_pattern definition to init

    Александр Монголин committed Sep 8, 2020
    Configuration menu
    Copy the full SHA
    d04a41e View commit details
    Browse the repository at this point in the history