New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid keeping reference to function nodes in CupyMemoryProfileHook #4300

Merged
merged 1 commit into from Feb 13, 2018

Conversation

Projects
None yet
3 participants
@niboshi
Copy link
Member

niboshi commented Feb 7, 2018

CupyMemoryProfileHook keeps FunctionNode references in the call history, thus the memory held by these functions are never released within the hook scope.

That can unnecessarily cause OOM error if multiple loops are measured at once.

hook = function_hooks.CupyMemoryProfileHook()
with hook:
    train_loop1()
    train_loop2()
    train_loop3()  # memory consumption is accumulated
hook.print_report()

(Note: the numbers in the profiling result is not affected by accumulation)

This fix is to keep function names in the call history instead of function objects.
This is backward-incompatible, because the content of call history (which is documented) is altered.

@niboshi niboshi force-pushed the niboshi:release-func-cupy-memory-profile-hook branch from 9d229fc to b182c70 Feb 7, 2018

@beam2d

This comment has been minimized.

Copy link
Member

beam2d commented Feb 13, 2018

LGTM

@beam2d beam2d merged commit baf8958 into chainer:master Feb 13, 2018

3 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage decreased (-0.0005%) to 90.129%
Details

@niboshi niboshi deleted the niboshi:release-func-cupy-memory-profile-hook branch Feb 13, 2018

@kmaehashi kmaehashi added this to the v4.0.0b4 milestone Feb 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment