forked from fastai/fastai
-
Notifications
You must be signed in to change notification settings - Fork 0
/
utils.ipython.html
143 lines (106 loc) · 6.29 KB
/
utils.ipython.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
---
title: utils.ipython
keywords:
sidebar: home_sidebar
---
<!--
#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: docs_src/utils.ipython.ipynb
# instructions: https://docs.fast.ai/gen_doc_main.html
-->
<div class="container" id="notebook-container">
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="IPython-Utilities">IPython Utilities<a class="anchor-link" href="#IPython-Utilities">¶</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Utilities to help work with ipython/jupyter environment.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>To import from <a href="/utils.ipython.html#utils.ipython"><code>fastai.utils.ipython</code></a> do:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">fastai.utils.ipython</span> <span class="k">import</span> <span class="o">*</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Workarounds-to-the-leaky-ipython-traceback-on-exception">Workarounds to the leaky ipython traceback on exception<a class="anchor-link" href="#Workarounds-to-the-leaky-ipython-traceback-on-exception">¶</a></h2><p>ipython has a feature where it stores tb with all the <code>locals()</code> tied in, which
prevents <code>gc.collect()</code> from freeing those variables and leading to a leakage.</p>
<p>Therefore we cleanse the tb before handing it over to ipython. The 2 ways of doing it are by either using the <a href="/utils.ipython.html#gpu_mem_restore"><code>gpu_mem_restore</code></a> decorator or the <a href="/utils.ipython.html#gpu_mem_restore_ctx"><code>gpu_mem_restore_ctx</code></a> context manager which are described next:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="output_wrapper">
<div class="output">
<div class="output_area">
<div class="output_markdown rendered_html output_subarea ">
<h4 id="gpu_mem_restore" class="doc_header"><code>gpu_mem_restore</code><a href="https://github.com/fastai/fastai/blob/master/fastai/utils/ipython.py#L35" class="source_link" style="float:right">[source]</a><a class="source_link" data-toggle="collapse" data-target="#gpu_mem_restore-pytest" style="float:right; padding-right:10px">[test]</a></h4><blockquote><p><code>gpu_mem_restore</code>(<strong><code>func</code></strong>)</p>
</blockquote>
<div class="collapse" id="gpu_mem_restore-pytest"><div class="card card-body pytest_card"><a type="button" data-toggle="collapse" data-target="#gpu_mem_restore-pytest" class="close" aria-label="Close"><span aria-hidden="true">×</span></a><p>No tests found for <code>gpu_mem_restore</code>. To contribute a test please refer to <a href="/dev/test.html">this guide</a> and <a href="https://forums.fast.ai/t/improving-expanding-functional-tests/32929">this discussion</a>.</p></div></div><p>Reclaim GPU RAM if CUDA out of memory happened, or execution was interrupted</p>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><a href="/utils.ipython.html#gpu_mem_restore"><code>gpu_mem_restore</code></a> is a decorator to be used with any functions that interact with CUDA (top-level is fine)</p>
<ul>
<li>under non-ipython environment it doesn't do anything.</li>
<li>under ipython currently it strips tb by default only for the "CUDA out of memory" exception.</li>
</ul>
<p>The env var <code>FASTAI_TB_CLEAR_FRAMES</code> changes this behavior when run under ipython,
depending on its value:</p>
<ul>
<li>"0": never strip tb (makes it possible to always use <code>%debug</code> magic, but with leaks)</li>
<li>"1": always strip tb (never need to worry about leaks, but <code>%debug</code> won't work)</li>
</ul>
<p>e.g. <code>os.environ['FASTAI_TB_CLEAR_FRAMES']="0"</code> will set it to 0.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="output_wrapper">
<div class="output">
<div class="output_area">
<div class="output_markdown rendered_html output_subarea ">
<h4 id="gpu_mem_restore_ctx" class="doc_header"><code>class</code> <code>gpu_mem_restore_ctx</code><a href="https://github.com/fastai/fastai/blob/master/fastai/utils/ipython.py#L57" class="source_link" style="float:right">[source]</a><a class="source_link" data-toggle="collapse" data-target="#gpu_mem_restore_ctx-pytest" style="float:right; padding-right:10px">[test]</a></h4><blockquote><p><code>gpu_mem_restore_ctx</code>()</p>
</blockquote>
<div class="collapse" id="gpu_mem_restore_ctx-pytest"><div class="card card-body pytest_card"><a type="button" data-toggle="collapse" data-target="#gpu_mem_restore_ctx-pytest" class="close" aria-label="Close"><span aria-hidden="true">×</span></a><p>No tests found for <code>gpu_mem_restore_ctx</code>. To contribute a test please refer to <a href="/dev/test.html">this guide</a> and <a href="https://forums.fast.ai/t/improving-expanding-functional-tests/32929">this discussion</a>.</p></div></div><p>context manager to reclaim RAM if an exception happened under ipython</p>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>if function decorator is not a good option, you can use a context manager instead. For example:</p>
<pre><code>with gpu_mem_restore_ctx():
learn.fit_one_cycle(1,1e-2)</code></pre>
<p>This particular one will clear tb on any exception.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
</div>
</div>