forked from fastai/fastai
-
Notifications
You must be signed in to change notification settings - Fork 0
/
callbacks.misc.html
100 lines (76 loc) · 5.28 KB
/
callbacks.misc.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
---
title: callbacks.misc
keywords:
sidebar: home_sidebar
---
<!--
#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: docs_src/callbacks.misc.ipynb
# instructions: https://docs.fast.ai/gen_doc_main.html
-->
<div class="container" id="notebook-container">
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="callbacks.misc">callbacks.misc<a class="anchor-link" href="#callbacks.misc">¶</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Miscellaneous callbacks that don't belong to any specific group are to be found here.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="output_wrapper">
<div class="output">
<div class="output_area">
<div class="output_markdown rendered_html output_subarea ">
<h2 id="StopAfterNBatches" class="doc_header"><code>class</code> <code>StopAfterNBatches</code><a href="https://github.com/fastai/fastai/blob/master/fastai/callbacks/misc.py#L5" class="source_link" style="float:right">[source]</a><a class="source_link" data-toggle="collapse" data-target="#StopAfterNBatches-pytest" style="float:right; padding-right:10px">[test]</a></h2><blockquote><p><code>StopAfterNBatches</code>(<strong><code>n_batches</code></strong>:<code>int</code>=<strong><em><code>2</code></em></strong>) :: <a href="/callback.html#Callback"><code>Callback</code></a></p>
</blockquote>
<div class="collapse" id="StopAfterNBatches-pytest"><div class="card card-body pytest_card"><a type="button" data-toggle="collapse" data-target="#StopAfterNBatches-pytest" class="close" aria-label="Close"><span aria-hidden="true">×</span></a><p>Tests found for <code>StopAfterNBatches</code>:</p><ul><li><code>pytest -sv tests/test_callbacks_misc.py::test_stop_after_n_batches</code> <a href="https://github.com/fastai/fastai/blob/master/tests/test_callbacks_misc.py#L22" class="source_link" style="float:right">[source]</a></li></ul><p>To run tests please refer to this <a href="/dev/test.html#quick-guide">guide</a>.</p></div></div><p>Stop training after n batches of the first epoch.</p>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><a href="/callbacks.misc.html#StopAfterNBatches"><code>StopAfterNBatches</code></a></p>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>There could be various uses for this handy callback.</p>
<p>The initial purpose of it was to be able to quickly check memory requirements for a given set of hyperparamaters like <code>bs</code> and <code>size</code>.</p>
<p>Since all the required GPU memory is setup during the first batch of the first epoch <a href="https://docs.fast.ai/tutorial.resources.html#gpu-memory-usage-anatomy">see tutorial</a>, it's enough to run just 1-2 batches to measure whether your hyperparameters are right and won't lead to Out-Of-Memory (OOM) errors. So instead of waiting for minutes or hours to just discover that your <code>bs</code> or <code>size</code> are too large, this callback allows you to do it seconds.</p>
<p>You can deploy it on a specific learner (or fit call) just like with any other callback:</p>
<pre><code>from fastai.callbacks.misc import StopAfterNBatches
[...]
learn = cnn_learner([...])
learn.callbacks.append(StopAfterNBatches(learn, n_batches=2))
learn.fit_one_cycle(3, max_lr=1e-2)</code></pre>
<p>and it'll either fit into the existing memory or it'll immediately fail with OOM error. You may want to add <a href="https://github.com/stas00/ipyexperiments/">ipyexperiments</a> to show you the memory usage, including the peak usage.</p>
<p>This is good, but it's cumbersome since you have to change the notebook source code and often you will have multiple learners and fit calls in the same notebook, so here is how to do it globally by placing the following code somewhere on top of your notebook and leaving the rest of your notebook unmodified:</p>
<pre><code>from fastai.callbacks.misc import StopAfterNBatches
# True turns the speedup on, False return to normal behavior
tune = True
#tune = False
if tune:
defaults.silent = True # don't print the fit metric headers
defaults.extra_callbacks = [StopAfterNBatches(n_batches=2)]
else:
defaults.silent = False
defaults.extra_callbacks = None</code></pre>
<p>When you're done tuning your hyper-parameters, just set <code>tune</code> to <code>False</code> and re-run the notebook to do true fitting.</p>
<p>The setting <code>defaults.silent</code> controls whether <a href="/basic_train.html#fit"><code>fit</code></a> calls print out any output.</p>
<p>Do note that when you run this callback, each fit call will be interrupted resulting in the red colored output - that's just an indication that the normal fit didn't happen, so you shouldn't expect any qualitative results out of it.</p>
</div>
</div>
</div>
</div>