-
Notifications
You must be signed in to change notification settings - Fork 759
/
ReparameterizationKLKLqp.html
222 lines (215 loc) · 10.6 KB
/
ReparameterizationKLKLqp.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="Content-Style-Type" content="text/css" />
<meta name="generator" content="pandoc" />
<title>Edward – ed.ReparameterizationKLKLqp</title>
<!-- Mobile Specific Metas -->
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- FONT -->
<link href="https://fonts.googleapis.com/css?family=Lora:400,400i,700" rel="stylesheet">
<!-- CSS -->
<link rel="stylesheet" href="/css/normalize.css">
<link rel="stylesheet" href="/css/skeleton.css">
<!-- Dynamically resize logo for mobile -->
<style type="text/css">
.logo-width {
width: 100%;
box-sizing: border-box;
margin-bottom: 15%;
}
/* Roughly the point when website is single column */
@media (max-width: 850px) {
.logo-width {
width: 50%;
box-sizing: border-box;
margin-bottom: 5%;
}
}
/* Downgrade API's header styles without explicitly modifying their type. */
div.nine.columns h1 { text-align: left; }
div.nine.columns h1 { font-size: 3.0rem; line-height: 1.25; letter-spacing: -.1rem; }
div.nine.columns h2 { font-size: 2.4rem; line-height: 1.3; letter-spacing: -.1rem; }
div.nine.columns h3 { font-size: 2.4rem; line-height: 1.35; letter-spacing: -.08rem; }
div.nine.columns h4 { font-size: 1.8rem; line-height: 1.5; letter-spacing: -.05rem; }
div.nine.columns h5 { font-size: 1.5rem; line-height: 1.6; letter-spacing: 0; }
@media (min-width: 550px) {
div.nine.columns h1 { font-size: 3.6rem; }
div.nine.columns h2 { font-size: 3.0rem; }
div.nine.columns h3 { font-size: 3.0rem; }
div.nine.columns h4 { font-size: 2.5rem; }
div.nine.columns h5 { font-size: 1.5rem; }
}
</style>
<!-- KaTeX -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.7.1/katex.min.js"></script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.7.1/katex.min.css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.7.1/contrib/auto-render.min.js"></script>
<!-- highlight.js -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.4.0/highlight.min.js"></script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.4.0/styles/github.min.css">
<!-- Favicon -->
<link rel="apple-touch-icon" sizes="57x57" href="/icons/apple-touch-icon-57x57.png">
<link rel="apple-touch-icon" sizes="60x60" href="/icons/apple-touch-icon-60x60.png">
<link rel="apple-touch-icon" sizes="72x72" href="/icons/apple-touch-icon-72x72.png">
<link rel="apple-touch-icon" sizes="76x76" href="/icons/apple-touch-icon-76x76.png">
<link rel="apple-touch-icon" sizes="114x114" href="/icons/apple-touch-icon-114x114.png">
<link rel="apple-touch-icon" sizes="120x120" href="/icons/apple-touch-icon-120x120.png">
<link rel="apple-touch-icon" sizes="144x144" href="/icons/apple-touch-icon-144x144.png">
<link rel="apple-touch-icon" sizes="152x152" href="/icons/apple-touch-icon-152x152.png">
<link rel="apple-touch-icon" sizes="180x180" href="/icons/apple-touch-icon-180x180.png">
<link rel="icon" type="image/png" href="/icons/favicon-32x32.png" sizes="32x32">
<link rel="icon" type="image/png" href="/icons/android-chrome-192x192.png" sizes="192x192">
<link rel="icon" type="image/png" href="/icons/favicon-96x96.png" sizes="96x96">
<link rel="icon" type="image/png" href="/icons/favicon-16x16.png" sizes="16x16">
<link rel="manifest" href="/icons/manifest.json">
<link rel="mask-icon" href="/icons/safari-pinned-tab.svg" color="#5bbad5">
<link rel="shortcut icon" href="/icons/favicon.ico">
<meta name="msapplication-TileColor" content="#da532c">
<meta name="msapplication-TileImage" content="/icons/mstile-144x144.png">
<meta name="msapplication-config" content="/icons/browserconfig.xml">
<meta name="theme-color" content="#ffffff">
</head>
<body>
<div class="container">
<div class="row" style="margin-top: 5%">
<div class="three columns">
<h1><a href="/">Edward</a></h1>
<a href="/">
<center>
<img src="/images/edward.png" class="logo-width" alt="Edward" />
</center>
</a>
<a class="button u-full-width" href="/api/">API</a>
<hr style="margin-top: 1rem; margin-bottom: 1.5rem;"/>
<a class="button u-full-width" href="/api/reference">Reference</a>
<a class="button u-full-width" href="/api/ed/criticisms">ed.criticisms</a>
<a class="button u-full-width" href="/api/ed/inferences">ed.inferences</a>
<a class="button u-full-width" href="/api/ed/models">ed.models</a>
<a class="button u-full-width" href="/api/ed/util">ed.util</a>
<a class="button u-full-width" href="/api/observations">observations</a>
<div class="row" style="padding-bottom: 5%"> </div>
<a class="button2 u-pull-right" style="padding-right:10%"
href="https://github.com/blei-lab/edward">
<span style="vertical-align:middle;">Github</span>
<img src="/images/github-mark.svg" style="vertical-align:middle;"
alt="Edward on Github" />
</a>
<div class="row" style="padding-bottom: 5%"> </div>
</div>
<div class="nine columns">
<div itemscope itemtype="http://developers.google.com/ReferenceObject">
<meta itemprop="name" content="ed.ReparameterizationKLKLqp" />
<meta itemprop="property" content="__init__"/>
<meta itemprop="property" content="build_loss_and_gradients"/>
<meta itemprop="property" content="finalize"/>
<meta itemprop="property" content="initialize"/>
<meta itemprop="property" content="print_progress"/>
<meta itemprop="property" content="run"/>
<meta itemprop="property" content="update"/>
</div>
<h1 id="ed.reparameterizationklklqp">ed.ReparameterizationKLKLqp</h1>
<h2 id="class-reparameterizationklklqp">Class <code>ReparameterizationKLKLqp</code></h2>
<p>Inherits From: <a href="../ed/VariationalInference"><code>VariationalInference</code></a></p>
<h3 id="aliases">Aliases:</h3>
<ul>
<li>Class <code>ed.ReparameterizationKLKLqp</code></li>
<li>Class <code>ed.inferences.ReparameterizationKLKLqp</code></li>
</ul>
<p>Defined in <a href="https://github.com/blei-lab/edward/tree/master/edward/inferences/klqp.py"><code>edward/inferences/klqp.py</code></a>.</p>
<p>Variational inference with the KL divergence</p>
<p><span class="math inline">\(\text{KL}( q(z; \lambda) \| p(z \mid x) ).\)</span></p>
<p>This class minimizes the objective using the reparameterization gradient and an analytic KL term.</p>
<p>The objective function also adds to itself a summation over all tensors in the <code>REGULARIZATION_LOSSES</code> collection.</p>
<h2 id="methods">Methods</h2>
<h3 id="__init__">
<code><strong>init</strong></code>
</h3>
<pre class="python"><code>__init__(
latent_vars=None,
data=None
)</code></pre>
<p>Create an inference algorithm.</p>
<h4 id="args">Args:</h4>
<ul>
<li><b><code>latent_vars</code></b>: list of RandomVariable or dict of RandomVariable to RandomVariable. Collection of random variables to perform inference on. If list, each random variable will be implictly optimized using a <code>Normal</code> random variable that is defined internally with a free parameter per location and scale and is initialized using standard normal draws. The random variables to approximate must be continuous.</li>
</ul>
<h3 id="build_loss_and_gradients">
<code>build_loss_and_gradients</code>
</h3>
<pre class="python"><code>build_loss_and_gradients(var_list)</code></pre>
<h3 id="finalize">
<code>finalize</code>
</h3>
<pre class="python"><code>finalize()</code></pre>
<p>Function to call after convergence.</p>
<h3 id="initialize">
<code>initialize</code>
</h3>
<pre class="python"><code>initialize(
n_samples=1,
kl_scaling=None,
*args,
**kwargs
)</code></pre>
<p>Initialize inference algorithm. It initializes hyperparameters and builds ops for the algorithm’s computation graph.</p>
<h4 id="args-1">Args:</h4>
<ul>
<li><b><code>n_samples</code></b>: int. Number of samples from variational model for calculating stochastic gradients.</li>
<li><p><b><code>kl_scaling</code></b>: dict of RandomVariable to tf.Tensor. Provides option to scale terms when using ELBO with KL divergence. If the KL divergence terms are</p>
<p><span class="math inline">\(\alpha_p \mathbb{E}_{q(z\mid x, \lambda)} [ \log q(z\mid x, \lambda) - \log p(z)],\)</span></p>
<p>then pass {<span class="math inline">\(p(z)\)</span>: <span class="math inline">\(\alpha_p\)</span>} as <code>kl_scaling</code>, where <span class="math inline">\(\alpha_p\)</span> is a tensor. Its shape must be broadcastable; it is multiplied element-wise to the batchwise KL terms.</p></li>
</ul>
<h3 id="print_progress">
<code>print_progress</code>
</h3>
<pre class="python"><code>print_progress(info_dict)</code></pre>
<p>Print progress to output.</p>
<h3 id="run">
<code>run</code>
</h3>
<pre class="python"><code>run(
variables=None,
use_coordinator=True,
*args,
**kwargs
)</code></pre>
<p>A simple wrapper to run inference.</p>
<ol type="1">
<li>Initialize algorithm via <code>initialize</code>.</li>
<li>(Optional) Build a TensorFlow summary writer for TensorBoard.</li>
<li>(Optional) Initialize TensorFlow variables.</li>
<li>(Optional) Start queue runners.</li>
<li>Run <code>update</code> for <code>self.n_iter</code> iterations.</li>
<li>While running, <code>print_progress</code>.</li>
<li>Finalize algorithm via <code>finalize</code>.</li>
<li>(Optional) Stop queue runners.</li>
</ol>
<p>To customize the way inference is run, run these steps individually.</p>
<h4 id="args-2">Args:</h4>
<ul>
<li><b><code>variables</code></b>: list. A list of TensorFlow variables to initialize during inference. Default is to initialize all variables (this includes reinitializing variables that were already initialized). To avoid initializing any variables, pass in an empty list.</li>
<li><b><code>use_coordinator</code></b>: bool. Whether to start and stop queue runners during inference using a TensorFlow coordinator. For example, queue runners are necessary for batch training with file readers. *args, **kwargs: Passed into <code>initialize</code>.</li>
</ul>
<h3 id="update">
<code>update</code>
</h3>
<pre class="python"><code>update(feed_dict=None)</code></pre>
<p>Run one iteration of optimization.</p>
<h4 id="args-3">Args:</h4>
<ul>
<li><b><code>feed_dict</code></b>: dict. Feed dictionary for a TensorFlow session run. It is used to feed placeholders that are not fed during initialization.</li>
</ul>
<h4 id="returns">Returns:</h4>
<p>dict. Dictionary of algorithm-specific information. In this case, the loss function value after one iteration.</p>
</div>
</div>
<div class="row" style="padding-bottom: 25%"> </div>
</div>
<script>
hljs.initHighlightingOnLoad();
renderMathInElement(document.body);
</script>
</body>
</html>