Hamiltonian Monte Carlo with Dual Averaging #728

emilemathieu · 2017-08-03T12:00:25Z

Hello to all !

This PR partially solve this issue #541 which is about implementing NUTS. As proposed in this topic, starting with the Dual Averaging is a good start.

The implemented inference method is in edward/inferences/hmcda.py and the corresponding test in tests/test-inferences/test_hmcda.py.

I hope you'll appreciate the PR ! :)

emilemathieu · 2017-08-06T14:17:32Z

@dustinvtran all checks have passed, could this PR be merged ?

dustinvtran

My sincere apologies for the delay. I finally had some time to re-read the NUTS paper and go through your implementation. Great job! (especially on the dynamic leapfrog implementation)

I only have minor suggestions below.

dustinvtran · 2017-08-11T03:09:33Z

edward/inferences/hmcda.py

+    step_size = self.find_good_eps()
+    sess = get_session()
+    init_op = tf.global_variables_initializer()
+    sess.run(init_op)


The initialize op shouldn't be needed inside inference.initialize(). It's called within the run() method or alternatively must be called manually after you call inference.initialize() on the algorithm.

Without this, I have: Attempting to use uninitialized value Variable.
I can't making it work without :/

dustinvtran · 2017-08-11T03:10:05Z

edward/inferences/hmcda.py

+    The updates assume each Empirical random variable is directly
+    parameterized by ``tf.Variable``s.
+    """
+


remove emptyline

dustinvtran · 2017-08-11T03:11:05Z

edward/inferences/hmcda.py

+    """Simulate Hamiltonian dynamics using a numerical integrator.
+    Correct for the integrator's discretization error using an
+    acceptance ratio. The initial value of espilon is heuristically chosen
+    with Algorithm 4


epsilon, Algorithm 4.

dustinvtran · 2017-08-11T03:11:13Z

edward/inferences/hmcda.py

+    """
+    self.scope_iter = 0  # a convenient counter for log joint calculations
+
+    # Find intial epsilon


dustinvtran · 2017-08-11T03:11:23Z

edward/inferences/hmcda.py

+    Parameters
+    ----------
+    n_adapt : float
+      Number of samples with adaption for epsilon


adaptation

dustinvtran · 2017-08-11T03:14:17Z

edward/inferences/hmcda.py

+    # Accept or reject sample.
+    u = Uniform().sample()
+    alpha = tf.minimum(1.0, tf.exp(ratio))
+    accept = u < alpha


Comparing with tf.log(u) < ratio should be more numerically stable than checking on the PDF scale.

dustinvtran · 2017-08-11T03:15:47Z

edward/inferences/hmcda.py

+    assign_ops.append(self.n_accept.assign_add(tf.where(accept, 1, 0)))
+    return tf.group(*assign_ops)
+
+  def do_not_adapt_step_size(self, alpha):


for methods built for internal implementation, prepend the name with _.

dustinvtran · 2017-08-11T03:39:25Z

edward/inferences/hmcda.py

+  def do_not_adapt_step_size(self, alpha):
+    # Do not adapt step size but assign last running averaged epsilon to epsilon
+    assign_ops = []
+    assign_ops.append(self.H_B.assign_add(0.0).op)


Is there a reason you add 0 to these variables? What happens if we don't use assign ops for them?

The tf.cond arguments; true_fn and false_fnmust have the same type of outputs, which is a list of ops in our case.
We could also do assign_ops.append(tf.assign(self.H_B, self.H_B).op).
I would be happy to receive any better idea.

emilemathieu · 2017-08-11T10:41:05Z

Thanks for your feedbacks @dustinvtran !

I've fixed all your suggestions but the initialization issue. I dot know how to fix that.

dustinvtran · 2017-08-11T14:26:28Z

edward/inferences/hmcda.py

+      Latent variable keys to samples.
+    """
+    self.scope_iter += 1
+    scope = 'inference_' + str(id(self)) + '/' + str(self.scope_iter)


Note we no longer use scope_iter and str(id(self)) implementation for scopes. See the latest versions of hmc.py and sghmc.py where we use scope = tf.get_default_graph().unique_name("inference").

dustinvtran · 2017-08-11T14:30:16Z

edward/inferences/hmcda.py

+    assign_ops.append(self.n_accept.assign_add(tf.where(accept, 1, 0)))
+    return tf.group(*assign_ops)
+
+  def _do_not__adapt_step_size(self, alpha):


_do_not_adapt_step_size

dustinvtran · 2017-08-11T14:34:37Z

Without this, I have: Attempting to use uninitialized value Variable.
I can't making it work without :/

Do you know which tf.Variable causes this to break? For example, is there a reason you wrote epsilon = tf.Variable(1.0, trainable=False) instead of epsilon = tf.constant(1.0)?

emilemathieu · 2017-08-11T15:22:08Z

I have updated to epsilon = tf.constant(1.0) but it still break without the initialization.

Could it come from the empirical variables self.latent_vars which are needed in find_good_eps ?

dustinvtran · 2017-08-14T01:50:02Z

Upon further investigation, the issue is data-dependent initialization. The tf.Variable epsilon depends on tf.Variables in the model and approximating families. This means that the variables have to go through separate session calls to the init ops so that the model / approximating families are initialized first. Related: tensorflow/tensorflow#4920

Not sure how to fix this just yet.

dustinvtran · 2017-08-14T01:50:14Z

tests/test-inferences/test_hmcda.py

+      # analytic solution: N(loc=0.0, scale=\sqrt{1/51}=0.140)
+      inference = ed.HMCDA({mu: qmu}, data={x: x_data})
+      inference.run(n_adapt=1000)
+      print(qmu.mean().eval())


remove print statements in test

dustinvtran · 2017-08-14T01:50:31Z

edward/inferences/hmcda.py

+  k = tf.constant(0)
+
+  def while_condition(k, v_z_new, v_r_new, grad_log_joint):
+     # Stop when k < n_steps


always use two space indent, including inside internal functions

emilemathieu · 2017-08-14T11:48:17Z

Prints and indents fixed.

Nice spotted for the initialization issue ! What about the workaround proposed by yaroslavvb (2sd comment) ?

dustinvtran · 2017-08-14T21:51:54Z

I hesitate because it (1) requires users to use a custom initialization scheme; and (2) depends on a tf.contrib library which can be unstable in its API / internal implementation. That said, I think it's worth trying as it may be the only viable solution; we can tweak/relax it later.

When replacing the init op inside inference.run() with the following, I wasn't able to get it to work. You're welcome to tweak it so it does work.

    if variables is None:
      # Force variables to be initialized after any variables they depend on.
      from tensorflow.contrib import graph_editor as ge
      def make_safe_initializer(var):
        """Returns initializer op that only runs for uninitialized ops."""
        return tf.cond(tf.is_variable_initialized(var),
                       tf.no_op,
                       lambda: tf.assign(var, var.initial_value).op,
                       name="safe_init_" + var.op.name).op

      safe_initializers = {}
      for v in tf.global_variables():
        safe_initializers[v.op.name] = make_safe_initializer(v)

      g = tf.get_default_graph()
      for v in tf.global_variables():
        var_name = v.op.name
        var_cache = g.get_operation_by_name(var_name + "/read")
        ge.reroute.add_control_inputs(var_cache, [safe_initializers[var_name]])

      init = tf.group(*safe_initializers.values())
    else:
      init = tf.variables_initializer(variables)

Emile Mathieu added 3 commits August 3, 2017 12:49

HMC Dual Averaging inference method

1e971d9

Test for HMCDA

cb4e119

add HMCDA in init files

abed004

emilemathieu mentioned this pull request Aug 3, 2017

No U-Turn Sampler #541

Open

Emile Mathieu added 2 commits August 6, 2017 14:42

fix HMCDA pep8 non compliance

57df441

remove non-ASCII characters

698cef6

dustinvtran reviewed Aug 11, 2017

View reviewed changes

suggestions fix

1cdf507

dustinvtran reviewed Aug 11, 2017

View reviewed changes

typo, scope_iter and scope fixes

8a031e6

emilemathieu force-pushed the master branch from cc3153f to 8a031e6 Compare August 11, 2017 15:37

dustinvtran reviewed Aug 14, 2017

View reviewed changes

Emile Mathieu added 2 commits August 14, 2017 11:57

remove prints

e5fba9f

indents fix

57d53cd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hamiltonian Monte Carlo with Dual Averaging #728

Hamiltonian Monte Carlo with Dual Averaging #728

emilemathieu commented Aug 3, 2017

emilemathieu commented Aug 6, 2017

dustinvtran left a comment

dustinvtran Aug 11, 2017

emilemathieu Aug 11, 2017

dustinvtran Aug 11, 2017

dustinvtran Aug 11, 2017

dustinvtran Aug 11, 2017

dustinvtran Aug 11, 2017

dustinvtran Aug 11, 2017

dustinvtran Aug 11, 2017

dustinvtran Aug 11, 2017

emilemathieu Aug 11, 2017 •

edited

emilemathieu commented Aug 11, 2017

dustinvtran Aug 11, 2017

dustinvtran Aug 11, 2017

dustinvtran commented Aug 11, 2017

emilemathieu commented Aug 11, 2017

dustinvtran commented Aug 14, 2017

dustinvtran Aug 14, 2017

dustinvtran Aug 14, 2017

emilemathieu commented Aug 14, 2017

dustinvtran commented Aug 14, 2017 •

edited

Hamiltonian Monte Carlo with Dual Averaging #728

Are you sure you want to change the base?

Hamiltonian Monte Carlo with Dual Averaging #728

Conversation

emilemathieu commented Aug 3, 2017

emilemathieu commented Aug 6, 2017

dustinvtran left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emilemathieu Aug 11, 2017 • edited

Choose a reason for hiding this comment

emilemathieu commented Aug 11, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dustinvtran commented Aug 11, 2017

emilemathieu commented Aug 11, 2017

dustinvtran commented Aug 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emilemathieu commented Aug 14, 2017

dustinvtran commented Aug 14, 2017 • edited

emilemathieu Aug 11, 2017 •

edited

dustinvtran commented Aug 14, 2017 •

edited