New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] Make it easier to add an ad hoc log prob term? #1454

Open
fritzo opened this Issue Oct 16, 2018 · 5 comments

Comments

Projects
None yet
3 participants
@fritzo
Copy link
Member

fritzo commented Oct 16, 2018

Current solutions to adding a custom log prob term include (1) subclassing ELBO #1059 #958, and (2) using a bogus statement

pyro.sample(name, dist.Bernoulli(logits=custom_term), obs=torch.tensor(1.))

By contrast STAN makes it easy to add custom loss terms.

Should we add a helper that adds a custom loss term, effectively adding a non-normalized log-prob via a hidden observe statement?

@eb8680

This comment has been minimized.

Copy link
Collaborator

eb8680 commented Oct 16, 2018

Other PPLs have a built-in factor helper. See https://github.com/uber/pyro/blob/dev/examples/rsa/search_inference.py#L33

We should also add a tutorial on writing custom losses as suggested in #958

@fritzo

This comment has been minimized.

Copy link
Member

fritzo commented Oct 17, 2018

@eb8680 great! What do you think about moving factor into pyro.primitives, adding docs, and using it in a couple examples (e.g. SS-VAE)?

@eb8680

This comment has been minimized.

Copy link
Collaborator

eb8680 commented Oct 17, 2018

What do you think about moving factor into pyro.primitives, adding docs, and using it in a couple examples (e.g. SS-VAE)?

We should write a custom FactorDistribution with the correct shape semantics and use that to implement factor. I actually don't think it's a good idiom and should be discouraged unless really necessary, so I'd rather not use it in existing examples, but maybe we could add a separate factor graph example?

@fritzo

This comment has been minimized.

Copy link
Member

fritzo commented Oct 18, 2018

Agreed, I'm not convinced it is a good idiom.

@max-ilse

This comment has been minimized.

Copy link

max-ilse commented Oct 23, 2018

Hi everybody,

I recently started using Pyro. Thanks for the great SS-VAE tutorial. It really helped me getting up to speed.

For an extension of the SS-VAE, I want to add an additional (weighted) MMD regulariser to my elbo, since I need samples from my variational posterior I thought about adding a bogus statement to my model function, e.g.

pyro.sample("MMD", dist.Bernoulli(logits=self.mmd_multiplier*self.calculate_mmd(z, y)), obs=torch.tensor(1.).cuda())

Or is there a better option?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment