Lecture 1: Intro and How to Read a Paper
- What is Gibbs sampling?
A MCMC technique for estimating many things (integrals, joint probability) of multivariate probability distributions. Thanks in part to JHU's own Don Geman (learned that yesterday). It is commonly used as a statistical inference technique as a randomized alternative to EM.
- What does factorial mean in this context?
Factorial in what number? Only definition is n! in mathematics or informally (of factors). It seems to want to refer to independence. The number of output states and outphttps://en.wikipedia.org/wiki/Gibbs_samplingut probabilities are "of factors" with the inputs? i.e. product of probabilities = independent.
- What does generative mean?
That the model can be run "backwards" to generate images of what the model thinks it saw.
My process
- 1st pass -- write down questions and everything you don't know
- 2nd pass -- fight the good fight
- 3rd pass -- clean up missing knowledge In reality this turns out to be about 1.75 careful readings.
-
If a paper builds off another paper, read that first.
- Ideally, I would have read the up/down training paper
-
Things I read
- Wiki page on RBM -- https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine
- Wiki page on Gibbs sampling -- https://en.wikipedia.org/wiki/Gibbs_sampling
- Write them down, let's look at mine
- https://www.scientificamerican.com/article/a-learning-secret-don-t-take-notes-with-a-laptop/
- http://www.npr.org/2016/04/17/474525392/attention-students-put-your-laptops-away
In the context of papers, I really believe this to be the case. I'm all surface or printed. It doesn't help you read better, just slow down and integrate information before you ask questions or take notes.