Writeup of posterior properties #2699
Conversation
|
|
||
| The updating from the prior :math:`p(\psi)=N\left(\mu_\psi,\Sigma_\psi\right)` | ||
| to the posterior :math:`p(\psi|d)=N\left(\mu_{\psi|d},\Sigma_{\psi|d}\right)`, assimilating measurements :math:`d`, | ||
| linear in :math:`\psi` is performed by the Kalman methods by employing the following equations |
There was a problem hiding this comment.
What is "linear in :math:\psi"?
There was a problem hiding this comment.
the measurements d, so the comma should perhaps be after psi? Suggestion: it could be made more clear by: ... assimilating measurements d that are linear in psi, is performed...
| K = \Sigma_{\psi}M^\top (M\Sigma_{\psi}M^\top + \Sigma_{d})^{-1} | ||
| \end{align} | ||
|
|
||
| is called tha Kalman gain, and :math:`M` is the linear measurement matrix, so that |
There was a problem hiding this comment.
Can a matrix be non-linear?
There was a problem hiding this comment.
It cannot! How about saying linear measurement operator (i.e., a matrix) (I think this is almost exactly what Evensen says)?
|
|
||
| is the best estimate of :math:`d` under the prior knowledge, and the error is assumed Gaussian with covariance :math:`\Sigma_d`. | ||
| The ensemble variants draw an :math:`N`-sample :math:`\{\psi\}_{i=1}^N` from the prior, | ||
| and perturbs observations :math:`d` using the distributions of measurements creating a corresponding observation-sample :math:`\{d\}_{i=1}^N`. |
There was a problem hiding this comment.
Should be "...and perturb", I think.
|
|
||
| where the estimated Kalman gain :math:`\hat{K}` is found by exchanging the prior covariance with an estimate based on its sample. | ||
| Thus, the ensemble methods combine a sample from the prior with a sample from the likelihood of observed data, to form a new sample from the posterior. | ||
| The posterior distribution that the posterior sample is conseptually sampled from, has mean and covariance found by |
| \hat{\mu}_{ml} = \arg\min_{\mu} |d-M\mu|_2 | ||
| \end{align} | ||
|
|
||
| Furthermore, for a monotone sequence in belief in measurements, a monotone sequence |
There was a problem hiding this comment.
I have a bit of a hard time parsing this sentence.
What if we write the following instead:
Furthermore, for a strictly decreasing sequence in belief in measurements, we expect the distance between the posterior and the maximum likelihood estimate to be strictly decreasing as well.
There was a problem hiding this comment.
I like the suggestion, except for the words "we expect". It either makes it sound like it is some stochastic convergence with an expectation so and so, or that we are not certain. How about skipping "we expect", and adding "... estimate will be strictly decreasing ..." later on?
There was a problem hiding this comment.
This is what happens when you say expect to a statistician...
"will be strictly decreasing" is nice,.
There was a problem hiding this comment.
The other day some friends discussed if they wanted pass/fail or graded exams. They concluded that it was dependent on expected grades and the existing GPA of past exams. For the case of perfect GPA, they said it did not matter as long as the expected grade was perfect. I pointed out that the only way the actual expectation is the perfect grade, is if other grades have zero probability, and thus, you not only feel and expect that you will get a perfect grade but you are certain with zero variation.
There was a problem hiding this comment.
I make parties fun 🎈
|
|
||
| - For the posterior estimate, we require that | ||
|
|
||
| a. The information in :math:`d` has been assimilated, creating a better estimate, so that :math:`|\hat{\mu}_{\psi|d}-\hat{\mu}_{ml}|_2<|\hat{\mu}_{\psi}-\hat{\mu}_{ml}|_2` and :math:`|\hat{\mu}_{\psi|d}-\hat{\psi}_{\psi}|_2<|\hat{\mu}_{\psi}-\hat{\mu}_{ml}|_2`. |
There was a problem hiding this comment.
Does
:math:`|\hat{\mu}{\psi|d}-\hat{\psi}{\psi}|_2
represent the distance between the assimilated mean and the prior mean?
Not sure about the notation \hat{\psi}_{\psi}
There was a problem hiding this comment.
Thanks, this is a typo! It should be the prior sample expectation-estimate, so \hat{\mu}_{\psi}
|
|
||
| In ert, the exact moments of the posterior is not calculated, as information is passed along only through the updated sample. |
There was a problem hiding this comment.
Should be "..are not calculated"
| In ert, the exact moments of the posterior is not calculated, as information is passed along only through the updated sample. | ||
| However, because the perturbations are guaranteed to sum to zero, the posterior sample inherits the exact posterior expectation as its sample mean, | ||
| and as a consequence, the maximum likelihood estimate is preserved. | ||
| This guarantees the path of the posterior estimate as above. |
|
Perhaps we should add a link to this in |
|
|
||
| a. We become more certain of our estimates as informative data is assimilated, thus :math:`0<\det(\Sigma_{\psi|d})<\det(\Sigma_{\psi})`. | ||
| b. We become increasingly certain in our estimates when increasingly informative data is assimilated: When a sequence of :math:`\sigma_d` decreases strictly, then so will the corresponding sequence of :math:`\det(\Sigma_{\psi|d})`. | ||
| c. The certainty of our estimate does not move from the priors, when assimilated data contains no information: When :math:`\sigma_d\to \infty` then :math:`\det(\Sigma_{\psi|d})\to\det(\Sigma_{\psi})` from below. |
There was a problem hiding this comment.
I think comma before "when" can be removed to improve flow.
Yeah I know, overly pedantic :D
There was a problem hiding this comment.
Pedantic is good, thanks!
| a. We become more certain of our estimates as informative data is assimilated, thus :math:`0<\det(\Sigma_{\psi|d})<\det(\Sigma_{\psi})`. | ||
| b. We become increasingly certain in our estimates when increasingly informative data is assimilated: When a sequence of :math:`\sigma_d` decreases strictly, then so will the corresponding sequence of :math:`\det(\Sigma_{\psi|d})`. | ||
| c. The certainty of our estimate does not move from the priors, when assimilated data contains no information: When :math:`\sigma_d\to \infty` then :math:`\det(\Sigma_{\psi|d})\to\det(\Sigma_{\psi})` from below. | ||
| d. If assimilated data is perfect without noise, then we are fully certain of the posterior estiamte: When :math:`\sigma_d\to 0` then :math:`\det(\Sigma_{\psi|d})\to 0` from above. |
There was a problem hiding this comment.
I think this is a bit clearer:
"If assimilated data is perfect, i.e., without noise, then ..."
There was a problem hiding this comment.
😬 should be updated now!
Codecov Report
@@ Coverage Diff @@
## main #2699 +/- ##
==========================================
+ Coverage 61.24% 65.04% +3.80%
==========================================
Files 341 651 +310
Lines 36647 53907 +17260
Branches 4610 4610
==========================================
+ Hits 22444 35065 +12621
- Misses 12722 17360 +4638
- Partials 1481 1482 +1
Continue to review full report at Codecov.
|
I agree. Does such a reference go at the top of the file, or somewhere else? |
I'd put it in a docstring after the #includes |
Blunde1
force-pushed
the
doc-posterior-properties
branch
from
46813d7
to
6d4c4a0
Compare
Issue
Resolves #2660
Approach
Using equations from Evensen with slightly modified notation (perhaps should be changed), introduce updating equations and discuss the properties that these leads to.
The properties are directly used in the code in tests for the updating module.
Implemented in #2628 and #2697