Feature/enable l2 dense solver #29
Conversation
src/fugw/solvers/utils.py
Outdated
| @@ -938,18 +1077,96 @@ def elementwise_prod_fact_sparse(a, b, p): | |||
| ) | |||
|
|
|||
|
|
|||
| def compute_approx_kl(p, q): | |||
There was a problem hiding this comment.
Not really clear what approx_kl is supposed to be (ref ?)
Again, for publica functions add at elast 1-line docstring.
There was a problem hiding this comment.
Indeed, this naming is misleading.
The idea is that what we are computing here is not exactly a KL divergence, because p and q are not necessarily probability measures.
@6Ulm and I suggest we replace _approx_ with unnormalized or generalized. What do you think?
There was a problem hiding this comment.
To clarify, the true KL divergence between any 2 positive vectors is KL(p | q) = < p, log (p/q) > - mass(p) + mass(q). That's why at the beginning, we denoted the first term approx_kl for approximation of the KL.
There was a problem hiding this comment.
Yaaay, unnormalized it is!
| loss_wasserstein = csr_sum( | ||
| elementwise_prod_fact_sparse(K1, K2, pi + gamma) | ||
| loss_wasserstein = ( | ||
| csr_sum(elementwise_prod_fact_sparse(K1, K2, pi + gamma)) / 2 |
This PR: