Hutch++

[Fr0do]

Resolves #47

[codecov]

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.23% 🎉

Comparison is base (64c2fd6) 78.62% compared to head (0f8262d) 78.85%.

❗ Current head 0f8262d differs from pull request most recent head 8014329. Consider uploading reports for the commit 8014329 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #52      +/-   ##
==========================================
+ Coverage   78.62%   78.85%   +0.23%     
==========================================
  Files          36       36              
  Lines        2975     2998      +23     
==========================================
+ Hits         2339     2364      +25     
+ Misses        636      634       -2     

Files Changed	Coverage Δ
cola/algorithms/diagonal_estimation.py	81.25% <100.00%> (+4.84%)	⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mfinzi]

Really nice work!

Maybe we can add a case to approx_diag test

Have you evaluated on some LinearOperators to verify that we get the improved convergence rate?

Also, I wonder if we will need to think about the error tolerances differently for this function. For the standard Hutchinson estimator we have a stopping criteria as the standard error of the estimate, but it doesn't really make sense in the context of H++. I wonder if we should be deciding the number of probes up front based on the bound in their paper?

[mfinzi]

Maybe lets separate this kwarg into
hutchpp=False
probe_dist='normal' or 'rademacher'
to keep things more straightforward

[mfinzi]

I wonder if it would be worth looking into the single pass estimator in https://arxiv.org/abs/2109.10659, @Fr0do do you have any thoughts on the differences between the different $O(1/\epsilon)$ algorithms?