CCA/CKA/OPD support

[brando90]

Is CCA or CKA or Orthogonal Procrustes Distance (OPD) available for captum?

[brando90]

https://discuss.pytorch.org/t/cca-cka-for-captum/107084

[brando90]

https://discuss.pytorch.org/t/what-is-a-good-cca-cka-library-for-pytorch-that-works-ideally-with-gpu/104889

[NarineK]

@brando90, thank you for the question. No, we don't have CCA / CKA implementation in captum library. We thought about it but we haven't added it yet.

[brando90]

@brando90, thank you for the question. No, we don't have CCA / CKA implementation in captum library. We thought about it but we haven't added it yet.

Thanks fo your reply Nerine!

What do you think of incorporating anatome's CCA/CKA etc into captum (since it seems to be an already working implementation of this in pytorch)?

[NarineK]

Thank you @brando90 for sharing the link. I'll look deeper into it and estimate the efforts required to implement it. You mentioned that currently anatome doesn't work reliably and ideally you'd like to have a more reliable implementation of the algorithms implemented there. Is that right ?

[brando90]

Yes thats exactly right! There are issues like weird cuda errors though I believe its their implementation.

…

On Jan 20, 2021, at 4:35 PM, NarineK ***@***.***> wrote: Thank you @brando90 <https://github.com/brando90> for sharing the link. I'll look deeper into it and estimate the efforts required to implement it. You mentioned that currently anatome doesn't work reliably and ideally you'd like to have a more reliable implementation of the algorithms implemented there. Is that right ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#572 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOE6LSEBO2NAIOGNP75E73S25LBVANCNFSM4VIQON5Q>.

[pietrolesci]

Hi @NarineK,

I hope you are doing well. Any update on adding support for CCA, PWCCA, CKA, SVCCA in Captum?
As mentioned by @brando90, the anatome library - his fork in particular - has some implementations already. Would be great to have it under the Captum umbrella.

Happy to help if needed.

Thanks a lot in advance for your consideration.

Best,
Pietro

[brando90]

Hi @NarineK,

I hope you are doing well. Any update on adding support for CCA, PWCCA, CKA, SVCCA in Captum? As mentioned by @brando90, the anatome library - his fork in particular - has some implementations already. Would be great to have it under the Captum umbrella.

Happy to help if needed.

Thanks a lot in advance for your consideration.

Best, Pietro

I've extended the original and tested it here (with all distances): https://github.com/brando90/ultimate-anatome/tree/master/sanity_checks I recommend to do that (unless you want to get in the hairy details in the original gitissue). My fork is in pypi so you can do pip install ultimate-anatome and it should work out of the box I hope. If not post issues in my repo.

[brando90]

my suggestion if you want to help is probably to test it more thoroughly e.g. reproducing all tests here: https://github.com/google/svcca/blob/master/tutorials/001_Introduction.ipynb and trying it in your code. Then we could fix my fork with pull requests + add the extra sanity checks.

[pietrolesci]

Hi @brando90,

Thanks for your answer. I definitely think that this can be a way forward. Though, having everything in Captum was a quite captivating idea (pun intended). I will definitely use your fork for now and will monitor Captum development to see whether this will be supported.

Best,
Pietro

[brando90]

Though, having everything in Captum was a quite captivating idea (pun intended).

Agreed. How else do you recommend going forward? I think the best is:

1. reproduce as many tests from google's SVCCA tutorial (to gain confidence)
2. then once we know we have a slightly solid version of my ultimate-anatome, push it to Captum - to captive future users ;)

Let me know if you think there is a better alternative. :)

Regards, Brando

[NarineK]

We have done some work CCA/CKA that hasn't been merged to master yet. @bilalsal can tell more about it.

[pietrolesci]

Hi @NarineK,

Thanks a lot for your answer.

To add to the discussion, I have recently found out that @AntixK (CC'ed) has published the Pytorch-Model-Compare library. I think there can be value in coordinating the efforts on this topic - I hope this can be useful to centralize the discussion.

Best,
Pietro

[brando90]

thanks!

Not sure how we can do that, perhaps pinging them here might be best, but for now my ultimate-anatome is working and I've put enough effort into it that that's what I will stick with for now unless there are excellent reasons to switch. I agree it's a pitty for the distributed efforts, alas! I guess it happens when people are secretive about research or perhaps it's hard to google with slightly different names.

Thanks Pietro.

---

if it has any value, I added the links to the libraries in the end of my readme.

[joshua-xia]

any progress? It is very important feature, Thanks!