-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Permutation tests for CCA #124
Comments
Funny you mention this as it's something vaguely on my radar! I actually made a start porting the quickperms from the PALM toolbox (with the permission of Winkler/Smith) here: https://github.com/jameschapman19/scikit-perm/blob/main/skperm/permutation_tests/cca_permutation_test.py Which would be a nice bonus with a ported version of permCCA (i.e. where the user can supply their own permutations based on exchangeability blocks). |
I got code for the CCA part mentioned by @LegrandNico. The full implementation in python using cca_zoo.
This is an incredibly lazy attempt. |
Thank you @htwangtw , I think that will be really helpful. I don't know how you want to integrate the permutation functionalities with the rest of the package. I can try to make something, but maybe will start with an example tutorial notebook see if we have everything running. |
I would guess you could adapt @htwangtw's code to look something like https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.permutation_test_score.html Could be rudimentary initially with the goal to get close to sklearn API which could run permutation tests in parallel |
Just for completeness: There are two other packages that might also help? The pyls package has implemented permutation tests: And there's also the resample package: |
I've put a version of this in cca_zoo.model_selection._validation with an API that hooks into scikit-learn permutation_test_score. It's not quite the same as what Winkler does for multiple latent dimensions (but it should be similar) but it works for 1 latent dimension. Should be able to build on this. |
I'll add a proper example but it should work like:
which returns score (average correlation across dimensions), permutation scores, p-value like scikit-learn |
Although I got a lot insights from your nice discussion here, one thing I am still confused: It's understandable that when permuting more variables, the random level is high, so that the canonical correlation coefficient is low compared to the reference experiment. Many thanks in advance! Cheers, |
I would like to implement (if not already available elsewhere) a Python version of the permutation tests for CCA described in
The paper comes with a repository (Matlab) that could be ported to Python without requiring additional dependencies.
The text was updated successfully, but these errors were encountered: