Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very slow estimation of correlation matrix #679

Closed
hynekcigler opened this issue Apr 15, 2020 · 3 comments · Fixed by jasp-stats/jaspRegression#1 or jasp-stats/jasp-desktop#4082
Closed
Assignees

Comments

@hynekcigler
Copy link

  • JASP version: 0.12
  • OS name and version: Windows 10
  • Analysis: Correlation (Regression -> Classical -> Correlation
  • Bug description: Very slow estimation of correlation matrix

I have not-so-big dataset (N=1000). If I run a correlations over all the 28 items (vah.a1-vah.b28), the estimation is extremely slow and the analysis fails soon. It is slow even if I run it over just e.g. three or four variables. If the dataset is loaded, these variables are set as ordinal. I tried to set them as continous, but the estimation is slow too. On the other hand, running e.g. factor analysis (which needs also an estimation of covariance matrix) is quick, so the problem is only in the correlation matrix. I tried Pearson and Spearman correlations, both are similarly slow.

Steps to reproduce:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error
@AlexanderLyNL
Copy link
Contributor

It is indeed quite slow. I did the computations for you on my own JASP development build:

vaha4.jasp.zip

Let me know if you can open it. I also added the Bayesian equivalent to the file for your convenience. Those computations are done relatively quickly it seems.

This is definitely not ideal, and it seems that some (permutation based?) computations are done. @Kucharssim is that correct? If so, can we not do that by default?

@hynekcigler
Copy link
Author

Thank you. Actually, I don't need these computations on my own as I use R usually, but my students are given this dataset for their assignment to perform exploratory factor analysis (and I would like them to report correlation matrix).
Anyway, the estimation is slow even if I did not ask for signifficance, where any permutation test is not needed (and even the permutation test performed directly in R takes just a few seconds).

@Kucharssim
Copy link
Member

Yes, this is likely caused by calculating CIs for Kendall’s tau which is by mistake now computed even if it’s not requested. I’ll fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants