Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MRG, ENH: Improve MATLAB compat #8515

Merged
merged 1 commit into from Nov 14, 2020
Merged

MRG, ENH: Improve MATLAB compat #8515

merged 1 commit into from Nov 14, 2020

Conversation

larsoner
Copy link
Member

Closes #8513

@ftadel can you give this branch a shot? Marking as backport-candidate so that we can backport then release 0.21.2 to make things easier once we get everything working (hopefully quickly!).

@larsoner
Copy link
Member Author

CircleCI passed the download stage so I killed it as that's all I really wanted to check -- it's going to die until #8516 lands anyway

@ftadel
Copy link

ftadel commented Nov 13, 2020

It seems to be working like a charm!
Now it's a bit frustrating not to be able to follow the progression of the task at all, but at least it works!
We'll explore replacement solutions when we have time, or when Matlab gets fixed.

@ftadel
Copy link

ftadel commented Nov 13, 2020

Now, I have a problem with joblib:

The example I ran, from @dimitriospantazis:

clf = py.sklearn.pipeline.make_pipeline(py.sklearn.preprocessing.StandardScaler(), py.sklearn.svm.SVC(pyargs('kernel', 'linear', 'C', 1)));
time_decod = py.mne.decoding.SlidingEstimator(clf, pyargs('n_jobs', int32(1), 'scoring', 'roc_auc', 'verbose', false));
scores = py.mne.decoding.cross_val_multiscore(time_decod, X, y, pyargs('cv', int32(5), 'n_jobs', int32(1), 'verbose', false));

It crashes if I set n_jobs to 4:

>> scores = py.mne.decoding.cross_val_multiscore(time_decod, X, y, pyargs('cv', int32(5), 'n_jobs', int32(4), 'verbose', false));
Error using parallel>_print (line 876)
Python Error: AttributeError: 'NoneType' object has no attribute 'write'
Error in parallel>__call__ (line 1006)
Error in parallel>run (line 128)
Error in base>cross_val_multiscore (line 441)

Together with this Matlab error:
image

I haven't investigated this at all. Posting it just in case you already have an idea of the issue.
Maybe it won't be possible to run any parallel processing from Matlab?...

@larsoner
Copy link
Member Author

@ftadel I'm not sure, there are at least some hints here about interoperability (but it's using MATLAB from Python):

https://stackoverflow.com/questions/55424095/error-pickling-a-matlab-object-in-joblib-parallel-context

I think we'll need to come up with a minimal joblib example and bother the joblib folks with it. It's possible that some backend other than loky would work.

@agramfort
Copy link
Member

agramfort commented Nov 14, 2020 via email

@ftadel
Copy link

ftadel commented Nov 14, 2020

basically add MNE_FORCE_SERIAL to environment variables

That's a good start, thanks. At least we can make sure everything will work without parallelization issues.

I think we'll need to come up with a minimal joblib example and bother the joblib folks

I'll work on this soon.
Thanks for your help!

When can we expect this PR to be in the "current version of MNE-Python that people get with pip install mne"?

@larsoner larsoner merged commit d73e0ab into mne-tools:master Nov 14, 2020
1 of 4 checks passed
@larsoner larsoner deleted the matlab branch November 14, 2020 18:33
larsoner added a commit that referenced this pull request Nov 14, 2020
@larsoner
Copy link
Member Author

@ftadel I can push out a 0.21.2 to PyPi on Monday

@ftadel
Copy link

ftadel commented Nov 15, 2020

I can push out a 0.21.2 to PyPi on Monday

You're incredibly reactive!! Thanks :)

I think we'll need to come up with a minimal joblib example and bother the joblib folks

joblib/joblib#1131

@larsoner
Copy link
Member Author

0.21.2 is out

https://pypi.org/project/mne/0.21.2/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Option to disable tqdm (crashes on Matlab/Windows10)
3 participants