different runs for different subjects #353

crsegerie · 2021-05-28T16:01:34Z

Closes #352.

During the process, I've moved up the get_subjects() function without changing it.

Before merging …

Changelog has been updated (docs/source/changes.md)

agramfort

is it already functional on your end @crsegerie ?

config.py

hoechenberger

Left some small suggestions

Is this already working as intended for you locally? :)

config.py

docs/source/changes.md

config.py

scripts/preprocessing/02-frequency_filter.py

change return type Co-authored-by: Richard Höchenberger <richard.hoechenberger@gmail.com>

change docstring if None return Co-authored-by: Richard Höchenberger <richard.hoechenberger@gmail.com>

docstring update Co-authored-by: Richard Höchenberger <richard.hoechenberger@gmail.com>

crsegerie · 2021-05-29T08:55:18Z

is it already functional on your end @crsegerie ?

Not quite.

There is still an error on my own :

02:08:52 [Step-03][sub-403, run-07] Loading filtered raw data from /home/charb/Desktop/parietal/time_in_wm/derivatives/mne-bids-pipeline/sub-403/meg/sub-403_task-wm_run-07_proc-filt_raw.fif
02:08:53 [Step-03][sub-403, run-08] Loading filtered raw data from /home/charb/Desktop/parietal/time_in_wm/derivatives/mne-bids-pipeline/sub-403/meg/sub-403_task-wm_run-08_proc-filt_raw.fif
Killed

The killed surprises me, because I do not see it in CI

And at the beginning of each step, the warning appears:

01:58:28 Extracted all the runs. Beware, not all subjects share the same list of run-names.

I should change it to only warn for the first step

Co-authored-by: Alexandre Gramfort <alexandre.gramfort@m4x.org>

hoechenberger · 2021-05-29T09:02:17Z

Killed

This most likely indicates that you're running out of memory … We'd need to see how to optimize things here. This problem seems unrelated to this PR though

hoechenberger · 2021-05-29T09:04:32Z

@agramfort Do you think instead of concatenating all filtered runs before creating epochs, we could create epochs for each run individually and then concat those? This would avoid the out-of-memory problem @crsegerie sees locally. 8+ runs is heavy stuff.

hoechenberger · 2021-05-29T09:08:44Z

@agramfort Do you think instead of concatenating all filtered runs before creating epochs, we could create epochs for each run individually and then concat those?

Ah, right, there is mne.concatenate_epochs()

I can take a stab at this sometime later today (separate PR)

…erie/mne-bids-pipeline into different-runs-by-subjects

config.py

Co-authored-by: Alexandre Gramfort <alexandre.gramfort@m4x.org>

crsegerie · 2021-05-31T07:49:45Z

There is a conflict I'm not sure how to deal with :

I do not understand the MKDOC part ?

hoechenberger · 2021-05-31T11:00:28Z

I do not understand the MKDOC part ?

I added this MKDOCS check to make it work with our documentation build, which uses MkDocs. It's a bit hacky but seems to work: When building the docs, we set this environment variable so the config "knows": "oh ok, so this is a doc build, not an actual run of the pipeline."

config.py

delete parentheses Co-authored-by: Richard Höchenberger <richard.hoechenberger@gmail.com>

config.py

hoechenberger

Just some nitpicks re docstrings, otherwise LGTM

hoechenberger · 2021-06-02T07:49:41Z

config.py

+
+
+def get_intersect_run() -> list:
+    '''Returns the intersection of all the runs of all subjects.'''


Can you use double-quotes """xxxx""" around the docstring please?

config.py

hoechenberger · 2021-06-02T08:06:58Z

Great job, @crsegerie!!!

agramfort · 2021-06-02T08:49:23Z

+1 really cool !

…

agramfort · 2021-06-02T09:13:53Z

🎉

crsegerie added 5 commits May 28, 2021 17:52

different runs for different subjects

9441ff9

changes.md

edb2e79

bug hash list

3ac7ec6

delete old comment

9eb1999

added print and value error to debug

e3b9e83

agramfort reviewed May 28, 2021

View reviewed changes

config.py Show resolved Hide resolved

crsegerie added 3 commits May 29, 2021 01:47

add returns section

91c7dff

typo

b3b5da1

delete print

b2823d8

hoechenberger reviewed May 29, 2021

View reviewed changes

config.py Outdated Show resolved Hide resolved

config.py Outdated Show resolved Hide resolved

config.py Outdated Show resolved Hide resolved

docs/source/changes.md Outdated Show resolved Hide resolved

agramfort reviewed May 29, 2021

View reviewed changes

config.py Outdated Show resolved Hide resolved

config.py Show resolved Hide resolved

scripts/preprocessing/02-frequency_filter.py Outdated Show resolved Hide resolved

crsegerie and others added 3 commits May 29, 2021 10:50

Update config.py

a1ac781

change return type Co-authored-by: Richard Höchenberger <richard.hoechenberger@gmail.com>

Update config.py

4a7b241

change docstring if None return Co-authored-by: Richard Höchenberger <richard.hoechenberger@gmail.com>

Update config.py

6bb960f

docstring update Co-authored-by: Richard Höchenberger <richard.hoechenberger@gmail.com>

Update scripts/preprocessing/02-frequency_filter.py

1998220

Co-authored-by: Alexandre Gramfort <alexandre.gramfort@m4x.org>

doctring

dee6c31

crsegerie added 2 commits May 29, 2021 11:10

Merge branch 'different-runs-by-subjects' of https://github.com/crseg…

8117031

…erie/mne-bids-pipeline into different-runs-by-subjects

correct verbose

a8c5cf7

hoechenberger mentioned this pull request May 29, 2021

Reduce memory usage when creating epochs #354

Closed

agramfort reviewed May 30, 2021

View reviewed changes

config.py Show resolved Hide resolved

config.py Show resolved Hide resolved

config.py Outdated Show resolved Hide resolved

Update config.py

3a99b15

Co-authored-by: Alexandre Gramfort <alexandre.gramfort@m4x.org>

crsegerie marked this pull request as ready for review May 31, 2021 07:42

crsegerie and others added 2 commits May 31, 2021 13:10

Merge branch 'main' into different-runs-by-subjects

6e8f1ed

style

93de5cb

MKDOCS check

34d3f2f

hoechenberger reviewed May 31, 2021

View reviewed changes

config.py Outdated Show resolved Hide resolved

crsegerie and others added 4 commits May 31, 2021 20:38

Update config.py

de63e4f

delete parentheses Co-authored-by: Richard Höchenberger <richard.hoechenberger@gmail.com>

Merge branch 'main' into different-runs-by-subjects

a407cc2

too many black line

9ab1146

resolve issue subject missing

fd2d4bb

hoechenberger reviewed Jun 1, 2021

View reviewed changes

config.py Outdated Show resolved Hide resolved

crsegerie and others added 2 commits June 1, 2021 19:20

delete arguments in comment

c1a5d1f

Merge branch 'main' into different-runs-by-subjects

7ff32ef

hoechenberger reviewed Jun 2, 2021

View reviewed changes

docstring

8afa62c

hoechenberger reviewed Jun 2, 2021

View reviewed changes

config.py Outdated Show resolved Hide resolved

Update config.py

9c17ab5

hoechenberger approved these changes Jun 2, 2021

View reviewed changes

hoechenberger merged commit d4061dc into mne-tools:main Jun 2, 2021

crsegerie deleted the different-runs-by-subjects branch June 2, 2021 08:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

different runs for different subjects #353

different runs for different subjects #353

crsegerie commented May 28, 2021 •

edited

agramfort left a comment

hoechenberger left a comment

crsegerie commented May 29, 2021 •

edited

hoechenberger commented May 29, 2021 •

edited

hoechenberger commented May 29, 2021 •

edited

hoechenberger commented May 29, 2021

crsegerie commented May 31, 2021

hoechenberger commented May 31, 2021

hoechenberger left a comment

hoechenberger Jun 2, 2021 •

edited

hoechenberger commented Jun 2, 2021

agramfort commented Jun 2, 2021 via email

agramfort commented Jun 2, 2021



		def get_intersect_run() -> list:
		'''Returns the intersection of all the runs of all subjects.'''

different runs for different subjects #353

different runs for different subjects #353

Conversation

crsegerie commented May 28, 2021 • edited

Before merging …

agramfort left a comment

Choose a reason for hiding this comment

hoechenberger left a comment

Choose a reason for hiding this comment

crsegerie commented May 29, 2021 • edited

hoechenberger commented May 29, 2021 • edited

hoechenberger commented May 29, 2021 • edited

hoechenberger commented May 29, 2021

crsegerie commented May 31, 2021

hoechenberger commented May 31, 2021

hoechenberger left a comment

Choose a reason for hiding this comment

hoechenberger Jun 2, 2021 • edited

Choose a reason for hiding this comment

hoechenberger commented Jun 2, 2021

agramfort commented Jun 2, 2021 via email

agramfort commented Jun 2, 2021

crsegerie commented May 28, 2021 •

edited

crsegerie commented May 29, 2021 •

edited

hoechenberger commented May 29, 2021 •

edited

hoechenberger commented May 29, 2021 •

edited

hoechenberger Jun 2, 2021 •

edited