Random permutation of FCMA

[yidawang]

1. add random voxel permutation to FCMA;
2. add more debugging and information logs;
3. minor tweak for docstring format, etc.

Thanks in advance for reviewing.

[xoltar]

Thanks, Yida! Looks pretty good, just a few small changes needed.

[xoltar]

_randomize_single_subject modifies data in place (and then returns the same array). _randomize_subject_list returns a new list, containing old arrays that have been modified in place, which is why these lines work even though you're ignoring the return value. I would say either write them both to work in-place, or write them both to return new values. Mixing the two makes for confusion. If you decide you want to have _randomize_single_subject return a new array instead of working in-place, consider numpy.random.permutation instead of shuffle.

[yidawang]

Thanks for the suggestion! Somehow I ignored the return value here although I should put it as activity_data1 = _randomize_subject_list(activity_data1, random). I would prefer working in-place, let me modify _randomize_subject_list accordingly.

[xoltar]

nVoxels

[yidawang]

done

[xoltar]

nVoxels

[yidawang]

done

[xoltar]

You could make this reshaping logic part of _randomize_single_subject (i.e. have it flatten, randomize, then reshape into whatever shape the original array was).

[yidawang]

I didn't do this because I would like to make the input argument data of _randomize_single_subject an nVoxels x nTRs 2D array for clarity purpose.

[xoltar]

logger does formatting itself, you can just do:
logger.debug('file %s is loaded and processed, with data shape $s', f.get_filename(), data.shape)

[yidawang]

done

[xoltar]

Same logging comments apply to this file.

[yidawang]

done

[xoltar]

Here too. No need for str(), either.

[yidawang]

f is a path here, which is not supported to be the argument of numpy.load(), and presumably nibabel.load(), for numpy version before 1.12. @mihaic should be able to further justify this.

[mihaic]

Right. However, Paths work with string formatting without explicit casting to str, e.g.:
"test %s" % pathlib.Path("/")

[yidawang]

OK, just realized that Bryn was referring the str usage below this line :) Done.

[mihaic]

You should not specify the default because you are duplicating the information in the function definition, which is visible both in the code and in the generated docs. (Of course, if you use None as a placeholder, you should explain what it stands for.)

[yidawang]

done

[mihaic]

Use alphabetical order, please. I will introduce a checker real soon now®.

[yidawang]

done

[mihaic]

I would say you should use the debug level for the messages you added in "io.py".

Also, do not break lines if they do not exceed the maximum line length.

Finally, the other message placement, before yield, makes more sense, but I would change the wording to "Starting to read [...]" and put a full stop at the end of the message.

[yidawang]

done, but I would put some brief logger info to indicate in the beginning of some stages (reading file, apply masking, separating epochs, etc.)

[mihaic]

Sure, but not in "io.py".

[yidawang]

Why not? I think within the io methods are the best places to go.
BTW, I am feeling that io.load_images_from_dir and io.load_images should be merged.

[yidawang]

anyway, I will leave io.py intact

[mihaic]

Because they are not useful. Why would a function log an info-level message saying it has been called every time it is called? Such messages could be emitted by the caller, if relevant.

We can discuss merging the load functions, but I would rather not, because "_dir" requires a suffix as well, so we would need to explain that.

[yidawang]

well, the suffix can be an optional argument. Let's discuss this in the next standup.

[mihaic]

It makes more sense to rename extension to suffix here.

[yidawang]

done

[mihaic]

Same.

[yidawang]

done

[mihaic]

Please write sentences in log messages, at least in this file.

[yidawang]

done

[xoltar]

This is better, but still mixed usage. If you're going to do the change in place, don't return a value. If you're going to return a value, it should be a new value. So if you want to use in-place, it should look like:

for subject in data_list:
    _randomize_single_subject(subject)

So neither _randomize_single_subject nor this function should return a value. Also, note since you're modifying in place, you don't need to assign to indexes, so you can use the simpler for loop.

[yidawang]

done

[xoltar]

Good! Now just need to remove the Returns section from docs of both of these functions since they don't return anything anymore.