Check speed of reading large number of events

[arnodelorme]

I have a 40-min datasets in which there are around 10,000 events. I notice that Matlab takes take to save and load this big structure. Looks like it is not due to the data size but an issue with Matlab structure... I had that feeling when I tried to save the structure separately. To save such a big structure, it takes a few minutes; if you have 20 datasets, it will take more than an hour just to load the data.

Here, you can find some datasets with a few thousands of events.
/data/projects/Greg/audioTraining/p05_backprojected/

Here, you can find some datasets a few tens of thousands of events.
/data/mobi/FunctionalVision/ARVO2015Paper/p07_Ozgurs3rdFunction/

Any one of the files would do. Please load and save them while measuring time. It should take a few minutes.

[cll008]

Neither of those file locations exist anymore. I can time this if you let me know where else we have similar data.

[arnodelorme]

Ask Makoto.

[cll008]

The functional vision dataset has been moved to \data\common\mobi\Experiments\FunctionalVision\p10_Ozgurs3rdFunction
Loading a dataset with 16334 events took just over 15 seconds, saving took under 25 seconds (Win10, MATLAB 2020a, EEGLAB 2021).

[arnodelorme]

Sounds good.

…

On May 26, 2021, at 11:24 AM, Clement Lee ***@***.***> wrote: Closed #75. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.