Starting from strip_analysis, some video get "randomly" skipped and not analyzed for no reason => Joss

[Josster]

On several folders containing each on average 10 videos, there is each time 1-2 video not processed
for no apparent reason.

[mnagaoglu]

Did you check the logs? Can you attach a log demonstrating the problem or a way to reproduce this?

…

On Jun 21, 2018, at 11:57 AM, Josster ***@***.***> wrote: On several folders containing each on average 10 videos, there is each time ### 1 video not processed### for no apparent reason. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#14>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AY_-V1sDI4sOIY_bcdiETpp4FNCH7HCrks5t--yLgaJpZM4UymRC>.

[Josster]

It is unclear I am investigating. The skipped video, when ran again through ReVAS get normally analysed...
Some stat so far: Number of video Actually analysed/Total Number per folder
ail_os 26/27
4/12 (!)
4/18
5/6
5/6
21/24
8/9
4/9

Here is a logs from jxl on 3 folders
C:\Users\spencer\Google Drive\SELAB code\Joss\Fixation_letters\bugs
21/24=>3 videos skipped
8/9 =>1 skipped
4/9 => 5 [skipped]

[mnagaoglu]

I wonder if it has to do with memory. Maybe there is a memory leakage somewhere and it gives up on some of the videos. When you run them separately, no problem… Hmm.. Need a way to test this hypothesis.

…

On Jun 21, 2018, at 12:21 PM, Josster ***@***.***> wrote: It is unclear I am investigating. The skipped video, when ran again through ReVAS get normally analysed... Some stat so far: Number of video Actually analysed/Total Number per folder ail_os 26/27 4/12 (!) 4/18 5/6 5/6 21/24 8/9 4/9 Here are some logs= from jxl on 3 folders 21/24=>3 videos skipped 8/9 =>1 skipped 4/9 => 5 [skipped] — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#14 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AY_-V73Xti9R04OHYmQ0WShAwlBzrWDSks5t-_JEgaJpZM4UymRC>.

[Josster]

ok, I isolated two Errors:
Error1
Error while processing E:\bxh_os\VA Fix Expt\10Deg Field\2_20_2015_13_41\bxh_os_11_160_dwt_nostim_gamscaled_bandfilt.avi. Proceeding to next video.) Undefined function or variable 'beginNaNs'. CoarseRef(270) < ExecuteModules(143) < make_general_channel/channel_general(929) < remoteParallelFunction(38).

it is probably because we might read a video not starting nor Ending with NaNs.
hence the variable beginNaNs could be beginNaNs=[];
so in CoarseRef.m, the condition in line 270
if frameNumber < (1 + beginNaNs) || any(badFrames == frameNumber)
should be rewritten by
if isempty(beginNaNs) if frameNumber < (1 ) || any(badFrames == frameNumber) ... else if frameNumber < (1 + beginNaNs) || any(badFrames == frameNumber) ....

==> I will correct this

[Josster]

Error 2
Error while processing E:\jxl_os\VA Fix Expt\10Deg\Set III\jxl_os_4_dwt_nostim_gamscaled_bandfilt.avi. Proceeding to next video.) Reference to non-existent field 'stabilizeVideo'. MakeMontage(288) < FineRef(98) < ExecuteModules(169) < make_general_channel/channel_general(929) < remoteParallelFunction(38).

=>Error 2 solved by Mehmet and Joss

[Josster]

Update about Error 1: bug with 'beginNaNs' in CoarseRef
The bug style exist in the GUI mode (both with parallelisation or NOT), but I cannot reproduce it in command line, by putting a breakpoint on the line 270 or adding this line and putting a breakpoint in it

[Josster]

so here is my temporary fix before line 270 in CoarseRef.m :
if ~exist('beginNaNs','var')
beginNaNs=0;
end
if ~exist('beginNaNs','var')
endNaNs=0;
end

[mnagaoglu]

What's the status on this issue? Did you temporary fix resolve the issue? If so, we can make it a temporary one.

By the way, I think the second if statement should be as follows, right? @Josster

if ~exist('endNaNs','var')
      endNaNs=0;
end

[mnagaoglu]

I solved the issue presumably in a more robust way by combining your method with also checking the value of beingNaNs and endNaNs when they do exist. Here it how it looks like:

try
        endNaNs = max(endNaNs1, endNaNs2);
        beginNaNs = max(beginNaNs1, beginNaNs2);
        if isempty(beginNaNs)
            beginNaNs = 0;
        end
        if isempty(endNaNs)
            beginNaNs = 0;
        end
    catch
        % just in case 
        if ~exist('beginNaNs','var')
              beginNaNs = 0;
        end
        if ~exist('endNaNs','var')
              endNaNs = 0;
        end
    end

Here is the link to the revision