json createDataDictionary overwrite

[CerenB]

I'm opening two logfiles (_event and _stim) consecuitively. When I only open the logfile (_event) with extraColumn, createDataDictionary nicely creates a json file with all the extra columns.
Then when I open the following, _stim, that json file for the logfile / _event got back to its default settings without the extraColumns.

Any idea why is it happening?

    % create  logfile with extra columns to save - BIDS
    logFile.extraColumns = cfg.extraColumns;
    [logFile]  = saveEventsFile('open', cfg, logFile); %dummy initialise

    % set the real length of columns
    logFile(1).extraColumns.LHL24.length = 12;
    logFile(1).extraColumns.PE4.length = 12;

    % actual inititalization
    logFile = saveEventsFile('open', cfg, logFile);
    
    % create response file - used for counting button press
    responseFile.extraColumns = cfg.responseExtraColumns;
    responseFile  = saveEventsFile('open_stim', cfg, responseFile);

[Remi-Gau]

Will look into this: will most likely write a test for that.
Trying to figure out on which branh / version you are working: what is the latest commit of CPP_BIDS when you do git log ?

[CerenB]

I can see the issue is for both _event and _stim initialisation in saveEventFile.m, it creates the same file name json. If it's not required to create json dictionary for _stim, it would not over write the file.

function logFile = initializeFile(cfg, logFile)

    logFile = initializeExtraColumns(logFile);

    createDataDictionary(cfg, logFile);

    % Initialize txt logfiles and empty fields for the standard BIDS
    %  event file
    logFile(1).fileID = fopen( ...
        fullfile( ...
        cfg.dir.outputSubject, ...
        cfg.fileName.modality, ...
        logFile.filename), ...
        'w');

    % print the basic BIDS columns
    fprintf(logFile(1).fileID, '%s\t%s\t%s', 'onset', 'duration', 'trial_type');
    fprintf(1, '%s\t%s\t%s', 'onset', 'duration', 'trial_type');

    printHeaderExtraColumns(logFile);

    % next line so we start printing at the right place
    fprintf(logFile(1).fileID, '\n');
    fprintf(1, '\n');

end

Here is my latest CPP_BIDS git log:

(base) mac-114-168:CPP_BIDS battal$ git log
commit 17ab12d096b9611a0dd62c6136b7d90285ee7ab7 (HEAD, CerenB/master, master, ceren_mod)
Merge: bc0aaa6 962c947
Author: Ceren <battal.ceren@gmail.com>
Date:   Mon Aug 31 12:34:40 2020 +0200

    Merge tag 'v1.0.0'

commit 962c947fe38094da9561eeba5daa44993505f2c0 (tag: v1.0.0)
Merge: f6e3b4c febf3e6
Author: Remi Gau <remi_gau@hotmail.com>
Date:   Sat Aug 22 17:20:21 2020 +0200

    Merge pull request #85 from cpp-lln-lab/remi-update_randomize
    
    fix typos in cfg files for mh_linter

[Remi-Gau]

Actually createDataDictionary adapts the name of the json to the name of the tsv you give it. It removes the .tsv extension and puts the .json instead.

And because you open the 2 files with 'open' or 'open_stim' they will have different .tsv filename, so they will have distinct json names.

    logFile = saveEventsFile('open', cfg, logFile);
    responseFile  = saveEventsFile('open_stim', cfg, responseFile);

[Remi-Gau]

I think I am able to reproduce the error.

[Remi-Gau]

Actually createDataDictionary adapts the name of the json to the name of the tsv you give it. It removes the .tsv extension and puts the .json instead.

And because you open the 2 files with 'open' or 'open_stim' they will have different .tsv filename, so they will have distinct json names.

    logFile = saveEventsFile('open', cfg, logFile);
    responseFile  = saveEventsFile('open_stim', cfg, responseFile);

OK and my message describes the theory not what it does in practice: "and that's why, kids, you should write tests for your code to make sure it does what you think it does!"

[CerenB]

I think the issue is with open_stim, it goes to createDataDictionary and re-writes the json file name with the _event file name. See from createDataDictionary.m

fileName = strrep(cfg.fileName.events, '.tsv', '.json');

[Remi-Gau]

yup I am fixing createDataDictionary to take the filename of the logfile instead.

also added tests to make sure it actually creates the right json.