Add TarchiveID to mri_protocol_violated_scans #211
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nicolasbrossard
requested changes
Dec 11, 2017
|
Thank you @nicolasbrossard. I made the changes and renamed $tarchiveInfo to $tarchiveInfoRef |
nicolasbrossard
approved these changes
Dec 20, 2017
|
@cmadjar |
|
tested it on a regular scan with no violated scans and one with a violated scan and the TarchiveID is properly populated in the latter case. since the Loris part is merged, I will merge this one as well so they both go to 19.0. |
cmadjar
pushed a commit
that referenced
this pull request
Jan 30, 2018
* Mapping empty date of birth and scan to undef when empty so that it can be inserted in MySQL 5.7 (#205) * Creected code that tests if two float numbers are equal (#206) * [Cleanup]: Remove duplicate DBI.pm; Redmine 13613 (#207) * Add TarchiveID to mri_protocol_violated_scans (#211) * Adding TarchiveID information in mri_protocol_violated_scans table * Cleaned up commented parts * Renamed $tarchiveInfo to $tarchiveInfoRef for consistencies * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 (#203) * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 * change xargs to deleting from the find command * Dave and Nick feedback * Nicolas feedback; escaping the directory name * Refactor smart matching code in DTI.pm (#215) * Better MINC toolkit sourcing command in README and updated VERSION file (#260) * Better MINC toolkit sourcing command and updated VERSION file * cecile feedback * Updating QC tables if the -deleteQCdata flag is not set. Will update FileID to NULL in files_qcstatus and feedback_mri_comments (#261) * README has also within it LORIS-MRI version number, so update (#268)
cmadjar
pushed a commit
that referenced
this pull request
Feb 22, 2018
* Initial import of files from http://code.google.com/p/dicom-archive/ * add two configurable options is_site and is_sge * Added new getIncomingDir config function * Added missing semi-colons. * Modifies the temporairy values of $data_dir, $get_dicom_info and $mail_user variables * Places back the forward slash in the email-address * Modifies the temporairy value for the tarchive_librairy location * Modified the tarchivelibdir temp value * Set the if_sge to false by default * Fixed the dcmodify --insert-tag option to --insert (option used in more recent dcmodify). * Updates the mri_upload table * Includes option to update the mri_upload table * Modification to the profileTemplate to be able to use DTIPrep pipeline * Added get_DTI_Site_CandID_Visit function in profile file * Cosmetic changes * Cosmetic changes * Fixed typos and indentation * Correct spelling mistakes * 1) added the functionality to insert the extension , 2) fixed indentation * Removed the filetype option * Changed the queries to use the prepared statements * Modified $d to $dir in profileTemplate * Remove .DS_Store file from DICOM source directory before creating new object. Other fails while reading content of dcm source directory * Modified getSubjectIDs so that it uses PatientName instead of PatientID lookupCenterID using patient name instead of PatientID * modified: profileTemplate * Moved reference to ~/.neurodb to .loris_mri directory * Some more files * Replaced .neurodb by .loris_mri in updateHeaders.pl * Removed the full path * DicomTar.pl should use proper $profile name instead of prod * Sometime DICOM Header may not have Invertion Time field (0018,9079). Set Inversion Time to 0 in tarchive_series to avoid error message: DBD::mysql::st execute failed: Data truncated for column 'InversionTime' * DICOM HEADER may not have Inversion time (0018,9079) set which cause tarchives_series sql table who expect a double to receive an empty string. * THIS IS A DUPLICATE from opionnalProfileFixInDicomTar pull request please ignore change in dicomTar.pl THIS IS A DUPLICATE from opionnalProfileFixInDicomTar pull request please ignore change in dicomTar.pl * removed useless numbers * Added $subjectID{'createVisitLabel'} option to profile template. * Changed $subjectID{'createVisitLabel'} type to boolean * added exit 0 at the end of the file * cosmetic change * Added a new option config $bin_dir * Fixed the option * Defined(@array) is deprecated. Removed error message on the console defined(@array) is deprecated at... (Maybe you should just omit the defined()?) * Removed all occurences of defined(@array). * replaced the sourcelocation column by DecompressedLocation * Added functionality to insert PET DICOMs in tarchive tables. * Fixed indentation in the pull request. * NIfTI support of the imaging pipeline (profileTemplate change). * Added -centerName option in call to dicomTar.pl. * Made ArchiveLocation relative in tarchive table. * Created a MySQL scripted patch to make ArchiveLocation in the tarchive table relative. * Added functionality to print query on one line. * Cleaned up and prepared statement for queries in main directory of dicom-archive-tools * Prepared mysql statement for DCMSUM.pm * Fixed bug inserted while integrating mysql prepare statements * Removed the Deprecated definition of array: !@Settings instead of !defined @Settings, to be in line with pull request #95 * One more consistency check on verbose and notification spool tables * One more explicit Done message * Fixes the insert in DCMSUM.pm so it does not set the creatinguser field in the tarchive table to NULL when the pipeline is run in auto-launch mode. * site or rather, study, no longer hard coded * removed extra ~ and the comments as per Cecile feedback * adding TarchiveSeriesID to the tarchive_files * Added the getSNRModalities function * added flair * typo * typo AGAIN * added comment based on SeriesUID and EchoTime * Removed getIncomingDir routine from profileTemplate; it is no longer used. Same as PR35 * Revert "adding TarchiveSeriesID to the tarchive_files" * isFileToBeRegisteredGivenProtocol in Profile now returns 1 if ne unknown * Incorporate comment of Dave * I cna not type wihtout making typos * re-adding TarchiveSeriesID to the tarchive_files * swap inserts * added comment * updated the comment with tarchive_files * Loris-MRI configurations moved to the front end from the $profile file: Redmine 8893 (#45) * Loris-MRI configurations moved to the front end from the $profile file: Redmine 8893 * went through all the files that get the settings from the prod file * removed prefix from the Get_DTI_Site function * consistency in DB versus NeuroDB * one dollar sign instead of 2; but have no clue why * Cecile required changes * Remove all .DS_Store files anywhere in the tree, and __MACOSX directory (#46) * Remove all .DS_Store files anywhere in the tree * The command will not complain even if it does not find a file to delete. * Incorporate the additional changes from Mouna * Do not store PatientName header in notification_spool table:Redmine 13506 * update README * Check consistent upload_id and filename when running the pipeline: Redmine 13507 (#202) * Check consistent upload_id and filename when running the pipeline: Redmine 13507 * make copy and paste a bit more intelligent * Mapping empty date of birth and scan to undef when empty so that it can be inserted in MySQL 5.7 (#205) * Creected code that tests if two float numbers are equal (#206) * [Cleanup]: Remove duplicate DBI.pm; Redmine 13613 (#207) * Add TarchiveID to mri_protocol_violated_scans (#211) * Adding TarchiveID information in mri_protocol_violated_scans table * Cleaned up commented parts * Renamed $tarchiveInfo to $tarchiveInfoRef for consistencies * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 (#203) * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 * change xargs to deleting from the find command * Dave and Nick feedback * Nicolas feedback; escaping the directory name * Refactor smart matching code in DTI.pm (#215) * Better MINC toolkit sourcing command in README and updated VERSION file (#260) * Better MINC toolkit sourcing command and updated VERSION file * cecile feedback * Updating QC tables if the -deleteQCdata flag is not set. Will update FileID to NULL in files_qcstatus and feedback_mri_comments (#261) * README has also within it LORIS-MRI version number, so update (#268) * Installer: Populate the default LORIS config settings for the MRI paths (Redmine 13915) (#272) * Populate Loris-MRI code path for new installs since the user is asked to enter the MRI project name * Populate other MRI data path with a better option than the default LORIS %PROJECTNAME * post installation checks * Add the condition Dave brought up during the imaging meeting; i.e. only change it for those that did not do it in the front end already * Ceciles feedback * oopsie
nicolasbrossard
pushed a commit
that referenced
this pull request
Feb 27, 2018
* Mapping empty date of birth and scan to undef when empty so that it can be inserted in MySQL 5.7 (#205) * Creected code that tests if two float numbers are equal (#206) * [Cleanup]: Remove duplicate DBI.pm; Redmine 13613 (#207) * Add TarchiveID to mri_protocol_violated_scans (#211) * Adding TarchiveID information in mri_protocol_violated_scans table * Cleaned up commented parts * Renamed $tarchiveInfo to $tarchiveInfoRef for consistencies * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 (#203) * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 * change xargs to deleting from the find command * Dave and Nick feedback * Nicolas feedback; escaping the directory name * Refactor smart matching code in DTI.pm (#215) * Better MINC toolkit sourcing command in README and updated VERSION file (#260) * Better MINC toolkit sourcing command and updated VERSION file * cecile feedback * Updating QC tables if the -deleteQCdata flag is not set. Will update FileID to NULL in files_qcstatus and feedback_mri_comments (#261) * README has also within it LORIS-MRI version number, so update (#268) * Installer: Populate the default LORIS config settings for the MRI paths (Redmine 13915) (#272) * Populate Loris-MRI code path for new installs since the user is asked to enter the MRI project name * Populate other MRI data path with a better option than the default LORIS %PROJECTNAME * post installation checks * Add the condition Dave brought up during the imaging meeting; i.e. only change it for those that did not do it in the front end already * Ceciles feedback * oopsie
nicolasbrossard
pushed a commit
that referenced
this pull request
Feb 28, 2018
…d resolve conflicts (#280) * Initial import of files from http://code.google.com/p/dicom-archive/ * add two configurable options is_site and is_sge * Added new getIncomingDir config function * Added missing semi-colons. * Modifies the temporairy values of $data_dir, $get_dicom_info and $mail_user variables * Places back the forward slash in the email-address * Modifies the temporairy value for the tarchive_librairy location * Modified the tarchivelibdir temp value * Set the if_sge to false by default * Fixed the dcmodify --insert-tag option to --insert (option used in more recent dcmodify). * Updates the mri_upload table * Includes option to update the mri_upload table * Modification to the profileTemplate to be able to use DTIPrep pipeline * Added get_DTI_Site_CandID_Visit function in profile file * Cosmetic changes * Cosmetic changes * Fixed typos and indentation * Correct spelling mistakes * 1) added the functionality to insert the extension , 2) fixed indentation * Removed the filetype option * Changed the queries to use the prepared statements * Modified $d to $dir in profileTemplate * Remove .DS_Store file from DICOM source directory before creating new object. Other fails while reading content of dcm source directory * Modified getSubjectIDs so that it uses PatientName instead of PatientID lookupCenterID using patient name instead of PatientID * modified: profileTemplate * Moved reference to ~/.neurodb to .loris_mri directory * Some more files * Replaced .neurodb by .loris_mri in updateHeaders.pl * Removed the full path * DicomTar.pl should use proper $profile name instead of prod * Sometime DICOM Header may not have Invertion Time field (0018,9079). Set Inversion Time to 0 in tarchive_series to avoid error message: DBD::mysql::st execute failed: Data truncated for column 'InversionTime' * DICOM HEADER may not have Inversion time (0018,9079) set which cause tarchives_series sql table who expect a double to receive an empty string. * THIS IS A DUPLICATE from opionnalProfileFixInDicomTar pull request please ignore change in dicomTar.pl THIS IS A DUPLICATE from opionnalProfileFixInDicomTar pull request please ignore change in dicomTar.pl * removed useless numbers * Added $subjectID{'createVisitLabel'} option to profile template. * Changed $subjectID{'createVisitLabel'} type to boolean * added exit 0 at the end of the file * cosmetic change * Added a new option config $bin_dir * Fixed the option * Defined(@array) is deprecated. Removed error message on the console defined(@array) is deprecated at... (Maybe you should just omit the defined()?) * Removed all occurences of defined(@array). * replaced the sourcelocation column by DecompressedLocation * Added functionality to insert PET DICOMs in tarchive tables. * Fixed indentation in the pull request. * NIfTI support of the imaging pipeline (profileTemplate change). * Added -centerName option in call to dicomTar.pl. * Made ArchiveLocation relative in tarchive table. * Created a MySQL scripted patch to make ArchiveLocation in the tarchive table relative. * Added functionality to print query on one line. * Cleaned up and prepared statement for queries in main directory of dicom-archive-tools * Prepared mysql statement for DCMSUM.pm * Fixed bug inserted while integrating mysql prepare statements * Removed the Deprecated definition of array: !@Settings instead of !defined @Settings, to be in line with pull request #95 * One more consistency check on verbose and notification spool tables * One more explicit Done message * Fixes the insert in DCMSUM.pm so it does not set the creatinguser field in the tarchive table to NULL when the pipeline is run in auto-launch mode. * site or rather, study, no longer hard coded * removed extra ~ and the comments as per Cecile feedback * adding TarchiveSeriesID to the tarchive_files * Added the getSNRModalities function * added flair * typo * typo AGAIN * added comment based on SeriesUID and EchoTime * Removed getIncomingDir routine from profileTemplate; it is no longer used. Same as PR35 * Revert "adding TarchiveSeriesID to the tarchive_files" * isFileToBeRegisteredGivenProtocol in Profile now returns 1 if ne unknown * Incorporate comment of Dave * I cna not type wihtout making typos * re-adding TarchiveSeriesID to the tarchive_files * swap inserts * added comment * updated the comment with tarchive_files * Loris-MRI configurations moved to the front end from the $profile file: Redmine 8893 (#182) * Loris-MRI configurations moved to the front end from the $profile file: Redmine 8893 * went through all the files that get the settings from the prod file * added the install scripts * removed prefix from the Get_DTI_Site function * added data_dir as projects like IBIS could have it different from mincPath * Greg feedback on DB::DBI, removal of defined, and a masked variable declaration * added the script for existing projects to populate their default database values with those from their prod file * function return value * IBIS has all sorts of differences, the updates in this commit account for those * data_dir renamed to dataDirBasepath * Cecile feedback and applying her comments to all other files * fixed the message where the values is kept at its default, and removed debug print * Loris-MRI configurations moved to the front end from the $profile file: Redmine 8893 (#45) * Loris-MRI configurations moved to the front end from the $profile file: Redmine 8893 * went through all the files that get the settings from the prod file * removed prefix from the Get_DTI_Site function * consistency in DB versus NeuroDB * one dollar sign instead of 2; but have no clue why * Cecile required changes * [ImagingUpload.pm] Look inside file type and not file path for the word dicom (#178) * Look inside file type and not file path for the word dicom * check for the whole dicom type string * spaces * spaces * Fix file count in mri_upload when deleting a minc file and then re-inserting (#180) * fixed minc deletion (#185) * Remove all .DS_Store files anywhere in the tree, and __MACOSX directory (#46) * Remove all .DS_Store files anywhere in the tree * The command will not complain even if it does not find a file to delete. * Incorporate the additional changes from Mouna * Removed files starting with . and __MACOSX directory from the uploaded scans: Redmine 11670 (#186) * started * first attempt finished * cleanup * included mounas feedback * Updating README with proper dpkg (#190) * [Installation] Fixing more installation errors (#191) * Fixing more errors * This will be much clearer * changed new_nmi to nmi * missed one :p * [SQL] Fixing MySQL 5.7 group by during visit creation (#192) * Fixing MySQL 5.7 group by during visit creation * Missed comma * Removing newVisitNo in group by * Added option to also re-insert deleted minc files (Same as PR#179) (#193) * added option to also re-insert * Remove profile changes * removing IBIS specific fields * No need to show another example of sql query * adding documentation and making sql more generic * more updates * fix path to run from mri directory * Option to re-insert deleted minc files: Based off of PR 179 * remove the original file * better readme * Cecile comments * relative to example scripts directory * Modified the deletemincsqlwrapper.pl so that when no files are found with the query, it prints it out so that the user do not feel like nothing happened. * revert to t1 from adniT1 * remove the wrong file * minor thing * chmod, run tarchiveLoader without seriesUID as an argument, and small messages change * [Install] Readme Version update, installer CentOS improvements (#188) * committing Readme version and installer changes * adding Readme changes * setting VERSION file to 18.0.0 * additional Readme updates * formatting in Readme * re-integrating updates * removing period * New installs now also have their Imaging Pipeline ConfigSettings populated based on the user answers during the installation process: Redmine 13199 * minor echo message change * rebasing removed Christine changes so add them again * update the dicom archive repo link (#199) Update submodule link for dicom_archive. * Do not store PatientName header in notification_spool table:Redmine 13506 * update README * Check consistent upload_id and filename when running the pipeline: Redmine 13507 (#202) * Check consistent upload_id and filename when running the pipeline: Redmine 13507 * make copy and paste a bit more intelligent * Mapping empty date of birth and scan to undef when empty so that it can be inserted in MySQL 5.7 (#205) * Creected code that tests if two float numbers are equal (#206) * [Cleanup]: Remove duplicate DBI.pm; Redmine 13613 (#207) * Add TarchiveID to mri_protocol_violated_scans (#211) * Adding TarchiveID information in mri_protocol_violated_scans table * Cleaned up commented parts * Renamed $tarchiveInfo to $tarchiveInfoRef for consistencies * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 (#203) * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 * change xargs to deleting from the find command * Dave and Nick feedback * Nicolas feedback; escaping the directory name * Refactor smart matching code in DTI.pm (#215) * Better MINC toolkit sourcing command in README and updated VERSION file (#260) * Better MINC toolkit sourcing command and updated VERSION file * cecile feedback * Updating QC tables if the -deleteQCdata flag is not set. Will update FileID to NULL in files_qcstatus and feedback_mri_comments (#261) * README has also within it LORIS-MRI version number, so update (#268) * Installer: Populate the default LORIS config settings for the MRI paths (Redmine 13915) (#272) * Populate Loris-MRI code path for new installs since the user is asked to enter the MRI project name * Populate other MRI data path with a better option than the default LORIS %PROJECTNAME * post installation checks * Add the condition Dave brought up during the imaging meeting; i.e. only change it for those that did not do it in the front end already * Ceciles feedback * oopsie * resolve some unresolved conflicts and remove all added empty lines * Ceciles feedback to remove duplicate getconfigsetting * feedback after discussing with Nick
cmadjar
pushed a commit
to cmadjar/Loris-MRI
that referenced
this pull request
Jun 28, 2018
* Mapping empty date of birth and scan to undef when empty so that it can be inserted in MySQL 5.7 (aces#205) * Creected code that tests if two float numbers are equal (aces#206) * [Cleanup]: Remove duplicate DBI.pm; Redmine 13613 (aces#207) * Add TarchiveID to mri_protocol_violated_scans (aces#211) * Adding TarchiveID information in mri_protocol_violated_scans table * Cleaned up commented parts * Renamed $tarchiveInfo to $tarchiveInfoRef for consistencies * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 (aces#203) * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 * change xargs to deleting from the find command * Dave and Nick feedback * Nicolas feedback; escaping the directory name * Refactor smart matching code in DTI.pm (aces#215) * Better MINC toolkit sourcing command in README and updated VERSION file (aces#260) * Better MINC toolkit sourcing command and updated VERSION file * cecile feedback * Updating QC tables if the -deleteQCdata flag is not set. Will update FileID to NULL in files_qcstatus and feedback_mri_comments (aces#261) * README has also within it LORIS-MRI version number, so update (aces#268)
cmadjar
added a commit
that referenced
this pull request
Jan 21, 2019
* Forward changes merged (today) into 19.0-dev to major (#209) * Mapping empty date of birth and scan to undef when empty so that it can be inserted in MySQL 5.7 (#205) * Creected code that tests if two float numbers are equal (#206) * [Cleanup]: Remove duplicate DBI.pm; Redmine 13613 (#207) * A first pass at creating this script * Using the MRIProcessingUtility to determine center name, candidate ID ... * few additions. Back up time before the weekend!! * Update major with the latest 19.0-dev branch changes (#265) * Mapping empty date of birth and scan to undef when empty so that it can be inserted in MySQL 5.7 (#205) * Creected code that tests if two float numbers are equal (#206) * [Cleanup]: Remove duplicate DBI.pm; Redmine 13613 (#207) * Add TarchiveID to mri_protocol_violated_scans (#211) * Adding TarchiveID information in mri_protocol_violated_scans table * Cleaned up commented parts * Renamed $tarchiveInfo to $tarchiveInfoRef for consistencies * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 (#203) * Get file list after removal of undesired directories like __MACOSX: Redmine 13508 * change xargs to deleting from the find command * Dave and Nick feedback * Nicolas feedback; escaping the directory name * Refactor smart matching code in DTI.pm (#215) * Better MINC toolkit sourcing command in README and updated VERSION file (#260) * Better MINC toolkit sourcing command and updated VERSION file * cecile feedback * Updating QC tables if the -deleteQCdata flag is not set. Will update FileID to NULL in files_qcstatus and feedback_mri_comments (#261) * README has also within it LORIS-MRI version number, so update (#268) * Some more code * Finalized a first working version of the file. Need to properly log and exit. More input checking and parsing metadata are required as well * Modified file_insertion.pl based on discussions from the imaging meeting * Implemented log file, notification and temporary exit codes * fixed some notifications * Checking that the uploadID is valid * Nicolas's feedback * added the exit codes from NeuroDB::ExitCodes documented the script using Perldoc added support to read a JSON file to insert metadata into parameter_file. Also fixed a bug in MRIProcessingUtility.pm introduced by a conflict when merging latest version of aces/minor renamed file_insertion.pl to imaging_non_minc_insertion.pl modified the notifications to use imaging non minc file insertion forgot to rename the .md file to imaging_non_minc_insertion.md also missed renaming the script in mass_perldoc script updated documentation and help section of the script with the correct name of the script * Integrated Nicolas' improvement feedback * Mélanie's feedback * Adding the cpan install Jll JSON command to the install script * Nicolas' feedback
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Make sure to source the patch in PR #3354 to create and populate the column TarchiveID in the mri_protocol_violated_scans table.
Updates to the insertion scripts so that the new TarchiveID field of mri_protocol_violated_scans table can be populated when inserting a violated scan.
NOTE: this is for the next minor release of LORIS