30 feature request we want logging #75

MarcelRosier · 2024-04-23T13:43:22Z

Review after #74 (includes changes)
Adds meta logging file for preprocessing, for each run a separatelog file is created (either at specified path or by default in the current dir with timestamp)
log example (please disregard linebreaks added by github):

console:

[INFO] 2024-04-23T15:40:24+0200: ============================ Starting preprocessing ============================
[INFO] 2024-04-23T15:40:24+0200: Logs are saved to /home/ivan_marcel/preprocessing/brainles_preprocessing_2024-04-23T15:40:24.359481.log
[INFO] 2024-04-23T15:40:24+0200: Received center modality: t1c and moving modalities: t1, t2, flair
[INFO] 2024-04-23T15:40:24+0200: --------------------------- Starting Coregistration ----------------------------
[INFO] 2024-04-23T15:40:24+0200: Coregistering 3 moving modalities to center modality...
[INFO] 2024-04-23T15:40:24+0200: Registering modality t1 (file=co__t1c__t1) to center modality...
[INFO] 2024-04-23T15:40:26+0200: Registering modality t2 (file=co__t1c__t2) to center modality...
[INFO] 2024-04-23T15:40:28+0200: Registering modality flair (file=co__t1c__flair) to center modality...
[INFO] 2024-04-23T15:40:30+0200: Coregistration complete. Output saved to /home/ivan_marcel/preprocessing/example/example_data/OtherEXampleFromTCIA/OtherEXampleFromTCIA_brainles/co-registration
[INFO] 2024-04-23T15:40:30+0200: ------------------------- Starting atlas registration --------------------------
[INFO] 2024-04-23T15:40:30+0200: Registering center modality to atlas...
[INFO] 2024-04-23T15:40:36+0200: Atlas registration complete. Output saved to /home/ivan_marcel/preprocessing/temporary_directory/atlas-space
[INFO] 2024-04-23T15:40:36+0200: Transforming 3 moving modalities to atlas space...
[INFO] 2024-04-23T15:40:36+0200: Transforming modality t1 (file=atlas__t1) to atlas space...
[INFO] 2024-04-23T15:40:37+0200: Transforming modality t2 (file=atlas__t2) to atlas space...
[INFO] 2024-04-23T15:40:38+0200: Transforming modality flair (file=atlas__flair) to atlas space...
[INFO] 2024-04-23T15:40:39+0200: Transformations complete. Output saved to /home/ivan_marcel/preprocessing/example/example_data/OtherEXampleFromTCIA/OtherEXampleFromTCIA_brainles/atlas-registration
[INFO] 2024-04-23T15:40:39+0200: ---------------------- Checking optional atlas correction ----------------------
[INFO] 2024-04-23T15:40:39+0200: Applying optional atlas correction for modality t1
[INFO] 2024-04-23T15:40:46+0200: Applying optional atlas correction for modality t2
[INFO] 2024-04-23T15:40:52+0200: Applying optional atlas correction for modality flair
[INFO] 2024-04-23T15:40:58+0200: Atlas correction complete. Output saved to /home/ivan_marcel/preprocessing/example/example_data/OtherEXampleFromTCIA/OtherEXampleFromTCIA_brainles/atlas-correction
[INFO] 2024-04-23T15:40:58+0200: Saving non skull-stripped images...
current image /home/ivan_marcel/preprocessing/temporary_directory/atlas-space/atlas__t1c.nii.gz
current image /home/ivan_marcel/preprocessing/temporary_directory/atlas-correction/atlas_corrected__t1c__t1.nii.gz
current image /home/ivan_marcel/preprocessing/temporary_directory/atlas-correction/atlas_corrected__t1c__t2.nii.gz
current image /home/ivan_marcel/preprocessing/temporary_directory/atlas-correction/atlas_corrected__t1c__flair.nii.gz
[INFO] 2024-04-23T15:41:03+0200: ---------------------- Checking optional brain extraction ----------------------
[INFO] 2024-04-23T15:41:03+0200: Starting brain extraction...
[INFO] 2024-04-23T15:41:03+0200: Extracting brain region for center modality...
File: /home/ivan_marcel/preprocessing/temporary_directory/atlas-space/atlas__t1c.nii.gz
preprocessing...
image shape after preprocessing: (103, 160, 160)
prediction (CNN id)...
0
1
2
3
4
exporting segmentation...
[INFO] 2024-04-23T15:41:21+0200: Applying brain mask to t1...
[INFO] 2024-04-23T15:41:22+0200: Applying brain mask to t2...
[INFO] 2024-04-23T15:41:22+0200: Applying brain mask to flair...
[INFO] 2024-04-23T15:41:23+0200: Brain extraction complete. Output saved to /home/ivan_marcel/preprocessing/example/example_data/OtherEXampleFromTCIA/OtherEXampleFromTCIA_brainles/brain-extraction
[INFO] 2024-04-23T15:41:23+0200: Saving skull-stripped images...
current image /home/ivan_marcel/preprocessing/temporary_directory/brain-extraction/atlas_bet_t1c.nii.gz
current image /home/ivan_marcel/preprocessing/temporary_directory/brain-extraction/brain_masked/brain_masked__t1.nii.gz
current image /home/ivan_marcel/preprocessing/temporary_directory/brain-extraction/brain_masked/brain_masked__t2.nii.gz
current image /home/ivan_marcel/preprocessing/temporary_directory/brain-extraction/brain_masked/brain_masked__flair.nii.gz
[INFO] 2024-04-23T15:41:25+0200: ============================ Preprocessing complete ============================

file:

[INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor [INFO | preprocessor | L217 ] 2024-04-23T15:38:38+0200: ============================ Starting preprocessing ============================
| L218 ] 2024-04-23T15:38:38+0200: Logs are saved to /home/ivan_marcel/preprocessing/brainles_preprocessing_2024-04-23T15:38:38.564688.log
| L219 ] 2024-04-23T15:38:38+0200: Received center modality: t1c and moving modalities: t1, t2, flair
| L223 ] 2024-04-23T15:38:38+0200: --------------------------- Starting Coregistration ----------------------------
| L228 ] 2024-04-23T15:38:38+0200: Coregistering 3 moving modalities to center modality...
| L233 ] 2024-04-23T15:38:38+0200: Registering modality t1 (file=co__t1c__t1) to center modality...
| L233 ] 2024-04-23T15:38:52+0200: Registering modality t2 (file=co__t1c__t2) to center modality...
| L233 ] 2024-04-23T15:39:05+0200: Registering modality flair (file=co__t1c__flair) to center modality...
| L255 ] 2024-04-23T15:39:19+0200: Coregistration complete. Output saved to /home/ivan_marcel/preprocessing/example/example_data/TCGA-DU-7294/TCGA-DU-7294_brainles/co-registration
| L260 ] 2024-04-23T15:39:19+0200: ------------------------- Starting atlas registration --------------------------
| L261 ] 2024-04-23T15:39:19+0200: Registering center modality to atlas...
| L269 ] 2024-04-23T15:39:27+0200: Atlas registration complete. Output saved to /home/ivan_marcel/preprocessing/temporary_directory/atlas-space
| L272 ] 2024-04-23T15:39:27+0200: Transforming 3 moving modalities to atlas space...
| L277 ] 2024-04-23T15:39:27+0200: Transforming modality t1 (file=atlas__t1) to atlas space...
| L277 ] 2024-04-23T15:39:29+0200: Transforming modality t2 (file=atlas__t2) to atlas space...
| L277 ] 2024-04-23T15:39:31+0200: Transforming modality flair (file=atlas__flair) to atlas space...
| L291 ] 2024-04-23T15:39:34+0200: Transformations complete. Output saved to /home/ivan_marcel/preprocessing/example/example_data/TCGA-DU-7294/TCGA-DU-7294_brainles/atlas-registration
| L296 ] 2024-04-23T15:39:34+0200: ---------------------- Checking optional atlas correction ----------------------
| L302 ] 2024-04-23T15:39:34+0200: Applying optional atlas correction for modality t1
| L302 ] 2024-04-23T15:39:40+0200: Applying optional atlas correction for modality t2
| L302 ] 2024-04-23T15:39:47+0200: Applying optional atlas correction for modality flair
| L323 ] 2024-04-23T15:39:54+0200: Atlas correction complete. Output saved to /home/ivan_marcel/preprocessing/example/example_data/TCGA-DU-7294/TCGA-DU-7294_brainles/atlas-correction
| L333 ] 2024-04-23T15:39:54+0200: Saving non skull-stripped images...
| L346 ] 2024-04-23T15:40:01+0200: ---------------------- Checking optional brain extraction ----------------------
| L351 ] 2024-04-23T15:40:01+0200: Starting brain extraction...
| L356 ] 2024-04-23T15:40:01+0200: Extracting brain region for center modality...
| L361 ] 2024-04-23T15:40:19+0200: Applying brain mask to t1...
| L361 ] 2024-04-23T15:40:20+0200: Applying brain mask to t2...
| L361 ] 2024-04-23T15:40:21+0200: Applying brain mask to flair...
| L374 ] 2024-04-23T15:40:21+0200: Brain extraction complete. Output saved to /home/ivan_marcel/preprocessing/example/example_data/TCGA-DU-7294/TCGA-DU-7294_brainles/brain-extraction
| L381 ] 2024-04-23T15:40:21+0200: Saving skull-stripped images...
| L393 ] 2024-04-23T15:40:23+0200: ============================ Preprocessing complete ============================

add usage of log files for each run

comment unused registrator imports in example

MarcelRosier added 7 commits April 23, 2024 09:34

rm temp dir

85210df

add: basic logging setup and calls

657f854

improved log messages and structure

ef482fc

simplify console log format

2a56513

add usage of log files for each run

fix wrong docstring

7daaf99

add option to specify custom logfile path

f40bbb6

comment unused registrator imports in example

add wrapper to ensure log handler is removed between runs

7b7efda

MarcelRosier requested a review from IsraMekki0 April 23, 2024 13:43

MarcelRosier self-assigned this Apr 23, 2024

MarcelRosier linked an issue Apr 23, 2024 that may be closed by this pull request

[feature request] we want logging #30

Closed

MarcelRosier requested a review from neuronflow April 23, 2024 13:43

neuronflow added 2 commits April 23, 2024 15:45

Merge branch 'main' into 30-feature-request-we-want-logging

c4a0217

Merge branch 'main' into 30-feature-request-we-want-logging

8251522

neuronflow approved these changes Jun 17, 2024

View reviewed changes

neuronflow enabled auto-merge June 17, 2024 16:07

neuronflow added the enhancement New feature or request label Jun 17, 2024

neuronflow merged commit 3f8ea1d into main Jun 17, 2024

neuronflow deleted the 30-feature-request-we-want-logging branch June 17, 2024 16:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

30 feature request we want logging #75

30 feature request we want logging #75

Uh oh!

MarcelRosier commented Apr 23, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

30 feature request we want logging #75

30 feature request we want logging #75

Uh oh!

Conversation

MarcelRosier commented Apr 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MarcelRosier commented Apr 23, 2024 •

edited

Loading