PEP8 check should be applied to all notebooks

[mingxin-zheng]

Describe the bug

Currently we use a pre-defined pattern to include/exclude notebooks in the CI/CD.

tutorials/runner.sh

Line 73 in f87bb12

pattern="-and -name '*' -and ! -wholename '*federated_learning*'\

Because some notebooks need user inputs (e.g. preparing a dataset manually), we should not use papermill to automate the execution in the CI/CD. However, the pattern also allows these notebook to skip the PEP8 check, which I think should be applied:

tutorials/runner.sh

Line 225 in f87bb12

files=($(echo $pattern | xargs find . -type f -name "*.ipynb" -and ! -wholename "*.ipynb_checkpoints*"))

Suggested Solution

Add a skip_run_papermill pattern to skip the notebooks that's not intended to be part of CI/CD.

# run papermill
if [ $doRun = true ]; then

	skipRun=false
	
	for skip_pattern in "${skip_run_papermill[@]}"; do
		if [[  $file =~ $skip_pattern ]]; then
			skipRun=true
			break
		fi
	done

	if [ $skipRun = true ]; then
		break
	fi

	echo Running notebook...
	notebook=$(cat "$filename")
    

skip_run_papermill=()
skip_run_papermill=("${skip_run_papermill[@]}" .*federated_learning*)
skip_run_papermill=("${skip_run_papermill[@]}" .*transchex_openi*)
skip_run_papermill=("${skip_run_papermill[@]}" .*unetr_*)
skip_run_papermill=("${skip_run_papermill[@]}" .*profiling_train_base_nvtx*)
skip_run_papermill=("${skip_run_papermill[@]}" .*benchmark_global_mutual_information*)
skip_run_papermill=("${skip_run_papermill[@]}" .*spleen_segmentation_3d_visualization_basic*)
skip_run_papermill=("${skip_run_papermill[@]}" .*full_gpu_inference_pipeline*)
skip_run_papermill=("${skip_run_papermill[@]}" .*generate_random_permutations*)
skip_run_papermill=("${skip_run_papermill[@]}" .*transforms_update_meta_data*)
skip_run_papermill=("${skip_run_papermill[@]}" .*video_seg*)
skip_run_papermill=("${skip_run_papermill[@]}" .*tcia_dataset*)
skip_run_papermill=("${skip_run_papermill[@]}" .*hovernet_torch*)
skip_run_papermill=("${skip_run_papermill[@]}" .*preprocess_detect_scene_and_split_fold*)
skip_run_papermill=("${skip_run_papermill[@]}" .*preprocess_to_build_detection_dataset*)
skip_run_papermill=("${skip_run_papermill[@]}" .*preprocess_extract_images_from_video*)
skip_run_papermill=("${skip_run_papermill[@]}" .*transfer_mmar*)
skip_run_papermill=("${skip_run_papermill[@]}" .*MRI_reconstruction*)

Expected behavior
PEP 8 check on all notebooks

[mingxin-zheng]

Adding @wyli for visibility