Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
1_stage_zstats.py
2.1_dataqa.py
2.2_mk_add10000.sh
2.3.1_make_zstatmask_allgood.py
2.3_make_zstatmask.py
2.4.1_package_data_into_numpy_allgood.py
2.4_package_data_into_numpy.py
2.5_mk_smoothed_dataset.sh
2.6_make_condition_specific_means.py
2.7_make_contrast_task_list.py
2.8_make_allmean_file.py
2_create_datafiles.py
3.1.1_project_data_across_runs.sh
3.1.2_make_ICA_overlays.py
3.1_project_data_into_ICs.py
3.2_run_melodic_smoothed.py
3.3.1_project_data_across_runs_smoothed.sh
3.3.2_make_ICA_overlays.py
3.3_project_data_into_smoothed_ICs.py
3_run_melodic.py
4.1.1_run_all_randomize_wholebrain.py
4.1.2_run_all_randomize_wholebrain.sh
4.1.3_collect_all_randomize_wholebrain.py
4.1_randomize_wholebrain.py
4_classify_task_wholebrain.py
5.1_estimate_parameters_from_run2.py
5.2.1_run_all_randomize_ICA.py
5.2.1_run_all_randomize_ICA.sh
5.2.3_make_ICA_acc_figure.py
5.2_classify_task_ICA_randperm.py
5.3_make_ICA_overlays.sh
5.4_project_mean_data_into_ICA_space.sh
5.5_make_single_slice_images.py
5.6_mk_task_polar_plots.py
5_classify_task_ICA.py
6.1.1_run_all_searchlight.py
6.1.1_run_all_searchlight.sh
6.1_classify_task_searchlight_3mm.py
6.2.1_run_all_randomize_searchlight.py
6.2.1_run_all_randomize_searchlight.sh
6.2_classify_task_searchlight_randomize.py
6.3_get_max_randstats.py
6_classify_task_searchlight.py
7.1_hierarchical_clustering_melodic.py
7_hierarchical_clustering.py
8.1_tsne_wholebrain.py
8.2_mds_wholebrain.py
8.3_mds_ICA.py
8_tsne_ICA.py
9.1.1_run_all_classify_subjects_rand.py
9.1.1_run_all_classify_subjects_rand.sh
9.1.2_run_all_classify_subjects_rand.sh
9.1_classify_subjects_rand.py
9_classify_subjects.py
Makefile
README
basedir.py
classify_task_ICA_randperm.py
get_all_zstats.sh
get_best_params.py
get_contrasts_to_use.py
get_randperm_data.py
run_melodic.sh
run_melodic_smoothed.sh
run_searchlight.sh
run_tsne_wb.sh
tsne.py

README

This is the code for the revised set of openfmri analyses, performed
in Nov/Dec 2012.


1: copy all relevant zstats to a central directory (/corral-repl/utexas/poldracklab/openfmri/shared2/zstats)

1_stage_zstats.py


2: create data files for analysis:

2_create_datafiles.py

QA:
computed meants using:
fslmeants -i zstat_run1.nii.gz -m /corral-repl/utexas/poldracklab/software_lonestar/fsl-5.0.1/data/standard/MNI152_T1_2mm_brain_mask.nii.gz  -o zstat_run1_meants.txt

additional qa using 2.1_dataqa.py
- this uncovered one subject for who the zstat image was empty (ds052_subctr158_task001_run001_zstat001.nii.gz which is ds052/sub011 - had no trials for this particular condition).  because of this I moved this subject to ds052/bad - then reran everything to get newly renumbered subjects


- QA shows substantially different variability across datasets
- but there don't appear to be any bad subjects

2.2: prepare data for ICA by adding 10000 to zstat images

2.2_mk_ad10000.sh


2.3: create a mask that only includes voxels present for all subjects (allow a few to have missing data - right now threshold is 3)

2.3_make_zstatmask.py

- then combine them with the command:

fslmaths zstat_run1_goodvoxmask.nii.gz -mul zstat_run2_goodvoxmask.nii.gz goovoxmask.nii.gz

- also create version with no missing subjects:
fslmaths zstat_run1_missingcount.nii.gz -add zstat_run2_missingcount.nii.gz all_missingcount
fslmaths all_missingcount.nii.gz -thr 0.1 -bin all_badvox

fslmaths goodvoxmask.nii.gz -sub all_badvox.nii.gz -thr 0.5 -bin all_goodvox


2.4: package data into numpy npy files for classifier

2.4_package_data_into_numpy.py


2.5: make a smoothed version of the dataset

Note: sigma for smoothing of 6mm = 6/(N.sqrt(N.log(2.0)*6.0)) = 2.9421370201494326


2.6: create mean versions of each dataset for cluster analysis

3: run ICA on run1 data at various dimensionalities

3_run_melodic.py

- project data into ICs

3.1_project_data_into_ICs.py
3.1.1_project_data_across_runs.sh
(had to move these by hand to the data directory from the code directory)
3.1.2_make_ICA_overlays.py


3.2: run melodic on 6mm smoothed data

3.2_run_melodic_smoothed.py
3.3_project_data_into_ICs.py
3.3.1_project_data_across_runs.sh
3.3.2_make_ICA_overlays.py


4: run classifier on wholebrain data

4_classify_task_wholebrain.py
results saved to classifier_results_SVM.npz
SVM accuracy=93%

- perform this 1000 times using randomized labels:

4.1_randomize_wholebrain.py
4.1.1_run_all_randomize_wholebrain.py
4.1.2_run_all_randomize_wholebrain.sh - launcher script to run all 1000
4.1.3_collect_all_randomize_wholebrain.py

SVM:
mean:  0.053225
95 pct:  0.075



5: run classifier on ICA components

NOTE: was originally run on unsmoothed ICA results.  these results have been moved to 
classifier/ICA_unsmoothed - all subsequent analyses should use smoothed ICA results

5.1_estimate_parameters_from_run2.py
- uses get_best_params.py

5_classify_task_ICA.py  (use smoothed ICA components)

5.2_classify_task_ICA_randperm.py
5.2.1_run_all_randomize_ICA.py
get_randperm_data.py - function to load data
5.2.3_make_ICA_acc_figure.py (TBD)

5.4_project_mean_data_into_ICA_space.sh - take mean for each task and project into ICA space for visualization

5.5_make_single_slice_images.py - make 3axis iamges for next step

5.6_mk_task_polar_plots.py - create polar task fingerprint plots

6: run searchlight classifier

6_classify_task_searchlight.py
- run 100 times to get mean across CV foldings
- using 6.1.1_run_all_searchlight.py/sh


results from 100 runs were collapsed using fslmerge and meaned using fslmaths


for testing, created 3mm version of run1 data using:

/corral-repl/utexas/poldracklab/software_lonestar/fsl-5.0.1/bin/flirt -in /corral-repl/utexas/poldracklab/openfmri/analyses/paper_analysis_Dec2012/data_prep/zstat_run1.nii.gz -applyxfm -init /corral-repl/utexas/poldracklab/software_lonestar/fsl-5.0.1/etc/flirtsch/ident.mat -out /corral-repl/utexas/poldracklab/openfmri/analyses/paper_analysis_Dec2012/data_prep/zstat_run1_3mm.nii.gz -paddingsize 0.0 -interp trilinear -ref /corral-repl/utexas/poldracklab/software_lonestar/atlases/MNI152lin_3mm.nii.gz


6.2_classify_task_searchlight_randomize.py: run 500 times with randomized labels

6.2.1_run_all_randomize_searchlight.py/sh


6.3_get_max_randstats.py

95% max : 0.150125



7: hierarchical clustering

7_hierarchical_clustering.p - perform on full dataset
7.1_hierarchical_clustering_melodic.py - perform on melodic components 

8: low-dimensional embedding 

- t-SNE

tried lower perplexity but still saw dispersed items, so sticking with perplexity=10

9: subject classification
- train multiclass machine for subjects on run 1, test on run 2
- use mask with only voxels that are nonzero for all subjects

with goodvox mask (badsubthresh=3):
N.mean(pred==labels2)
Out[4]: 0.7556179

with allgood mask (badsubthresh=0):
In [2]: N.mean(pred==labels2)
Out[2]: 0.7528089887640449

9.1 - run with random labels


cat subject_classifier/*.txt > all_subclass_random.txt

In [2]: d=N.loadtxt('all_subclass_random.txt')
In [3]: import scipy.stats
In [4]: p=scipy.stats.scoreatpercentile(d,95)
In [5]: p
Out[5]: 0.011376404494381894
In [6]: N.mean(d)
Out[6]: 0.0045337078651685337