Testing the scoring system using the ground-truth TRK data

[aritche]

Hi,

I am trying to understand how to use the scoring system, and to keep things simple I am trying to run the system to evaluate the ground-truth .trk files.

The steps I follow are:

1. export the path, i.e.:
   export PYTHONPATH=~/Desktop/ismrm_2015_tractography_challenge_scoring
2. Run the scoring script on the CST_Left.trk ground-truth file downloaded from http://www.tractometer.org/ismrm_2015_challenge/data
   ./scripts/score_tractogram.py ~/Downloads/ISMRM_2015_Tracto_challenge_ground_truth_bundles_TRK_v2/CST_left.trk scoring_data/ results/

However, I get the following error:

WARNING:root:The number of streamlines specified in header ([0]) does not match the actual number of streamlines contained in this file (7217). The latter will be used.
WARNING:root:The number of streamlines specified in header ([0]) does not match the actual number of streamlines contained in this file (7217). The latter will be used.
WARNING:root:The number of streamlines specified in header ([0]) does not match the actual number of streamlines contained in this file (7217). The latter will be used.
Need "vox_to_ras" field to get affine with which to convert points; this is present for headers >= version 2
Traceback (most recent call last):
  File "./scripts/score_tractogram.py", line 175, in <module>
    main()
  File "./scripts/score_tractogram.py", line 168, in main
    segments_dir, base_name, args.verbose)
  File "/home/aritche/Desktop/ismrm_2015_tractography_challenge_scoring/challenge_scoring/metrics/scoring.py", line 154, in score_submission
    full_strl = [s for s in streamlines_gen]
  File "/home/aritche/Desktop/ismrm_2015_tractography_challenge_scoring/challenge_scoring/io/streamlines.py", line 94, in _get_tracts_over_grid
    str(er))
ValueError: 
------ ERROR ------

TrackVis header is malformed or incomplete.
Please make sure all fields are correctly set.

If I instead use the .tck files downloaded from the same website, I don't have any issues and the expected results are generated (f1 for everything is 0, except for CST_left which is 0.8294, valid bundles = 1, etc.).

What do I change in the .trk header files to fix this problem?

[jchoude]

Hi @aritche !

Just to make sure: how did you setup your environment to run the scorer? I feel like this might be a discrepancy in behavior between python 2 and 3. @AntoineTheb is currently finalizing PR #6 to migrate and fully support Python 3. If that's the issue, the solution should come soon.

[EmmaRenauld]

Hi @aritche !

Nearly two years later, we have made modifications! The code is now in python3, and the files are now loaded with dipy. This should solve your issues!