Part 2: Calculating Features

Calculating Features

In the home page we identified six elements to this process. The step is described in Part 1.

1. It is up the user to give a best effort of cropping, zooming and rotating the various experiments by hand, then placing the data into the folder in the right format

---

Handled in this section:

2. The images volume is partitioned into subvolumes for parallelization and robustness checking in later steps.
3. Keypoints and feature descriptors are calculated per subvolume

*** Explained in [Step 3 link](https://github.com/dgoodwin208/Registration/wiki/Part-3:-Calculating-Registration)

4. Correspondence calculations per subvolume
5. Affine transformation are calculated on random subsets of the correspondences
6. Non-linear transform applied across the entire sample using the validated correspondences

Step 3: Partitioning the Data

This is automatically part of the CalculateDescriptorsForTileAtIndices.m function, but describing it in detail for description of the method.

The code in this repository treats subvolumes independently until the final step of calculating the Thin Plate Spline. This is done both for parallelization (explained below ) and for calculating affine transforms to validate quality of keypoint correspondences.

The uniform partitions are defined in loadExperimentParams.m in two sets as params.COLS_DESC and params.COLS_TFORM. The DESC variables This allows the user to change the number of pieces under which the descriptors are calculated to vary the parallelism, and the TFORM variables are used to vary the resolution of the affine transforms. That is, if the TFORM parameters are too course, it might not be possible to find suitable affine transforms and so many legitimate point correspondences may be erroneously thrown out. Alternatively, if the user sets the TFORM params too fine, there might not be enough keypoints to properly assess good correspondences.

By default, the values in loadExperimentParams.m are set as the following, which worked fine for the images in Murray et al 2015. These images were on the order of 4000x3000x100 voxels.

%how many subsections to calculate the descriptors?
params.ROWS_DESC = 5;
params.COLS_DESC = 5;
%over how many subsections to calculate the piecewise affine transforms?
params.ROWS_TFORM = 5;
params.COLS_TFORM = 5;

Step 4: Calculating Keypoints and Descriptors

If you call CalculateDescriptorsForTileAtIndices(x) it should just work! (where x is the index of the particular experiment you want to process). This calculates the 3D Harris Corner keypoints and the 3D SIFT Descriptors for experiment x's fluorescence channel, as specified by params.REGISTERCHANNEL.

If you want to calculate the descriptors on a subset of the subvolumes, either refer to the documentation or the parallelization section below. As a benchmark, a sample image of 1000x1000x132 took about 3 hours to complete using the example_parfor.m script on a 2012 MacBook Pro with 4 cores and 16GB RAM.

The primary work is done by SWITCH_tile_processing.m. It identifies keypoints via Harris3D.m and the 3D SIFT descriptors using Paul Scovanner's 3DSIFT Code.

Output

Because each subsection of the data can take many hours to calculate, CalculateDescriptorsForTileAtIndices.m saves the descriptors into the OUTPUT folder in the format of [y_min]_[y_max]_[x_min]_[x_max].mat of the subvolume processed. For example: 739-924_1-208.mat` could be a bottom left section of some z_stack.

Parallelization:

It is computationally expensive to calculate the 3D Harris Corners keypoints and to identify the most distinctive points via 3D SIFT. Fortunately, the process is embarrassingly parallel. The CalculateDescriptorsForTileAtIndices(run_num,[start_idx,end_idx]) function allows you to specify which of the subvolumes you'd like to operate on. This allows for parallelization to occur via your method of choice - such as on a cluster with bash scripts or using MATLAB's Parallel Toolbox. If you do have access

parpool(); % spawn MATLAB parallel pool with the max number of cores
loadExperimentParams; % management file for the code system
experiment_number_to_process = 1; % pick your experimental round to process
parfor i = 1:params.ROWS_DESC*params.COLS_DESC
    CalculateDescriptorsForTileAtIndices(experiment_number_to_process ,i,i);
end