the depth for RGBDSLAM dataset for InfiniTAM

[LiliMeng]

Hi,
Thanks a lot for this great code!
Could TUM RGBD dataset be used to test InfiniTAM? (after changing the rgb png format to ppm and depth png format to pgm) The result seems very wired (https://www.youtube.com/watch?v=mXt8AYPDwq4)

   Thanks a lot! 

[connerbrooks]

Looks like the intensities for the depth values are flipped, so the depth being read by InfiniTAM is the inverse of the correct depth image.

I am working on a reader that would use the association files that can be created for the TUM datasets, I'm not sure if thats something upstream would want but I think it could be useful.

[LiliMeng]

@connerbrooks Thanks a lot, could you send me a copy of that reader when it works for TUM dataset? Thanks again!

[chiwunau]

Excuse me, I have encountered a similar problem, any updates?

[connerbrooks]

I have written the majority of a reader for the TUM data sets which can be found here (requires libpng and zlib); this reader uses an association file. The issue is that the depth images are scaled meter values as described here, and I have not been able to change these values to what InfiniTAM expects.
To change how each uchar from the depth image is read into the ITMShortImage edit this line (this will also need a more generic approach that works for all image sources). Let me know if you have any ideas.

[mikesapi]

@LiliMeng You may supply png images to InfiniTAM directly.

In order to get InfiniTAM running on the TUM Dataset I had to scale the depth values by a scalar factor of (1/5000).

In InfiniTAM you may specify the transformation to apply to the depth values directly in the calibration file.
For instance, on the "freiburg1" sequences you may supply the following calibration file to InfiniTAM:

640 480
591.123473 590.076012
331.038659 234.047543

640 480
517.306408 516.469215
318.643040 255.313989

1 0 0 0
0 1 0 0
0 0 1 0

affine 0.0002 0.0

[C8PAN]

@connerbrooks I have cloned your feature/tumReader branch. I modified line 35 of the file InfiniTAM.cpp as: imageSource = new TUMFileReader(calibFile, filename1, filename2);
I download the TUM data rgbd_dataset_freiburg1_desk and created the associate.txt file. I used the calibration as @mikesapi suggested. I run the program using:
./InfiniTAM ~/Documents/data/rgbd_dataset_freiburg1_desk/calib.txt ~/Documents/data/rgbd_dataset_freiburg1_desk/associate.txt ~/Documents/data/rgbd_dataset_freiburg1_desk
But only the RGB images are show, Could you please let me know how to make it work?
Thank you.

[JackHenry1992]

hi, @mikesapi , Can you give your reader code or detail step about TUM Dataset?

[nrupatunga]

@LiliMeng @connerbrooks @sgolodetz we were trying infiniTAM on RGBD dataset as well, we weren't successful and also tried methods mentioned above in this thread. Do you have any pointers to get it running?

[zhaozhongch]

Although it has been years, I put the answer hereto run TUM rgbd dataset here in case people want to run exactly as the demo
1: Download TUM rgbd dataset, generate data that has the same format as the original demo dataset. I use Matlab script as the following

%rgb
root_dir_rgb = '/home/zhaozhong/dataset/tum/rgbd_dataset_freiburg1_desk/rgb/';
root_dir_depth = '/home/zhaozhong/dataset/tum/rgbd_dataset_freiburg1_desk/depth/';

target_dir_rgb = '/home/zhaozhong/dataset/tum/rgbd_dataset_freiburg1_desk/ppm_rgb/';
target_dir_depth = '/home/zhaozhong/dataset/tum/rgbd_dataset_freiburg1_desk/pgm_depth/';

imagefiles_rgb = dir(strcat(root_dir_rgb,'*.png'));     
imagefiles_depth = dir(strcat(root_dir_depth,'*.png')); 

nfiles_rgb = length(imagefiles_rgb);    % Number of files found
nfiles_depth = length(imagefiles_depth);

for ii=1:nfiles_rgb
   currentfilename = imagefiles_rgb(ii).name;
   output_name = '';
   if ii > 1000
       output_name = sprintf('%d.ppm', ii-1);
   elseif ii>100
       output_name = strcat('0',sprintf('%d.ppm', ii-1));
   elseif ii>10
       output_name = strcat('00',sprintf('%d.ppm', ii-1));
   elseif ii>0
       output_name = strcat('000',sprintf('%d.ppm', ii-1));
   end
   display(ii);
   currentimage = imread(strcat(root_dir_rgb,currentfilename));
   imwrite(currentimage, strcat(target_dir_rgb,output_name));
end

for ii=1:nfiles_depth
   currentfilename = imagefiles_depth(ii).name;
   display(ii);
   if ii > 1000
       output_name = sprintf('%d.pgm', ii-1);
   elseif ii>100
       output_name = strcat('0',sprintf('%d.pgm', ii-1));
   elseif ii>10
       output_name = strcat('00',sprintf('%d.pgm', ii-1));
   elseif ii>0
       output_name = strcat('000',sprintf('%d.pgm', ii-1));
   end
   currentimage = imread(strcat(root_dir_depth,currentfilename));
   imwrite(currentimage, strcat(target_dir_depth,output_name));
end

Change the directory to the place you store and output ppm pgm image

2: If you use TUM dataset freburgh1 as @mikesapi answer, simply use the following as the calib file

640 480
591.123473 590.076012
331.038659 234.047543

640 480
517.306408 516.469215
318.643040 255.313989

1 0 0 0
0 1 0 0
0 0 1 0

affine 0.0002 0.0

Note the rgb and the depth has been aligned so we have an identity matrix in the above file.
What's more, the meaning of affine 0.0002 0.0 can be seen in question #143 sgolodetz's answer. In short, the scale used by TUM is 5000, then we need to use 1/5000 = 0.0002 after affine.

3: Run! At where you put the executable file, using the following commands to run, just replace the path to where you put the rgb and depth.

./InfiniTAM/InfiniTAM /home/zhaozhong/dataset/tum/rgbd_dataset_freiburg1_desk/calib.txt /home/zhaozhong/dataset/tum/rgbd_dataset_freiburg1_desk/ppm_rgb/%04i.ppm /home/zhaozhong/dataset/tum/rgbd_dataset_freiburg1_desk/pgm_depth/%04i.pgm