Skip to content
/ DFV Public

[CVPR'22] DFVDFF: Deep Depth from Focus with Differential Focus Volume


Notifications You must be signed in to change notification settings


Repository files navigation

DFVDFF: Deep Depth from Focus with Differential Focus Volume

This is a PyTorch implementation of our CVPR-2022 paper:

Deep Depth from Focus with Differential Focus Volume

Fengting Yang, Xiaolei Huang, and Zihan Zhou

Please feel free to contact Fengting Yang ( if you have any questions.


The code is developed and tested with

  • Python 3.6
  • Pytorch 1.0.0 (w/ Cuda 10.0) and 1.6.0 (w/ Cuda 10.2)
  • More details are available in requirements.txt

Data Preparation


The data used in our experiment are from FoD500, DDFF-12, and Mobile Depth.

  • For FoD500, we directly use the original raw data without pre-processing.
  • For DDFF12, we provide our pre-processed data here. If you prefer to generate the training and validation split by yourself. Please download the Lightfield (24.5GB) and Depth (57.9MB) from the website, and follow the instruction in the next section to prepare the train and validation set. The DDFF-12 test set is only needed if you wish to submit your test result to the leaderboard. You can directly use the pre-processed test set at the ddff-pytorch repository.
  • For Mobile Depth, we need to reorganize the files. Please follow the steps shown in the next section. Note that no ground truth is provided in this dataset, and we only use it for qualitative evaluation.


For FoD500 dataset, no data pre-processing is needed.

For DDFF-12 dataset, please first modify the data_pth and out_pth in data_preprocess/ and then run

python data_preprocess/ 

to get the focal stack images. The path variables must be corrected, according to your data location. Next, run

python data_preprocess/ --img_folder <OUTPUT FORM MY_REFOCUS> --dpth_folder <PATH_TO_DEPTH> --outfile <PATH_TO H5 OUTPUT>

This will generate a .h5 file for training and validation. The reason we do not use the official training and validation split is that some stacks in their validation set are actually from the same scene included in their training set. We wish no scene overlapping between training and validation set for a more accurate validation.

For Mobile depth dataset, please modify the path variables in data_preprocess/ and then run it.

python data_preprocess/ 


Given the DDFF-12 h5.file in <DDFF12_PTH>, and FoD data folder in <FOD_PTH>, please run

CUDA_VISIBLE_DEVICES=0 python --stack_num 5 --batchsize 20 --DDFF12_pth <DDFF12_PTH> --FoD_pth <FOD_PTH> --savemodel <DUMP_PTH>  --use_diff 0/1

to train the model. --use_diff 0 refers to the simple focus volume model (Ours-FV), and --use_diff 1 corresponds to the differential focus volume model (Ours-DFV). We have shared Our-FV and Our-DFV checkpoint pre-trained on the FoD500 and DDFF-12 training set. Please note this is not the final model for our DDFF-12 submission which we also include the DDFF-12 validation set in the training.



To evaluate on the DDFF-12 validation set, run

python --stack_num 5 --loadmodel <CKPT_PTH> --data_path  <DDFF12_PTH> --use_diff 0/1

The number generate at the end shows the metrics mse, rms, log_rms, abs_rel, sq_rel, a1, a2, a3, Bump., avgUnc. in order. Please check the DDFF-12 dataset paper for their meaning, except avgUnc. which is introduced by us to evaluate the network uncertainty to its prediction.

Also if you are not using our pre-trained checkpoint, please comment the following lines in We add these lines at the paper submssion for the reviewer to better reproduce our results.


Lines 37 to 41 in 3744207

# !!! Only for users who download our pre-trained checkpoint, comment the next four line if you are not !!!
if os.path.basename(args.loadmodel) == 'DFF-DFV.tar' :
args.use_diff = 1
args.use_diff = 0

For website submission or visualization on the test set.

python --data_path <TEST_SET_PTH> --loadmodel <CKPT_PTH> --use_diff 0/1 --outdir <DUMP_PTH>


To generate test results, run

python --data_path <FOD_PTH> --loadmodel <CKPT_PTH> --use_diff 0/1 --outdir <FOD_DUMP_PTH>

The code will also provide the avgUnc. result on FoD500. Next, the evaluation results can be generated by running

python --res_path <FOD_DUMP_PTH>

Mobile depth

To generate qualitative results, run

python --data_path <FOD_PTH> --loadmodel <CKPT_PTH> --use_diff 0/1 --outdir <FOD_DUMP_PTH>


Parts of the code are developed from HSM, DDFF-pytorch, DDFF-toolbox and DefocusNet.


[CVPR'22] DFVDFF: Deep Depth from Focus with Differential Focus Volume








No releases published