Repository for different network models related to flow/disparity from the following papers:
NOTE: We only provide deployment code for these networks. We do not publish any training code and also do not offer support about questions for training networks.
- Install tensorflow (1.4) (pip3 install tensorflow-gpu==1.4)
- Compile and install lmbspecialops. Please use the branch
- Install netdef_slim
- Clone this repository
- Change your directory to the network directory (Eg: FlowNet3)
- Run download_snapshots.sh. This takes a while to download all snapshots
- Now you should be ready to run the networks. Change your directory to a network type (Eg: css).
Use the following command to test the network on an image pair:
python3 controller.py eval image0_path image1_path out_dir
The networks are executed using the controller.py scripts in the respective folders. Just running this controller will produce several output files in a folder (note that you can also obtain this output just as numpy arrays and write it to some custom files; see next section).
For optical flow we use the standard
The other modalities use a custom binary format called
.float3. To read
.float3 files to numpy arrays, please use the
from netdef_slim.utils.io import read occ_file = 'occ.float3' occ_data = read(occ_file) # returns a numpy array # to visualize import matplotlib.pyplot as plt plt.imshow(occ_data[:,:,0])
The eval method of the controller writes to the disk by default.
To avoid writing to disk, create a Controller object and use the
eval method available in the
net_actions member variable.
This can be useful if you want to process the output of our networks in memory and not incur additional disk I/O.
import netdef_slim as nd nd.load_module('FlowNet3/css/controller.py') c = Controller() out = c.net_actions.eval(img0, img1) # out is an OrderedDict with the following structure #OrderedDict(['flow.fwd', np.array[...], 'occ.fwd', np.array[...], 'occ_soft.fwd', np.array[...], 'mb.fwd', np.array[...], 'mb_soft.fwd', np.array[...], ])
netdef_models is under the GNU General Public License v3.0