Skip to content
Implementation of Weighted Voxel (Xie et al., ICIMCS 2018).
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
core
datasets
utils
.gitignore
LICENSE
README.md
config.py
requirements.txt
runner.py

README.md

Weighted-Voxel

This repository contains the source codes for the paper Weighted Voxel: a novel voxel representation for 3D reconstruction (Xie et al. 2018).

Abstract

3D reconstruction has been attracting increasing attention in the past few years. With the surge of deep neural networks, the performance of 3D reconstruction has been improved significantly. However, the voxel reconstructed by extant approaches usually contains lots of noise and leads to heavy computation. In this paper, we define a new voxel representation, named Weighted Voxel. It provides more abundant information, facilitating the subsequent learning and generalization steps. Unlike regular voxel which consists of zero-one, the proposed Weighted Voxel makes full use of the structure information of voxels. Experimental results demonstrate that Weighted Voxel not only performs better in reconstruction but also takes less time in training.

Cite this work

If you find this work useful in your research, please consider citing:

@inproceedings{xie2018weighted,
    title={Weighted Voxel: a novel voxel representation for 3D reconstruction},
    author={Xie, Haozhe and Yao, Hongxun and Sun, Xiaoshuai and Zhou, Shangchen and Tong, Xiaojun},
    booktitle={International Conference on Internet Multimedia Computing and Service {ICIMCS} 2018},
    year={2018},
    organization={ACM}
}

Project Page

The project page is available at https://haozhexie.com/project/weighted-voxel.

Overview

Overview of Weighted Voxels

Overview of Weighted Voxels

The generation of Weighted Voxels can be regarded as applying a convolutional kernel on regular voxels. The value of each voxel in the Weighted Voxel is weighted summed over voxel values of its immediate neighbors. More formally, the value in Weighted Voxel can be calculated as

Weighted Voxel Eqn

where upsilon_i_j_k_in_0_1 denotes the value in the regular voxel, and omega is set to 26. Specially, we define upsilon_i_j_k_eq_0 when i_eq_ne_1, j_eq_ne_1 or k_eq_ne_1.

Methods Illustration

Methods Illustration

The network architecture of 3D-R2N2 and Weighted Voxel. Both of them consist of an encoder, a 3D convolutional LSTM, and a decoder. In 3D-R2N2, the reconstructed voxels are composed of zeros and ones, while in Weighted Voxel, the voxel values are filled with integers.

Comparison of Reconstruction Results with 3D-R2N2

Reconstruction Samples

Reconstruction samples of (a) cars (b) cabinets (c) speakers (d) sofas on the ShapeNet testing dataset. The Weighted Voxel preserves more structural details of 3D objects

Prerequisites

Install pygpu

Please follow the instruction on the homepage of gpuarray.

Install Other Python Denpendencies

pip3 install -r requirements.txt

Create .theanorc

Please paste following lines to ~/.theanorc:

[cuda]
root = /opt/cuda    # Please change it with your CUDA installation path

[global]
device = cuda0      # Please change it with your GPU device ID
floatX = float32

Download Dataset

Use following commands to download the ShapeNet dataset:

cd /path/to/the/repository
mkdir -p datasets/ShapeNet
cd datasets/ShapeNet

wget http://cvgl.stanford.edu/data2/ShapeNetRendering.tgz
tar -xf ShapeNetRendering.tgz

wget http://cvgl.stanford.edu/data2/ShapeNetVox32.tgz
tar -xf ShapeNetVox32.tgz

Use following command to generate Weighted Voxel dataset:

cd /path/to/the/repository
python utils/binvox_weighting.py datasets/ShapeNet/ShapeNetVox32 datasets/ShapeNet/WeightedShapeNetVox32

Get Started

Use following command to train the neural network:

python3 runner.py 

Use following command to test the neural network:

python3 runner.py \
      --test \
      --weights output/weights.npy

The pretrained model can be downloaded from here (206 MB).

License

This project is open sourced under MIT license.

You can’t perform that action at this time.