Skip to content
🎥 A Java implementation of Kinect Fusion running on Tornado VM.
Java Python C Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
conf
doc/images
lib
opencl
scripts
src
.gitignore
AUTHORS.md
LICENSE
README.md
compile.sh
downloadDataSets.sh
pom.xml

README.md

KFusion-TornadoVM

A Java implementation of the Kinect Fusion application running on TornadoVM. It can run on existing datasets as well as in real-time with input frames from an attached RBG-d camera. Detailed explanation and performance evaluation can be found in the following publication.

Releases

  • KFusion-TornadoVM 0.1.0 - 11/02/2019: Initial release of Kinect Fusion on TornadoVM.

Output

If you enable the GUI while running KFusion you will see a real-time 3D space reconstruction similar to the image below:

KFusion GUI output

In addition, you will see output text with performance metrics across the frames that KFusion processes:

frame	acquisition	preprocessing	tracking	integration	raycasting	rendering	computation	total    	X          	Y          	Z         	tracked   	integrated
0	0.006214	0.003313	0.016029	0.027997	0.000000	0.001509	0.047339	0.055061	0.000000	0.000000	0.000000	0	1
1	0.000499	0.000389	0.002222	0.000608	0.000000	0.000000	0.003220	0.003719	0.000000	0.000000	0.000000	0	1
2	0.000409	0.000418	0.002108	0.000618	0.000000	0.000000	0.003144	0.003554	0.000000	0.000000	0.000000	0	1
3	0.000584	0.000431	0.003003	0.000593	0.000552	0.000000	0.004579	0.005163	0.000000	0.000000	0.000000	0	1
4	0.000665	0.000422	0.013228	0.000599	0.000369	0.000310	0.014618	0.015592	-0.004392	0.001433	0.000935	1	1
...
Summary: time=6.50, frames=882, FPS=135.79

How to start?

This implementation runs on TornadoVM to achieve GPU acceleration and real-time performance. Hence, you need to install Tornado following the instructions from Tornado-INSTALL

After you successfully build Tornado, then issue the following commands to install locally its JAR files:

$ cd path/to/tornado
$ ./tornadoLocalInstallMaven

Finally, you can install KFusion-TornadoVM by issuing the following commands:

# Setup:
export KFUSION_ROOT="${PWD}"
export PATH="${PATH}:${KFUSION_ROOT}/bin"
export JAVA_HOME=/path/to/graal/jdk1.8.0_131
export TORNADO_ROOT=/path/to/tornado
export PATH="${PATH}:${TORNADO_ROOT}/bin/bin/"
export TORNADO_SDK=${TORNADO_ROOT}/bin/sdk

## Compile and run KFusion-TornadoVM
$ mvn clean install -DskipTests

## Run KFusion-TornadoVM GUI 
$ kfusion kfusion.tornado.GUI

## Run Benchmarking mode
$ kfusion kfusion.tornado.Benchmark <config file>

How to get the datasets?

KFusion-Tornado uses the ICL-NUIM datasets.
We provide a script to automatically download and compose the video files in raw format.
Alternatively, when running the Java program (KFusion-TornadoVM) the first time, it will download the corresponding video raw-file and install it locally.

Option a) Automatically

The first time you run the application and if the raw file is not installed locally (~/.kfusion_tornado), then the program will ask you if you want to download it auotmatically:

$ kfusion kfusion.tornado.Benchmark conf/bm-traj3.settings 
KFussion Accelerated with Tornado
	: Reading configuration file: /home/juan/.kfusion_tornado/living_room_traj3_loop.raw
Data Set file does not exist. Do you want to download it automatically? (~2GB) 
Press [yes/no] (default: yes) : 

Option b) Manually

Run the downloadDataSets.sh <url> <fileName>, for example:

$ bash downloadDataSets.sh  http://www.doc.ic.ac.uk/~ahanda/living_room_traj2_loop.tgz living_room_traj2_loop.raw

Running KFusion-Tornado

KFusion can run in two modes receiving input from:

  1. RGB-d camera where you select the input source from the drop-down menu:
## Run KFusion-Tornado GUI 
$ kfusion kfusion.tornado.GUI
  1. Pre-defined datasets again through the GUI selection or:
## Run KFusion-Tornado GUI 
$ kfusion kfusion.tornado.Benchmark <config file>

In our examples, we use images from the ICL-NUIM dataset which will be downloaded automatically when issuing the following command:

$ kfusion kfusion.tornado.Benchmark conf/bm-traj2.settings 

Note:

  • Sample configuration files are under the conf/ directory.

Selected Publications

  • James Clarkson, Juan Fumero, Michalis Papadimitriou, Foivos S. Zakkak, Maria Xekalaki, Christos Kotselidis, Mikel Luján (The University of Manchester). Exploiting High-Performance Heterogeneous Hardware for Java Programs using Graal. Proceedings of the 15th International Conference on Managed Languages & Runtime. preprint

  • Sajad Saeedi, Bruno Bodin, Harry Wagstaff, Andy Nisbet, Luigi Nardi, John Mawer, Nicolas Melot, Oscar Palomar, Emanuele Vespa, Tom Spink, Cosmin Gorgovan, Andrew Webb, James Clarkson, Erik Tomusk, Thomas Debrunner, Kuba Kaszyk, Pablo Gonzalez-de-Aledo, Andrey Rodchenko, Graham Riley, Christos Kotselidis, Björn Franke, Michael FP O'Boyle, Andrew J Davison, Paul HJ Kelly, Mikel Luján, Steve Furber. Navigating the Landscape for Real-Time Localization and Mapping for Robotics and Virtual and Augmented Reality. In Proceedings of the IEEE, 2018.

  • C. Kotselidis, J. Clarkson, A. Rodchenko, A. Nisbet, J. Mawer, and M. Luján. Heterogeneous Managed Runtime Systems: A Computer Vision Case Study. In Proceedings of the 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, VEE ’17, link

Citation

Please use the following citation if you use Tornado in your work.

@inproceedings{Clarkson:2018:EHH:3237009.3237016,
 author = {Clarkson, James and Fumero, Juan and Papadimitriou, Michail and Zakkak, Foivos S. and Xekalaki, Maria and Kotselidis, Christos and Luj\'{a}n, Mikel},
 title = {{Exploiting High-performance Heterogeneous Hardware for Java Programs Using Graal}},
 booktitle = {Proceedings of the 15th International Conference on Managed Languages \& Runtimes},
 series = {ManLang '18},
 year = {2018},
 isbn = {978-1-4503-6424-9},
 location = {Linz, Austria},
 pages = {4:1--4:13},
 articleno = {4},
 numpages = {13},
 url = {http://doi.acm.org/10.1145/3237009.3237016},
 doi = {10.1145/3237009.3237016},
 acmid = {3237016},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {Java, graal, heterogeneous hardware, openCL, virtual machine},
} 

Acknowledgments

This work was initially supported by the EPSRC grants PAMELA EP/K008730/1 and AnyScale Apps EP/L000725/1, and now it is funded by the EU Horizon 2020 E2Data 780245 and the EU Horizon 2020 ACTiCLOUD 732366 grants.

Collaborations

For academic collaborations please contact Christos Kotselidis.

Users Mailing list

A mailing list is also available to discuss Tornado related issues:

tornado-support@googlegroups.com

Contributors

This work was originated by James Clarkson and it is currently maintained by:

License

The work is published under the Apache 2.0 license: License

You can’t perform that action at this time.