Open Illimitable Space System
OpenISS is a motion capture data aggregation and delivery framwork for VFX that has library instances abstracting various middleware and cameras for many application types.
Applications include AI+Art, performing arts, interactive film, facial animation, NUI applications using OpenGL, Processing, TensorFlow, Keras, and MARF.
The OpenISS framework provides a uniform abstraction layer over libfreenect, libfreenect2, librealsense, OpenNI2, NiTE2, NuiTrack, and ROS and has REST and SOAP web services. There are different front-end modules as well based on OpenGL, Processing, and web browsers (JS).
OpenISS is inspired by the development of ISSv1, ISSv2, ISSv3 and MultiCamTk++, but is being built from scratch using C/C++ and various wrappers and uses similar or same open-source libraries, middleware and toolkits for sensors and creative coding. Various wrappers are being developed. See background on the inspirational ISS below. OpenISS API is likewise for the first time made it possble to access Kinect 1 and 2 as well as OpenCV as REST and SOAP services for creative near-realtime online broadcasting. OpenISS is poised to be the core replacement for ISSv2's pipeline. It also serves as an educational tool for graduate and undergraduate students in computer vision, computation arts, pattern recognition, AI, machine learning, and game development. It is designed to be portable.
- OpenISS component development began in 2016 in C
- Expanded to C++ and Java since and ongoing
- Project Lead: Serguei A. Mokhov
- Jashanjot Singh: Gesture and Hand Tracking Framework, NiTE2/NuiTrack, and ROS support, SWIG support for Java/Processing
- Haotao (Eric) Lai: Person Re-identifcation, Green Screening, device abstracion (Kinects 1 and 2, RealSense D435), TensorFlow/Keras/Python abstractions; Kinect 2 web service
- Yiran (Bernie) Shen: Facial Recognition and data (landmarks and expressions) provider framework abstraction over OpenFace, OpenCV, including TensortFlow for VFX
- Konstantinos (Kosta) Psimoulis: Web Services lead (REST and SOAP), SimpleOpenNIRS/liblrealsense2 support
- Jonathan Llewellyn: actvity recognition, VFX pipeline, softbody linkage
- Yuhao Mao: Art and AI, Sound Visualization, Style Transfer
- Chao Wang: Style Transfer, Magenta support, motion-based audio support, finger tracking
- Zihao Song: Singnal Processing for Audio and Image Processing effects and MARF
- See here for more documentation.
Early C API and Lunux Builds
See detailed list here.
Updates for Linux and C in EL6 (CentOS 6.x), CSI230-101 Fall 2017 course students teams:
- Calum Phillips, Rosser Martinez, Matthew Roy
- Alex Rader, Cory Smith, Nicholas Robbins
Original build automation contributors for Linux in EL6 (CentOS 6.x), CSI230-101 Fall 2016 course students teams:
- Brian Baron, Colin Brady, Robert Gentile
- Gabriel Pereyra, Justin Mulkin, Duncan Carrol, Lucas Spiker
- We are working on our Docker images here
- Please use
- More documentation to follow
- Please use
- More documentation to follow
EL7 (RHEL, CentOS, Scientific Linux) are as of September 2018 default and preferred build, so the development effort focuses around this platform, but it is known to run on macOS and Ubuntu.
EL6 is now considered legacy.
Notice, EL6 requires a newer kernel for
proper USB3 and NVIDIA support. If you prefer
to install dependencies manually, you can follow
the scripts referenced here (
el6.sh) and repeat their
relevant steps one by one.
Install: yum install git git clone https://github.com/OpenISS/OpenISS.git git submodule update --init --recursive Navigate to OpenISS/src Run the command: make deps This will install all of the 3rd party dependencies Compile: cd OpenISS/src make Clean: Navigate to OpenISS/src Run the command: make clean Removes the installed dependencies
- OS X (mostly there)
- Windows (partially there for some components)
- Java (in progress)
- C++ (mostly there)
- Processing (in progress)
- Max (in progress)
- PureData (in progress)
- Python (started)
Background on the ISSv1, ISSv2, ISSv3
OpenISS’s core goals are to enable achieving something akin to the original ISS below in an open educational setting. Then, to build custom applications based on it, using OpenISS API as a core.
- Origins are in Dr. Miao Song’s thesis:
- SIGGRAPH and SIGGRAPH Asia
References and Acknowledgements
- TravisCI EL6 and EL7 builds on GitHub: Derek Weitzel, Building CentOS packages on Travis-CI, https://djw8605.github.io/2016/05/03/building-centos-packages-on-travisci/