Skip to content
A Nuke client plug-in which connects to a Python server to allow Machine Learning inference in Nuke.
Branch: master
Clone or download
johannabar Speed up EXR file reading in preprocessing step
Use scanlines to directly read the crop of EXR images.
Latest commit 1228597 Jun 17, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Models Speed up EXR file reading in preprocessing step Jun 17, 2019
Plugins Add trainingTemplateTF model Jun 7, 2019
.gitignore Add trainingTemplateTF model Jun 7, 2019
CMakeLists.txt Global renaming from 'DL' to 'ML' May 7, 2019
INSTALL.md Correct a link typo May 27, 2019
LICENSE Clarifying licensing May 27, 2019
README.md Update links in ReadMe Jun 7, 2019

README.md

Python-based Machine Learning Frame Server for Nuke

This repository contains the client-server system enabling Machine Learning (ML) inference in Nuke. This work is split into two parts: a client Nuke plug-in Plugins/Client/ and the Python frame server Plugins/Server.

The following models are provided as examples:

  • blur: a simple gaussian blur operation
  • Mask-RCNN
  • trainingTemplateTF: a training template written in TensorFlow which enables simple image-to-image training. Instructions on how to use this template are found here.

Example of Nuke doing DensePose inference.

Introduction

The Machine Learning (ML) plug-in connects Nuke to a Python server to apply ML models to images. The plug-in works as follows:

  • The Nuke node can connect to a server given an ip address and port,
  • The Python server responds with the list of available Machine Learning (ML) models and options,
  • The Nuke node displays the models in an enumeration knob, from which the user can choose,
  • On every renderStripe call, the current image and model options are sent from the Nuke node to the server,
  • The server does an inference on the image using the chosen model/options. This inference can be an actual inference operation of a machine learning model, or just some other image processing code,
  • The resulting image is sent back to the Nuke node.

Installation

Please find installation instructions in INSTALL.md.

Known Issues

  1. The GPU can run out of memory when doing model inference. To run Mask-RCNN, it is necessary to have a GPU memory of at least 6GB.
  2. If you get the following error: "The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine." Please refer to issue#10 [Thanks to samhodge]

License

The source code is licensed under the Apache License, Version 2.0, found in LICENSE.

Contacts

This plug-in was initially created by Sebastian Lutz (https://v-sense.scss.tcd.ie/?profile=sebastian-lutz).

References

  • Mask R-CNN. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. IEEE International Conference on Computer Vision (ICCV), 2017.
  • DensePose: Dense Human Pose Estimation In The Wild. Riza Alp Güler, Natalia Neverova, Iasonas Kokkinos. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
You can’t perform that action at this time.