Python-based Machine Learning Frame Server for Nuke

This repository contains the client-server system enabling Machine Learning (ML) inference in Nuke. This work is split into two parts: a client Nuke plug-in Plugins/Client/ and the Python frame server Plugins/Server.

The following models are provided as examples:

blur: a simple gaussian blur operation
Mask-RCNN
trainingTemplateTF: a training template written in TensorFlow which enables simple image-to-image training. Instructions on how to use this template are found here.

Example of Nuke doing DensePose inference.

Introduction

The Machine Learning (ML) plug-in connects Nuke to a Python server to apply ML models to images. The plug-in works as follows:

The Nuke node can connect to a server given an ip address and port,
The Python server responds with the list of available Machine Learning (ML) models and options,
The Nuke node displays the models in an enumeration knob, from which the user can choose,
On every renderStripe call, the current image and model options are sent from the Nuke node to the server,
The server does an inference on the image using the chosen model/options. This inference can be an actual inference operation of a machine learning model, or just some other image processing code,
The resulting image is sent back to the Nuke node.

Installation

Please find installation instructions in INSTALL.md.

Known Issues

The GPU can run out of memory when doing model inference. To run Mask-RCNN, it is necessary to have a GPU memory of at least 6GB.
If you get the following error: "The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine." Please refer to issue#10 [Thanks to samhodge]

License

The source code is licensed under the Apache License, Version 2.0, found in LICENSE.

Contacts

Johanna Barbier (Johanna.Barbier@foundry.com)
Dan Ring (Dan.Ring@foundry.com)

This plug-in was initially created by Sebastian Lutz (https://v-sense.scss.tcd.ie/?profile=sebastian-lutz).

References

Mask R-CNN. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. IEEE International Conference on Computer Vision (ICCV), 2017.
DensePose: Dense Human Pose Estimation In The Wild. Riza Alp Güler, Natalia Neverova, Iasonas Kokkinos. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

License

jhamot/nuke-ML-server

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages