Skip to content

ml4a-ofx + osc-modules
Compare
Choose a tag to compare
@genekogan genekogan released this 11 Jun 11:34
· 163 commits to master since this release
v0.1
516b4d0

Binaries for all applications found in ml4a-ofx including the apps folder and osc-modules. Please refer to the documentation in each app for individual instructions.

Installation

In order to complete installation, run the setup.sh script to download necessary large data files.

If you wish to use KinectOSC, you must also download the zip file openni.zip and place the folder inside ('openni') into the data folder.

Known issues

Apps using ofxDarknet (YoloLive, ReverseObjectSearchFast, and GoBot) require an NVIDIA graphics card and CUDA to function. Even with those, they are a bit unstable, and often cause segmentation faults on startup and crash. Proceed with caution with those... your results may vary!

The other apps, including the ones based on ofxCcv and ofxFaceTracker2 are more or less stable.

Apps includes

More documentation can be found in the individual READMEs. This release includes:

ml4a-ofx apps

  • AudioClassifier + AudioClassifier2 : Train a classifier on microphone audio recordings. AudioClassifier2 is w.i.p., will merge soon.
  • AudioTSNEViewer : View a collection of audio samples organized via t-SNE on extracted audio features. Works both on a collection (folder) of individual files, or a single audio file which can is segmented by onsets or beats.
  • ConvnetClassifier : Train an image classifier on a webcam feed.
  • ConvnetRegressor : Train an image regression on a webcam feed.
  • ConvnetViewer : View the activations of a trained convnet.
  • DoodleClassifier : Similar to convnet classifier, but optimized toward classifying images of hand-drawings.
  • FaceClassifier : Train a model and do real time classification of face poses.
  • FaceDTW : Perform facial gestures over time and classify them using Dynamic Time Warping.
  • FaceRegressor : Train a model and do real time regression of face poses.
  • Gobot : Automatically scrape Go board position from online-go.com and make next move recommendations.
  • ImageTSNEViewer : View a collection of images organized via t-SNE on extracted features from a convnet, where the feature extraction is done from a python script and the results are imported into the app.
  • ImageTSNELive : The same as ImageTSNEViewer (above), but handles the feature extraction and t-SNE assignment internally, and optionally allows for assignment to grid.
  • ReverseImageSearch : Search for most similar images in an image collection, given a query image. Analyze it first, then save the results for real-time browsing.
  • ReverseImageSearchFast : Fast search and retrieval of similar images in a database to a query image, webcam feed, video, or screengrabber.
  • ReverseObjectSearchFast : Fast search and retrieval of similar images in a database to multiple objects found in a query image, webcam feed, video, or screengrabber.
  • SimpleRegression : Boiled down example showing how to train a model and do real time regression with mouse position as the input.
  • YoloLive : Do real-time object detection from 9000 classes on a webcam feed, video, or screengrabber.

osc-modules

  • AbletonOSC : Receives data to control Ableton Live music parameters.
  • AudioUnitOSC : Receives data to control Audio Unit modules.
  • ConvnetOSC : Sends convolutional neural network-based feature vectors of a live webcam stream (using ofxCcv).
  • DarknetOSC : Sends convolutional neural network-based feature vectors of a live webcam stream (using ofxDarknet).
  • FaceTracker2OSC : Sends coordinates of face found in a webcam stream.
  • KeyboardOSC : Execute keystrokes via OSC commands.
  • KinectOSC : Sends skeleton data of a single tracked skeleton from a Kinect.
  • LeapMotionOSC : Sends finger point data detected by Leap Motion hand tracker.
Assets 4
Loading