Skip to content

ml4a-ofx + osc-modules

Latest
Latest
Compare
Choose a tag to compare
@genekogan genekogan released this 06 Aug 04:35
· 15 commits to master since this release
v1.1
88e8d2c

Binaries for all applications found in ml4a-ofx including the apps folder and osc-modules. Please refer to the documentation in each app for individual instructions.

Installation

In order to complete installation, run the setup script to download necessary large data files. If you're on OSX 10.12+, this script also disables Mac's translocation of the apps so they can access the data folder.

sh setup.sh

If you wish to use KinectOSC, you must also download the zip file openni.zip and place the folder openni inside the data folder.

Apps

More documentation can be found in the individual READMEs. This release includes:

ml4a-ofx apps

  • AudioClassifier : Train a classifier on microphone audio recordings. AudioClassifier2 is w.i.p., will merge soon.
  • AudioTSNEViewer : View a collection of audio samples organized via t-SNE on extracted audio features. Works both on a collection (folder) of individual files, or a single audio file which can is segmented by onsets or beats.
  • ConvnetClassifier : (deprecated in favor of ConvnetPredictor) Train an image classifier on a webcam feed.
  • ConvnetRegressor : (deprecated in favor of ConvnetPredictor) Train an image regression on a webcam feed.
  • ConvnetViewer : View the activations of a trained convnet.
  • ConvnetPredictor : A combination of ConvnetRegressor and ConvnetClassifier.
  • DoodleClassifier : Similar to convnet classifier, but optimized toward classifying images of hand-drawings.
  • FaceClassifier : (deprecated in favor of FacePredictor) Train a model and do real time classification of face poses.
  • FaceDTW : Perform facial gestures over time and classify them using Dynamic Time Warping.
  • FaceRegressor : (deprecated in favor of FacePredictor) Train a model and do real time regression of face poses.
  • FacePredictor : A combination of FaceClassifier and FaceRegressor.
  • ImageTSNEViewer : View a collection of images organized via t-SNE on extracted features from a convnet, where the feature extraction is done from a python script and the results are imported into the app.
  • ImageTSNELive : The same as ImageTSNEViewer (above), but handles the feature extraction and t-SNE assignment internally, and optionally allows for assignment to grid.
  • ReverseImageSearch : Search for most similar images in an image collection, given a query image. Analyze it first, then save the results for real-time browsing.
  • ReverseImageSearchFast : Fast search and retrieval of similar images in a database to a query image, webcam feed, video, or screengrabber.
  • SimpleRegression : Boiled down example showing how to train a model and do real time regression with mouse position as the input.

osc-modules

  • AbletonOSC : Receives data to control Ableton Live music parameters.
  • AudioUnitOSC : Receives data to control Audio Unit modules.
  • ConvnetOSC : Sends convolutional neural network-based feature vectors of a live webcam stream (using ofxCcv).
  • FaceTracker2OSC : Sends coordinates of face found in a webcam stream.
  • KeyboardOSC : Execute keystrokes via OSC commands.
  • KinectOSC : Sends skeleton data of a single tracked skeleton from a Kinect.
  • LeapMotionOSC : Sends finger point data detected by Leap Motion hand tracker.

Troubleshooting

If when you open any of the applications which require external files, such as ConvnetOSC, ConvnetPredictor, or FaceOSC, and it throws you an error that it can't find the model file, make sure you've run setup.sh and that the files image-net-2012.sqlite3 and shape_predictor_68_face_landmarks.dat are found inside the data folder. If they still are giving you problems, try individually moving each app you want to use into the data folder and then moving it back out to its original location, then re-running. If you are still having problems, post an issue on the issue tracker.

Assets 4