Skip to content

openFrameworks addon for visualizing and interfacing with pre-trained models in Caffe: Convolutional Architectures for Fast Feature Embedding. Requires Caffe, openFrameworks 64-bit, glog, hdf5, OpenCV, CUDA, pkmMatrix, and pkmHeatmap. Pre-trained models not included but can be found linked in Caffe's "Model Zoo" and placed in the bin/data direct…

Geekrick88/ofxCaffe

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
src
 
 
 
 
 
 
 
 
 
 
 
 
 
 

ofxCaffe

Interface for Caffe: Convolutional Architectures for Fast Feature Embedding from BVLC.

img0 img1 img2 img3

Current Models

  • VGG ILSVRC 2014 (16 Layers): 1000 Object Categories
  • VGG ILSVRC 2014 (19 Layers): 1000 Object Categories
  • BVLC GoogLeNet: 1000 Object Categories
  • Region-CNN ILSVRC 2013: 200 Object Categories (Region proposals not yet implemented)
  • BVLC Reference CaffeNet: 1000 Object Categories
  • BVLC Reference CaffeNet (Fully Convolutional) 8×8: 1000 Object Categories
  • BVLC Reference CaffeNet (Fully Convolutional) 34×17: 1000 Object Categories
  • MIT Places-CNN Hybrid (Places + ImageNet): 971 Object Categories + 200 Scene Categories = 1171 Categories

Instructions

(Warning: these probably won't work and will require edits/your help)

  • Install Caffe and all dependencies (-lglog -lgflags -lprotobuf -lleveldb -lsnappy -llmdb -lboost_system -lhdf5_hl -lhdf5 -lm -lopencv_core -lopencv_highgui -lopencv_imgproc -lcblas)
  • Install openFrameworks 64-bit for osx I used this 1 year old branch which is about 1000 commits behind the master... I think the oF community is getting ready to release a stable 64-bit osx version??? Nick Hardeman's OSX 64-bit openFrameworks branch
  • clone this repo into of_directory/addons/ofxCaffe
  • clone ofxOpenCv2461 into of_directory/addons/ofxOpenCv2461 (could possibly be replaced by other opencv libraries, likely not the one that ships with openframeworks though; I've included OSX compiled OpenCV 2461 libraries in the addon/libs/opencv folder)
  • clone pkmMatrix into of_directory/../pkm/pkmMatrix (This is OSX only due to its depency on Accelerate.framework; provides vectorized operations; can be replaced with OpenCV; please submit pull request and I wiil merge...)
  • clone pkmHeatmap into of_directory/../pkm/pkmHeatmap (Converts grayscale images to RGB JET colormap using GPU)
  • Go to the Caffe Model Zoo and download all necessary .caffemodel files into the bin/data directory (could make a script for this...; I know this last step is ridiculous. You have to go into the code and see what the names are and I probably changed the names too... well some of them anyways. I will make a script if there is enough interest...)

Example Project: Visualization

  • '1': Toggle predicted label output
  • '2': Toggle layer parameters
  • '3': Toggle layer outputs
  • '4': Toggle probabilities graph
  • '[' / ']': Change the current layer visualized
  • '-' / '+': Change the current model
  • '0': Toggle webcamera image

Troubleshooting

  • First make sure you can run Caffe and all tests (make runall)
  • Check the Project.xcconfig defines and make sure they match up with where things should be (library files/source code)

To Do

  • Properly crop images and mirror them to produce batch images
  • R-CNN region proposals
  • Possibly other models can support region proposals and still be fast
  • Add Network in Network
  • Add Flickr style Fine Tuning
  • Alternatives to visualizing layers? Deconvnets? Projections?

About

openFrameworks addon for visualizing and interfacing with pre-trained models in Caffe: Convolutional Architectures for Fast Feature Embedding. Requires Caffe, openFrameworks 64-bit, glog, hdf5, OpenCV, CUDA, pkmMatrix, and pkmHeatmap. Pre-trained models not included but can be found linked in Caffe's "Model Zoo" and placed in the bin/data direct…

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages