Skip to content
openFrameworks Addon For Neuraltalk2
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
dependencies
src
.gitignore
LICENSE
Makefile
README.md
addons.make
config.make

README.md

ofxIPNeuraltalk2

NeuralTalk Day walk with Voice

NeuralTalk Day walk with Voice

NeuralTalk Night Walk With Voice

NeuralTalk Night Walk With Voice

openFrameworks, addon for IP Cameras using Neuralktalk2 for automatic captioning. This is using the ofxIpVideoGrabber addon's example code. We've just modified it to read captions from neuraltalk2 and display it.

If you haven't seen this video already, go check it out. It demenonstrates how deep learning can be used for automatic captioning based on what the webcam sees.

What we've done here is, instead of using a webcam attached directly to the computer, we're feeding neuraltalk2 with feeds from an IP camera. In addition to that we've added text to speech so it reads what the camera sees as well. Useful for a blind person interested in hearing what the camera sees.

This is a hack my friend and I worked on after seeing the original video demo of what neuraltalk2 can do.

We also modified neuraltalk2's eval.lua5 script and so it's possible to read the image feed from the IP camera.

Dependencies

  1. Neuraltalk2 2
  2. Our modified version of eval.lua 5
  3. sudo apt-get install libttspico0 libttspico-utils libttspico-data
  4. Ubuntu
  5. voice.sh6

Contributors

linuxkay

You can’t perform that action at this time.