johnny-five camera support

[josephfinlayson]

I have one of these (Miniature TTL Serial JPEG Camera with NTSC Video):

https://www.adafruit.com/products/1386

Is it possible to get these working with johnny-five?

[rwaldron]

Once there is Software Serial support in Firmata, we can revisit this :)

[divanvisagie]

@rwaldron What about an on hold label for these types of issues?

[rwaldron]

@divanvisagie go for it

[YesPapa]

Hello,

What about this issue? Is it possible to use the johnny-five library with a camera?

Thank you.

[rwaldron]

Johnny-Five itself does not support a generic camera, as it's out of scope for the framework. I recommend using node-opencv along with Johnny-Five to create robots with camera capabilities. I'm building a rover that uses an ffmpeg stream via websockets to the browser display: https://www.youtube.com/watch?v=xr7hjaZHKFU

[YesPapa]

Ok. Thank you.

[soundanalogous]

I have serious doubts that SoftwareSerial support in Firmata would support a camera. It would be better handled by wrapping the Adafruit library and then requesting the buffer from Firmata than trying to relay the buffer data through SoftwareSerial.

[tawalke]

This issue was a few years ago, I was curious if anything has changed on this front. I have the Grove Serial Camera Kit and the Grove Fingerprint Sensor. Both capture images and it would be great to have all my code using the J5 framework for these on multiple devices.

[dtex]

@rwaldron wrote:

Johnny-Five itself does not support a generic camera, as it's out of scope for the framework.

We probably should have closed this back then. Please don't take this as a rebuke, it's just a function of what Johnny-Five is intended to be. Rather than an all-things for everyone solution, johnny-five focuses on lower level devices (servos, motors, sensors, etc) and embraces the node/npm ecosystem for more complex and device or manufacturer specific support.

Johnny-Five is not particularly opinionated about how you write your code. It's just Javascript with a healthy helping of events and call backs so it will play nice with other libraries.

Rather than just closing the issue, I want to at least point you in the right direction. Are you using an Arduino Uno connected to your computer over USB or something more?

[tawalke]

Hi @dtex : Thanks for not closing the issue and your response. You are right it did kinda sound like a rebuke initially, but I get it. I had a deadline so I rewrote all the code completely over in C and took out the JS parts to finish quickly.

However, I would love to help people in my situation that would love to use Johnny-Five with specific sensors so I would LOVE to be pointed in the right direction. Is there a way for me to take my C code used for specific sensors and convert it to JavaScript for use in the Johnny-Five framework. I'm sure it would help others looking to use J5 for a holistic solution across different devices.

Please advise how I can and should go about this.

BTW: I was building a solution that needed sensors to work across 5 different types of devices and that's why this option was so attractive to me. The devices were was Raspberry Pi3, BeagleBone Black, custom Qualcomm, ESP8266-12E, and Intel device for legacy work.

[dtex]

I think the concern raised by @soundanalogous is the first thing that needs to be addressed. He's a smart cookie and knows firmata better than.. well.. anyone else on earth, so his comment warrants serious consideration.

In short, can softwareSerial in firmata even support the camera?

Serial support landed in firmata (for Arduino's and most Arduino compatible devices) a while back so we have everything we need to try it out and it sounds like you might be our motivated contributor.

Please advise how I can and should go about this.

So part of the battle is making sure you are sending all the appropriate commands to get the camera up and running, and that you are listening for the appropriate commands and knowing what to do with them. You've got all that handled in C, so it's just a matter of porting your code (no magic there).

Next would be making sure that you know how to send and receive serial data in Johnny-Five. The only class in J5 that has code for working with serial is GPS and I think this code block will be the most valuable to help you get up and running.

[dtex]

Oh, and I'm going to close this issue but not because it is going away. We've added a new wiki page that tracks all of our new feature, device, class and module requests in a single place. If anything, it should make it easier for new contributors to find issues to work on. We can also group issues with those feature requests so everyone can see the demand for those that are most wanted.

[tawalke]

Thanks!! Checking out the wiki page and the code now.

[developer239]

+1 😄

[developer239]

@rwaldron Could you please explain how to use opencv and arduino together? I have no experience with IOT so I am slowly trying to read as much information as possible for a project that I have in my mind.

I have experience with https://github.com/justadudewhohacks/opencv4nodejs though.

If I understand it correctly we only need to:

1. connect a camera to Arduino UNO R3
2. initialize johny-five
3. then use the camera with opencv:

const devicePort = 0;
const wCap = new cv.VideoCapture(devicePort);

Is this correct?