optical flow in opencv_apps

[pandemie]

Hi! First of all I'm quite new to ROS so I'm not sure whether this is a silly question.

What role do the opencv_apps play in this repository? Are those meant just as code examples and not meant to be used within 'proper' applications?

In short: is it advisable to use opencv_apps/Flow.msg and fback_flow_nodelet in my application or should I roll my own? The fact that fback_flow_nodelet unconditionally draws the flow visualization suggests that it's really meant more as an example application, rather than a general nodelet for optical flow.

---

If opencv_apps are meant to be useful apart from serving as a code example I would suggest a few changes (and gladly provide pull requests) to the optical flow part.

In particular I think that the definition of the flow messages should be different. In Flow.msg the flow vector is called velocity, but what fback_flow_nodelet and lk_flow_nodelet really compute are the motions between two frames in pixels irrespective of the time passed between the two frames. So either the nodelets need to scale the movement by the time between the frames, so that the velocity field in Flow.msg get a proper unit (px/s e.g.), or velocity needs to be renamed to displacement and a time field needs to be added to Flow.msg.

PS: great repo! 👍 😄

[vrabaud]

@k-okada created that package: it is an adaptation of the OpenCV samples for ROS.
I'll let him decide where he wants to take it.
I personally think it is good to improve those samples and make them more usable. We could envision a vision_msgs package.

[k-okada]

My initial motivation for this package is to used in 'proper' applications , so thanks you for your advice.

As for your time, I think both nodelet outputs FlowArrayStamped, and if you look at header.stamp of current and previous messages, you can get the time passed between the two frames. And I think usually optical flow filed uses same time duration for every flow vectors, so I think we need to adding time to each Flow.msg ? How do you think about add time to FlowArray or FlowArrayStamped message? Or do you think we should add time field to Flow msg?

Renaming from velocity to displacement may good idea.

[pandemie]

Thanks for the response.

I was not sure whether you can rely on no message drops, but now I realized that you can detect message drops by checking header.seq, so the time passed between the two frames should be enough.

Another suggestion is that there should be a difference between dense and sparse optical flow in the messages. Dense optical flow is in opencv an Image of type CV_32FC2. Perhaps there should be a DenseFlow ROS message with the appropriate sensor_msgs/Image? That would enable more efficient processing in algorithms that operate on dense flow. For the sparse flow I would keep the current definition. I'm not sure what the "ROS way" is here.

[vrabaud]

Issue got moved to ros-perception/opencv_apps#2