Copter: high altitude Non-GPS navigation using downward facing camera

[rmackay9]

This idea has appeared a few times over the years so here is an issue to consolidate the conversation about it.

The idea is to use a downward camera (probably mounted on a gimbal) to capture pictures of the terrain below at regular intervals and then compare these to set of images taken from known locations, find the best match and then extrapolate the vehicle's current position. This could then be used instead of the GPS to allow the vehicle to get safely home.

Some ideas:

• A "mutual information" algorithm has been suggested as a good way to compare the live images from the stored images.
• The zoom and heading of the two sets of images would be different so it might be necessary to crop and rotate the live images to better match the stored images.
• The onboard terrain database could be useful in figuring out how much to zoom/crop the live image
• The EKF normally requires position updates at 4hz (or higher) in order to maintain a good position and velocity estimate. If the images can't be captured at this rate it might be necessary to augment the position estimate with an optical flow based velocity estimate. It is possible that optical flow values could be calculated from the live video and the onboard terrain database.
• The position and velocity estimates could be fed into the EKF using the VISION_POSITION_ESTIMATE and VISION_SPEED_ESTIMATE messages and then GPS/Non-GPS transitions could be used to handle the switch over between GPS and pictures.

[Ryanf55]

Why is this tagged only to Copter? Couldn't it be used in Plane too?

[rmackay9]

@Ryanf55,

Yes, it could/should work for Plane as well. The initial focus is on copters because they need the feature more than planes. Planes with an airspeed sensor can already dead reckoning fairly accurately for a couple of minutes at least. For copters it's only 30 seconds or less.