SLAM like concept

[Neon22]

paper - http://rpg.ifi.uzh.ch/docs/ICRA14_Forster.pdf
video - https://www.youtube.com/watch?v=2YnIMfw6bJY
code - https://github.com/uzh-rpg/rpg_svo

looks exceptional. Not sure if enough RAM resources but hey - worth a look... (IMHO)

[Neon22]

As a side note check out the P3P here - looks efficient
http://rpg.ifi.uzh.ch/software_datasets.html

I used smop to convert the matlab .m files to python.
worked OK but Toon stuff did not make it... YMMV

[AlexisTM]

It IS awesome, but it uses way too much RAM. This is the job of an Odroid XU4 to achieve 60fps. 2 Go RAM and SVO uses 2 cores (because the way it is built). The cores run @2Ghz.

I would propose another way to do it which is possible at least. AprilTags (Because those are AWESOME).

• https://github.com/ProjectArtemis/aprilslam

We can put AprilTags in the room and compute the 6D (Position/orientation) at any time by seeing at least one tag.

With the following configuration:

sensor.set_framesize(sensor.QVGA)
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.skip_frames()
sensor.set_auto_gain(False)  # must turn this off to prevent image washout...
sensor.set_auto_whitebal(False)  # must turn this off to prevent image washout...

I am running a code similar to this:

    sensor.set_windowing(0,0, 170,130)
    img = sensor.snapshot()
    tags = img.find_apriltags()
    allTagsInImage.append(TagDetected(tags, current))

    sensor.set_windowing(0,110, 170,130)
    img = sensor.snapshot()
    tags = img.find_apriltags()
    allTagsInImage.append(TagDetected(tags, current))

   sensor.set_windowing(150,0, 170,130)
    img = sensor.snapshot()
    tags = img.find_apriltags()
    allTagsInImage.append(TagDetected(tags, current))

    sensor.set_windowing(150,110, 170,130)
    img = sensor.snapshot()
    tags = img.find_apriltags()
    allTagsInImage.append(TagDetected(tags, current))

The result is a scan on the whole QVGA image instead of only the QQVGA.

Results

QQVGA unique scan:

Pros:

• See the whole image
• 12Hz (20Hz if you add a binary mask)
• Can see close tags without loosing it

Cons:

• Only 2 meter detection for a 36h11 tag (15cm measured from the black border)

Quad scan on QVGA

Pros:

• Up to 4 meter detection for a 36h11 tag (15cm measured from the black border). It linearly increase with the width of the tags.
• Higher precision

Cons:

• 3Hz (5Hz with a binary mask)
• Loose big tags if in the frontier of the images (If the tag is bigger than 10 pixels on the image, it can be unseen if on the center of the image)

Solutions:

• Crop a bit more the images to have more common space between images and do not look at 10 pixels at the borders as the tags you see in that area is less precise .

16 or 9 scans on VGA

Pros:

• Up to 5 meters detection for a 36h11 tag (15cm measured from the black border). It linearly increase with the width of the tags.
• Highest precision

Cons:

• So many places we can miss a too big tag.
• Low frequency (0.8 Hz for full scan and 1.2Hz if scanning the 9 images in the middle)
• The image is less sharp

[kwagyeman]

Cool, what's your goal with the detection range?

[kwagyeman]

Um, so, about the windowing you're doing. The AprilTags function accepts an ROI. You don't have to change the camera settings. Just adjust the ROI for the find_apriltags function on each call. You can then run it on a qvga grayscale image (not VGA because you will run out of RAM). Just make sure the ROI itself is not bigger than 160x120.

Um, also, if you want a longer range... maybe two cameras? One with a telephoto lens and one without?

[AlexisTM]

Oh, damn, I did not notice the ROI input on the find_apriltags function. Thanks

The aim is to detect tags from a copter to achieve precision landing.

Having two cameras could be an option but is unpractical. Also, for this application, not finding the tag is worse than having the best position information. That's why I am operating with the default lens, wide angle, and also why I want a longer detection range. Finally, a longer range also mean we can have a bigger offset on the position and still detecting the tag.

Note that with an image of 170x130, the find_apriltags do not crash.

[kwagyeman]

Okay, hmmm, well, if you use the 4d tag you see that from farther away. But, you're going to increase the error rate. That said, you can filter bad tags out by looking at the hamming value to see how many bits were wrong. Just don't accept tags with a hamming value bigger than 0.

[AlexisTM]

I tried but the tag 36h11 had almost the same result as the 16h5 (I only tried on the full QQVGA image and obtained 2.5m for detection from time to time and 2 meters for continuous tracking).

Also, I prefer to use the 36h11 for robustness. It is built on a drone, so a false positive is worse than a tag not detected.

I was also wondering, if I change the QVGA to VGA, do I have to skip frames? If I do, from the time I change from QVGA to VGA to enable a higher precision tracking, I could loose the tag.

[kwagyeman]

Um, so, 0.8 Hz sounds bad, but, once you detect a tag you can change your tracking algorithm to center on it. What's the speed of the system when looking for tags? I ask because having a high frame rate shouldn't be a concern if the system is moving slowly.

[AlexisTM]

The speed of the system is arround 1m/s, with angular velocities (a multicopter). It is slow though when landing and should be able to track the tag.

The implementation I am thinking of:

If we can change the resolution whithout skipping frames, I see 2 modes : SCANNING and TRACKING. Switch to tracking if we have the tag to track. Go to scanning if we loose it.

• SCANNING : QQVGA scanning + QVGA (quad scanning) (~2.4Hz) and we are sure not to miss a tag.
• TRACKING : Switch to VGA and window on the position of the tracked tag (12Hz).

If we cannot change the resolution without skipping frames, I would go for QVGA (or VGA and resizing the image on software for scanning).

[kwagyeman]

Changing the res will drop frames... that said, the camera is always dropping frames. So, the better question is, what's the maximum time you don't have a frame you'd be comfortable with? At max you'll have to wait 2-3 dropped frames while the res changes. At QVGA and under that's 3/120th of a second.

[kwagyeman]

VGA would have to wait 3/60 of a second.

[AlexisTM]

This could be fast enough! If we start the conversion from QVGA to VGA on the last image scanned, then the camera will be in VGA right after the find_apriltags function. Therefore, we will never be waiting for the VGA image to be ready.

If the tracked tag was not found, then switch back to QVGA and then wait 3/120 seconds.

As we got 3Hz on scanning the QVGA image, the time lost represent 75ms. 1.075 seconds instead of 1.000, then we loose 7.5%. Looks feasible.